Ivo Welch, October 2023

Fama-Macbeth X-Sect Method (1973)

Importance

This subject is so important that it is assigned multiple times. It is to be explained once by the outgoing to the incoming class to give a basic orientation and roadmap. This will be helpful over the summer and especially in Fall when life gets more serious. In Fall, I will briefly go over these slides again.

As part of the learning process, the first cohort will teach it to the second cohort in Spring. (I will be there to help, if needed.)

Explanation of FM Technique

  • Fama-Macbeth (FM) is the easiest correct way to do a quick check whether a signal is likely to work.

  • (One giant pooled regression is even easier, but should only be used at the exploratory phase, because its standard errors are incorrect.)

  • In ASAM, we run these regressions on a yearly basis. In the real world, F-M regressions are usually run on a monthly basis.

One Regression in Each Time Period

  • In real-world funds, this is usually done once a month — because they trade more often than we do.

  • Run one cross-sectional regression in one year.
    • Ignore all other years for the moment.
    • The regression inputs are (constructed from) the 2,000 or so stocks in your (ex-ante!) investing universe
    • Each stock has (lagged) signals and a (subsequent) stock return.
    • This one-year x-sect regression gives one coefficient per signal
    • x-sect means cross-sectional
  • Can you believe the OLS T-statistics on this one regression’s coefficients?
    • Absolutely not. The residuals are correlated, e.g., by industry. OLS requires iid.
    • ergo, you will ignore the T-stats completely
    • keep only the coefficients
    • (PS: not shown and known by few, each coefficient estimate is the RoR on a a particular min-var portfolio built based on the signal.)
    • Each regression tells you whether your signal associated with positive or negative rates of return in this one following year.
  • PS: It can be shown that the x-sect coefficient are also the rate of return on a zero-investment portfolio that invested according to the signal.

Combine Many Yearly One-Year Regressions

  • After you run the regressions in each year, you have a time-series of coefficients
    • each of which is also the yearly rate of return on a signal-based pfio.
  • Are the portfolio returns (coefficients) independent across years?
    • Most likely yes. Why?
  • Can you rely on T-stats/normality inference about the mean of the time-series of yearly rates of return and coefficients?
    • Yes. Unlike xs correlations, ts-correlations must be low. They are nearly uncorrelated
    • If not, you could get rich!

Why the Name FM?

  • This method is generally referred to as a Fama-Macbeth (FM) regression.
    • FM1973 did not invent all of it, but they did it much better.

  • Nowadays, FM means simply a two-stage procedure of ``pooled time-series coefficient averages from many cross-sections.’’

  • Despite its age, FM is not obsolete. Every quant fund in the world runs these.

  • Every halfway-competent quant knows this technique.

Beyond FM: Reflect on How Many Years Should Be In Line

  • How many years of performance data on a strategy do you want to use?
    • As many as possible?
    • What is still relevant? Should 1933 still count? 1973? 1993?
  • Even for the best of signals and strategies, how often do you think there will be positive abnormal performance?
    • Maybe 51-53 out of 100 years?
    • PS: The questions is indicative, but not fully insightful
      • you can build a bad E(r) strategy that wins 95 out of 100 years, too.
      • like doubling up on roulette
  • How many years of performance data on a strategy do you want to use?
    • As many as possible?
    • What is still relevant? Should 1933 still count? 1973? 1993?
  • Conceptual question for all historical data analysis

Combine Many Yearly One-Year Regressions

  • $\Longrightarrow$ after you run the regressions in each year, you have a time-series of coefficients
    • and each of which is also the yearly rate of return on a pfio.
  • Are the portfolio returns (coefficients) independent across years?
    • Most likely yes.
  • Can you rely on normality inference for the mean and T-stat of the time-series of yearly rates of return and coefficients?
    • Yes. They they are nearly uncorrelated
    • If not, you could get rich!

Why the Name FM?

  • This method is generally referred to as a Fama-Macbeth (1973) (FM) regression.
    • FM did not invent it, but they did it much better.

  • Nowadays, FM means simply ``pooled time-series coefficient averages from many cross-sections.’’

  • Despite its age, FM is not obsolete. Every quant fund in the world runs these.

  • Everyone halfway competent knows this technique.

A Stylized Example of FM

Example Data

  • Four Stocks, A through D
  • Seven Years, 1-7

  • One Signal (i.e., one signal specific to each stock every year): employee-growth.

  • Note: Signals could be anything known. For example,
    • The signal could be the market-cap, for example.
    • The signal could be the market-cap or earnings or market-beta, for example.
    • Q: Could the market rate of return be such a signal?
  • I will use R to illustrate my example. If you use and know only python, all the better—reimplement the example in python to confirm that you understand the example and that your answers are the same as mine.

# one signal, with realization for each of 4 stocks A-D, and 7 years
empgrowA <- c( 3,  4,  1, -3,  0,  2,  NA )
retA <- c( NA, -6, 12, -1, -22, 6,  -6 )/100.0

empgrowB <- c(-2, -2,  1,  1,  2,  4,  NA )
retB <- c( NA, -2,-34,  1,  13, 12, 36 )/100.0

empgrowC <- c(-1,  3, -4, -1,  4, -2,  NA )
retC <- c( NA, -7, 25,  2, -19, -6, -10)/100.0

empgrowD <- c( 0,  4, -2,  2, -1,  1,  NA )
retD <- c( NA, -2, 29, 11, -5, -30, 7  )/100.0

Let’s print this better

d <-  data.frame( time=1:length(empgrowA), empgrowA, retA, empgrowB, retB, empgrowC, retC, empgrowD, retD )
print( d )
year empgrowA retA empgrowB retB empgrowC retC empgrowD retD
1 3   -2   -1   0  
2 4 -0.06 -2 -0.02 3 -0.07 4 -0.02
3 1 0.12 1 -0.34 -4 0.25 -2 0.29
4 -3 -0.01 1 0.01 -1 0.02 2 0.11
5 0 -0.22 2 0.13 4 -0.19 -1 -0.05
6 2 0.06 4 0.12 -2 -0.06 1 -0.30
7   -0.06   0.36   -0.10   0.07

Known Signals

  • To be investable, the investing universe must be known before you form the portfolio.

  • To be investable, the signal must be known before you form the portfolio.

  • Example:
    • Do you know the empgrowA[t=2]=4 in time 2? Can you use it to short A to earn the retA[t=2]= -0.06 rate of return?
    • In this example, no. The signal is known only at the end of year 2.

  • You can only use sig[t] to invest at the end of time t to earn the rate of return at time t+1.

  • PS: (You can ignore the risk-free rate. Its effects on the slopes are always minimal.)

Step 1: Align Signals With Subsequent Returns

  • Align known signals with (subsequent) rates of returns in each year.

  • You want to regress the lagged signal on the rate of return
    • you could also say you want to regress the future rate of return on the current signal
  • Call the lagged signal lsig. This is how R aligns it properly:
d <- within(d, {
    lempgrowD <- c(NA, empgrowD[1:6])
    lempgrowC <- c(NA, empgrowC[1:6])
    lempgrowB <- c(NA, empgrowB[1:6])
    lempgrowA <- c(NA, empgrowA[1:6])
})
d <- subset(d, TRUE, select=c(lempgrowA,retA,lempgrowB,retB,lempgrowC,retC,lempgrowD,retD))
print( d )
lgempgrowA retA lgempgrowB retB lgempgrowC retC lgempgrowD retD
3 -0.06 -2 -0.02 -1 -0.07 0 -0.02
4 0.12 -2 -0.34 3 0.25 4 0.29
1 -0.01 1 0.01 -4 0.02 -2 0.11
-3 -0.22 1 0.13 -1 -0.19 2 -0.05
0 0.06 2 0.12 4 -0.06 -1 -0.30
2 -0.06 4 0.36 -2 -0.10 1 0.07
  • Make sure you understand this new data alignment. Look back at the main table.

Step 2: Regress Signals on Subsequent Returns in One Year

  • Now regress the subsequent rate of return on the known signal in x-section.

  • Here, the first observation are the rates of return at time 2, explained by signals from time 1.

  • the dependent variable is the rate of return on each stock:

y <- subset( d[2,], TRUE, select=c(retA, retB, retC, retD) )
print( y )
year retA retB retC retD
2 -0.06 -0.02 -0.07 -0.02
  • the independent variable is the lagged signal on each stock:
x <- subset(d[2,], TRUE, c(lempgrowA,lempgrowB,lempgrowC,lempgrowD))  ## the lagged signal!
print( x )
year lgempgrowA lgempgrowB lgempgrowC lgempgrowD
2 3 -2 -1 0
  • Run one x-sect regression, explaining y with x. For time 2, this gives:
print( coef( lm( as.numeric(y) ~ as.numeric(x)) ) )
(Intercept) as.numeric(x)
-0.0425 -0.0050

  • Note that the regression automatically included an intercept

    • We will ignore this intercept. Consider it useless.

    • The second coefficient, the slope, tells us whether a higher (lagged) signal meant, on average, a higher average rate of return in this one year.

Step 3: Run an Equivalent Regression in Every Year

  • Let’s repeat such x-sect regressions, one in each year:
coefs <- c()
for (t in 2:7) {
  y <- with(d[t,], c(retA,retB,retC,retD ))
  x <- with(d[t,], c(lempgrowA,lempgrowB,lempgrowC,lempgrowD))

  coefs <- rbind(coefs, coef( lm( y ~ x ) ))
  options(digits=3)
  cat("\nThe two coefficients explaining returns at time ", t, " are:\n")
  print( last(coefs) )
}

coefs <- as.data.frame(coefs)
names(coefs) <- c("a.notused", "b.sigeffect")
p(coefs)
a.notused b.sigeffect
-0.043 -0.005
-0.134 0.095
0.023 -0.009
-0.069 0.054
-0.089 0.036
-0.016 0.067
  • Because the regressions include an intercept, R calls the coefficient on signal A coefs[2] and not coefs[1]

Step 4: Run a T-Test On Coefficients’ Time-Series

  • The average slope coefficient (b.sigeffect) measures how signals influence subsequent rates of return, on average.

  • You can run a T-test to see whether this effect has a mean of zero:

mt <- function(x) c(mean=mean(x), sd=sd(x), T=mean(x)/sd(x)*sqrt(length(x)-1))

mt(coefs[,2])  ## Effect
  x
mean 0.039
sd 0.041
T 2.152

  • Conclude: Here, stocks with more signals had higher average rates of return, almost statistically significantly so.

  • Q: What does the regression coefficient of 0.0395 suggest?
    • for example, compare a stock with a signal of 0 vs one with a signal of 3.
    • how would their rates of return be expected to differ?
  • You could include several signals at the same time

FM Implementation Details and Presentation

  • Sigmas could be non-linear, dummies, etc.

  • FM does not give you the weights of the investment strategy. For this, we will use the BJSFF techniques later.

Important Advice

  • Don’t be sloppy explaining what you are doing.
    • Tell sample (types and time periods)
    • Describe exact alignment
    • Tell selection criteria and throw-out criteria
    • Tell signals
    • Tell returns

  • Specify exactly what your universe / data selection requirements are.

  • For example, here is a set of useful criteria. To be included, a stock had to
    • be among the 2,000 largest-lagged-marketcap stocks (good for ASAM),
    • with $>200$ days of valid returns over the calendar year preceding the investment (good for ASAM),
    • share codes 10 or 11 (always a good idea),
    • a marketcap of at least $100 million (good for ASAM),
    • and have a price above $1 at the end of the preceding period (good idea).
  • Be very careful with ex-post data requirements. Ideally none.
    • Super-Bad: have a stock price above $1 at the end of the period.

  • Specify exactly what the signal (the independent variable in the regs) is.

  • For example,
    • An indicator that takes 1 if
      • the lag marketcap is at least $500 million,
      • the firm is in the S\&P100 index,
      • and the lag-book-to-market ratio of the firm is among the top 50 stocks.
      • and the twice-laged-book-to-market ratio of the firm is among the top 100 stocks.
      • Otherwise, 0.
    • Note that even simple signals can have many variations. For example,
      • Not an indicator, but the lag-book-to-market-ratio itself
      • 0 if the firm is not in the S\&P100.
    • Note that each signal-function variation is a different strategy!
    • You can try many variants, but don’t go crazy or you will just overfit (=torture) the data.

  • Again, be precise. Do not forget that the inclusion criteria and signal must be known by the time you plan to invest. It is easy to get miraculous results when you make a mistake here.

  • In your description, show the signal inputs for a few sample stocks, together with what subsequent stock returns these signal inputs are actually predicting in your FM reg.

Are your F-M Inferences Robust? Still Relevant?

  • Print and plot the two (or more) FM coefficients for each year for about 40 years worth of coefficients. Do not forget to tell us the N and the $R^2$ of each regression.
    • You can usually ignore the intercept. (It has nothing to do with the finance ``alpha.’’)
    • Your prime interest is the (average) coefficient on your signal across all 40 annual coefficients.

  • Do not run regression just among stocks who have signal, but among all stocks.

  • Perhaps give an earnings or dividend-yield example

  • Fama-Macbeth regressions are sometimes run with log returns. There are good reasons why one should do so, and good reasons why one should not do so. Log-returns are not real returns.

ASAM Considerations

Analysis Time Units

  • For ASAM, we can work with yearly rates of return, which is much more convenient.

  • This is because ASAM mostly wants to avoid active trading.

  • Recall first assignments:

    • Create an annual stock return data set first.

    • Create a second data set, where you exclude the first week in January (because the first Jan week is often strange!)

/teaching/asam/02-asam-famamacbeth.html Last modified: