Frisch–Waugh–Lovell theorem

In econometrics, the Frisch–Waugh–Lovell (FWL) theorem is named after the econometricians Ragnar Frisch, Frederick V. Waugh, and Michael C. Lovell.^[1][2][3]

The Frisch–Waugh–Lovell theorem states that if the regression we are concerned with is:

${\displaystyle Y=X_{1}\beta _{1}+X_{2}\beta _{2}+u}$

where ${\displaystyle X_{1}}$ and ${\displaystyle X_{2}}$ are ${\displaystyle n\times k_{1}}$ and ${\displaystyle n\times k_{2}}$ matrices respectively and where ${\displaystyle \beta _{1}}$ and ${\displaystyle \beta _{2}}$ are conformable, then the estimate of ${\displaystyle \beta _{2}}$ will be the same as the estimate of it from a modified regression of the form:

${\displaystyle M_{X_{1}}Y=M_{X_{1}}X_{2}\beta _{2}+M_{X_{1}}u,}$

where ${\displaystyle M_{X_{1}}}$ projects onto the orthogonal complement of the image of the projection matrix ${\displaystyle X_{1}(X_{1}^{\mathsf {T}}X_{1})^{-1}X_{1}^{\mathsf {T}}}$. Equivalently, M_X1 projects onto the orthogonal complement of the column space of X₁. Specifically,

${\displaystyle M_{X_{1}}=I-X_{1}(X_{1}^{\mathsf {T}}X_{1})^{-1}X_{1}^{\mathsf {T}},}$

and this particular orthogonal projection matrix is known as the residual maker matrix or annihilator matrix.^[4][5]

The vector ${\textstyle M_{X_{1}}Y}$ is the vector of residuals from regression of ${\textstyle Y}$ on the columns of ${\textstyle X_{1}}$.

The most relevant consequence of the theorem is that the parameters in ${\textstyle \beta _{2}}$ do not apply to ${\textstyle X_{2}}$ but to ${\textstyle M_{X_{1}}X_{2}}$, that is: the part of ${\textstyle X_{2}}$ uncorrelated with ${\textstyle X_{1}}$. This is the basis for understanding the contribution of each single variable to a multivariate regression (see, for instance, Ch. 13 in ^[6]).

The theorem also implies that the secondary regression used for obtaining ${\displaystyle M_{X_{1}}}$ is unnecessary when the predictor variables are uncorrelated (this never happens in practice): using projection matrices to make the explanatory variables orthogonal to each other will lead to the same results as running the regression with all non-orthogonal explanators included.

It is not clear who did prove this theorem first. However, in the context of linear regression, it was known well before Frisch and Waugh paper. In fact, it can be found as section 9, pag.184, in the detailed analysis of partial regressions by George Udny Yule published in 1907.^[7] In this paper, Yule stresses the central role of the result in understanding the meaning of multiple and partial regression and correlation coefficients. See the first paragraph of section 10 on pag. 184 of Yule's 1907 paper.

Yule's results were generally known by 1933 as Yule did include a detailed discussion of partial correlation, his novel partial correlation notation introduced in 1907 and the "Frisch, Waugh and Lovell" theorem, as chapter 10 of his, quite successful, Statistics textbook first issued in 1911 which, by 1932, had reached its tenth edition.^[8]

Frisch did quote Yule's results on pag. 389 of a 1931 paper with Mudgett.^[9] In this paper Yule's formulas for partial regressions are quoted, and explicitly attributed to Yule, in order to correct misquotes of the same formulas by another Author. In fact, while Yule is not explicitly mentioned in their 1933 paper, Frisch and Waugh use, for the partial regression coefficients, the notation first introduced by Yule in his 1907 paper and in general use by 1933.

References

1. Frisch, Ragnar; Waugh, Frederick V. (1933). "Partial Time Regressions as Compared with Individual Trends". Econometrica. 1 (4): 387–401. JSTOR 1907330.
2. Lovell, M. (1963). "Seasonal Adjustment of Economic Time Series and Multiple Regression Analysis". Journal of the American Statistical Association. 58 (304): 993–1010. doi:10.1080/01621459.1963.10480682.
3. Lovell, M. (2008). "A Simple Proof of the FWL Theorem". Journal of Economic Education. 39 (1): 88–91. doi:10.3200/JECE.39.1.88-91.
4. Hayashi, Fumio (2000). Econometrics. Princeton: Princeton University Press. pp. 18–19. ISBN 0-691-01018-8.
5. Davidson, James (2000). Econometric Theory. Malden: Blackwell. p. 7. ISBN 0-631-21584-0.
6. Mosteller, F.; Tukey, J. W. (1977). Data Analysis and Regression a Second Course in Statistics. Addison-Wesley.
7. Yule, George Udny (1907). "On the Theory of Correlation for any Number of Variables, Treated by a New System of Notation". Proceedings of the Royal Society A. 79: 182–193.
8. Yule, George Udny (1932). An Introduction to the Theory of Statistics 10th edition. London: Charles Griffin &Co.
9. Frisch, Ragnar; Mudgett, B. D. (1931). "Statistical Correlation and the Theory of Cluster Types". Journal of the American Statistical Association. 21(176): 375–392.

Further reading

• Davidson, Russell; MacKinnon, James G. (1993). Estimation and Inference in Econometrics. New York: Oxford University Press. pp. 19–24. ISBN 0-19-506011-3.
• Davidson, Russell; MacKinnon, James G. (2004). Econometric Theory and Methods. New York: Oxford University Press. pp. 62–75. ISBN 0-19-512372-7.
• Hastie, Trevor; Tibshirani, Robert; Friedman, Jerome (2017). "Multiple Regression from Simple Univariate Regression" (PDF). The Elements of Statistical Learning : Data Mining, Inference, and Prediction (2nd ed.). New York: Springer. pp. 52–55. ISBN 978-0-387-84857-0.
• Ruud, P. A. (2000). An Introduction to Classical Econometric Theory. New York: Oxford University Press. pp. 54–60. ISBN 0-19-511164-8.
• Stachurski, John (2016). A Primer in Econometric Theory. MIT Press. pp. 311–314.