# how to use projection matrices in econometrics

We have the below expression: $$P_{M_{1}.X_{2}} \cdot y$$ I guess its interpretation is that they are the fitted values obtained from the regression of $$y$$ on residuals that are obtained from the regression of $$X_2$$ on $$X_1$$.

But what do we really mean by this? what is the thing we do actually with that regressions?

and How we can show that $$P_x - P_1 = P_{M_{1}.X_{2}}$$

• What does $x$ denote in $P_x$? And what is $P_1$? Is your model linear? Please give more details... Nov 30, 2022 at 21:26
• $P_x$ vector of fitted values from the regression of $y = X_1 \cdot \beta_1 +X_2 \cdot \beta_2 + u$ and $P_1$ is the one for $y = X_1 \cdot \beta_1 + u_1$ Dec 1, 2022 at 6:45
• @Tatanik501 Actually $P$ is the projection matrix which is a special case of a projection operator in the context of a finite dimensional subspace. Dec 9, 2022 at 8:43

As the above comment pointed out, it is always good to provide exact definitions when asking a mathematical question.

But since your notation (or the notation in the book/lecture notes you are reading) is fairly standard, I'll fill in the definition from my interpretation and then give an answer.

In a generic multivariate regression $$y = X \beta + e$$, we use $$P_X$$ to denote the projection matrix $$P_X := X (X'X) ^{-1} X'$$ and use $$M_X := I - P_X$$ to denote the annihilator matrix.

If we partition the big $$X$$ matrix into two parts as in $$X = (X_1, X_2)$$ and partition the big $$\beta$$ vector into $$\beta = (\beta_1', \beta_2')'$$ in the compatible way, then we can write $$y = X_1 \beta_1 + X_2 \beta_2 + e$$.

Theorem. The result we want to prove is that the projection matrix onto $$X$$ can be broken into two (orthogonal) parts: the projection matrix onto $$X_1$$ and the projection matrix onto $$\tilde{X_2}$$, where $$\tilde{X_2} := M_{X_1} X_2 = (I - P_{X_1}) X_2$$ is the residual from the projection of $${X_2}$$ onto $${X_1}$$.

Algebraically, what we want to show is $$P_X = P_{X_1} + P_{\tilde{X_2}}$$.

Before we prove this, we note that this result is a type of the so-called Frisch-Waugh-Lovell theorem in econometrics (https://en.wikipedia.org/wiki/Frisch%E2%80%93Waugh%E2%80%93Lovell_theorem).

Proof. Using $$X = (X_1, X_2)$$, we expand $$P_X = X (X'X) ^{-1} X' = (X_1, X_2) \left[\begin{pmatrix} X_1' \\ X_2' \end{pmatrix} (X_1, X_2)\right]^{-1} \begin{pmatrix} X_1' \\ X_2' \end{pmatrix} = (X_1, X_2) \begin{pmatrix} X_1'X_1 & X_1'X_2 \\ X_2'X_1 & X_2'X_2 \end{pmatrix}^{-1} \begin{pmatrix} X_1' \\ X_2' \end{pmatrix}.$$

We use the simplified notation $$A := \begin{pmatrix} A_{11} & A_{12} \\ A_{21} & A_{22} \end{pmatrix} := \begin{pmatrix} X_1'X_1 & X_1'X_2 \\ X_2'X_1 & X_2'X_2 \end{pmatrix}$$ and also use $$A^{-1} := \begin{pmatrix} A^{11} & A^{12} \\ A^{21} & A^{22} \end{pmatrix}$$ to denote the inverse of $$A$$.

Now, using bock matrix inverse formula (https://en.wikipedia.org/wiki/Block_matrix), we can derive \begin{align} A^{22} & = A_{22.1}^{-1} := (A_{22} - A_{21} A_{11}^{-1} A_{12})^{-1} \\ & =(X_2'X_2 - X_2'X_1 (X_1'X_1)^{-1} X_1'X_2)^{-1} \\ & =(X_2' M_{X_1} X_2)^{-1} \\ & =(\tilde{X_2}' \tilde{X_2})^{-1},\\ & \\ A^{21} & = - A^{22} A_{21} A_{11}^{-1} \\ & = - (\tilde{X_2}' \tilde{X_2})^{-1} X_2'X_1 (X_1'X_1)^{-1},\\ & \\ A^{12} & = A^{21\prime}, \\ & \\ A^{11} & = A_{11}^{-1} + A_{11}^{-1} A_{12} A^{22} A_{21} A_{11}^{-1} \\ & = (X_1'X_1)^{-1} + (X_1'X_1)^{-1} X_1'X_2 (\tilde{X_2}' \tilde{X_2})^{-1} X_2'X_1 (X_1'X_1)^{-1}. \end{align}

With these formulas, we go back to $$P_X$$ to derive \begin{align} P_X & = (X_1, X_2) \begin{pmatrix} X_1'X_1 & X_1'X_2 \\ X_2'X_1 & X_2'X_2 \end{pmatrix}^{-1} \begin{pmatrix} X_1' \\ X_2' \end{pmatrix} \\ & = (X_1, X_2) A^{-1} \begin{pmatrix} X_1' \\ X_2' \end{pmatrix} \\ & = (X_1, X_2) \begin{pmatrix} A^{11} & A^{12} \\ A^{21} & A^{22} \end{pmatrix} \begin{pmatrix} X_1' \\ X_2' \end{pmatrix} \\ & = X_1 A^{11} X_1' + X_1 A^{12} X_2' + X_2 A^{21} X_1' + X_2 A^{22} X_2' \\ & = X_1 (X_1'X_1)^{-1} X_1' + X_1 (X_1'X_1)^{-1} X_1'X_2 (\tilde{X_2}' \tilde{X_2})^{-1} X_2'X_1 (X_1'X_1)^{-1} X_1' \\ & - X_1 (X_1'X_1)^{-1} X_1'X_2 (\tilde{X_2}' \tilde{X_2})^{-1} X_2' \\ & - X_2 (\tilde{X_2}' \tilde{X_2})^{-1} X_2'X_1 (X_1'X_1)^{-1} X_1' \\ & + X_2 (\tilde{X_2}' \tilde{X_2})^{-1} X_2' \\ & = P_{X_1} + P_{X_1} X_2 (\tilde{X_2}' \tilde{X_2})^{-1} X_2' P_{X_1} \\ & - P_{X_1} X_2 (\tilde{X_2}' \tilde{X_2})^{-1} X_2' - X_2 (\tilde{X_2}' \tilde{X_2})^{-1} X_2' P_{X_1} + X_2 (\tilde{X_2}' \tilde{X_2})^{-1} X_2' \\ & = P_{X_1} + (X_2 - P_{X_1} X_2) (\tilde{X_2}' \tilde{X_2})^{-1} (X_2' - X_2' P_{X_1}) \\ & = P_{X_1} + \tilde{X_2} (\tilde{X_2}' \tilde{X_2})^{-1} \tilde{X_2}' \\ & = P_{X_1} + P_{\tilde{X_2}}. \\ \end{align} Q.E.D.

P.S. Let me know if there is any typo. This is a lot of typing. :)

• Thanks a lot for your detailed answer :) it really helps me to see full algebraical expression to understand what we really do. I tried to do something similar but I was lost. I appreciate your time. Dec 10, 2022 at 9:18