# Why isn't the “annihilator” matrix a zero matrix?

I am struggling to understand why M is not null since: $$\mathbf M=I−X(X′X){^-}^1X′=I−XX{^-}^1X'{^-}^1X′=I-I=[0]$$ What's wrong with that reasonning?

• Because $X$ need not be square matrix, and so $(X'X)^{-1}\ne X^{-1}X'^{-1}$. You can't invert a non-square matrix. – Herr K. Mar 25 '16 at 18:53
• @Herr K. Thanks! Of course the number of parameters is not always the number of observations! I do not know how to mark your comment as the answer though... – jeake Mar 25 '16 at 18:58
• You can't. People don't get points for good comments. They are just there to be helpful in simple cases that can be addressed quickly. Just give him a nice smile like this :) It's etiquette ;^))) – Kitsune Cavalry Mar 26 '16 at 8:41
• @HerrK It is short, but you could write it up as an answer. (You have my vote...) – Giskard Apr 24 '16 at 20:43

The inverse of some matrix $X$, $X^{-1}$, is defined for square matrices only (i.e. when $X$ has the same number of rows and columns). In the typical econometric applications, the data matrix $X$ usually has far more rows (observations) than columns (regressors). Formally speaking, the matrix $X$ in your definition of $M$ has dimension $n\times k$ but $n\ne k$ (actually $n\gg k$), and so it's not a square matrix.
Therefore the second equality in your derivation is not valid; in particular, $(X'X)^{-1}\ne X^{-1}(X')^{-1}$, because $X^{-1}$ and $(X')^{-1}$ are not defined.
Let $X$ be an $n \times k$ matrix with linearly independent columns. Let $S \equiv \text{span}(X)$.
Regression can be thought of as the following problem: Given an n-dimensional vector $y \in \mathcal{R}^n$, find the vector in $\hat{y} \in S$ that is closest to $y$ -- i.e. $$\hat{y} = \arg\min_{z \in \mathcal{R}^n} ||y - z||$$ It can be shown that the solution to this problem is given by $$\hat{y} = Py = X (X' X)^{-1} X' y$$ where we refer to the matrix $P = X (X' X)^{-1} X'$ as the "projector matrix." Then the matrix $M = (I - P)$ allows you to get the residuals of your regression by $\hat{u} = (I - P)y = y - \hat{y}$. As you said, if $X$ is square then $P$ reduces to the identity and the residuals are 0. This is insightful for several reasons.
• Geometrically, if $X$ is square and has $n$ linearly independent columns then $\text{span}(X)$ is all of $\mathcal{R}^n$. Then the projection problem reduces to picking the vector in $\hat{y} \in \mathcal{R}^n$ that is closest to some vector in $y \in \mathcal{R}^n$. Obviously, if your vector is coming from the span of the set, the minimum distance choice would be the vector itself. We see this because as you showed $P = I$ in this case. Then $M = (I - P) = 0$ which means you have 0 residuals and you have a perfect fit of your data.