# Covariance between regression coefficient and residual

Given regression equation Y=Xβ + ε which satisfies all classical assumptions including those of homoscadasticity

Estimated regression equation using OLS is the following -

Y=Xb + e

Where, b is the OLS estimator for β and e is the estimated residual term for equation where E(e|x) =0

Note: b=(X'X)$$^{-1}$$X'Y

We can find, Cov(b,e|x) = E(b.e|x) - E(b).E(e|x)

=E(b.e|x) = E((X'X)$$^{-1}$$X'Y.e|x)=E((X'X)$$^{-1}$$X'(Xβ + ε).e|x)

=E((β.e + (X'X)$$^{-1}$$X'ε.e)|x)

=(X'X)$$^{-1}$$X'E(ε.e)

Solving for E(ε.e) gives σ²(n-k) [I wrote e= Y-Xb, substituting for b and some manipulation gave me e= ε + X(X'X)$$^{-1}$$X'ε]

This gives Cov(b,e|x)=(X'X)$$^{-1}$$X'σ²(n-k)

I think in this expression - (X'X)$$^{-1}$$X' - I shouldn't have X' in the right (as the formula of var(b) suggests). What am I missing?

• In your last equality E(β.e|x + (X'X)−1X'ε.e|x) is kind of vaudou for most humans. – Bertrand Sep 10 at 13:48
• @Bertrand Please have a look at the edited question. Does this make sense now? – Elina Gilbert Sep 10 at 14:24
• i) You should be careful with the dimensions of your vectors: $b$ is $K \times 1$ and $e$ is $N \times 1$, so the covariance is $K \times N$. It is better to transpose $e$ in $E[be^T]$. ii) After taking carefully into account the dimensions, it turns out that $E[\varepsilon e^T]$ is a $N \times N$ matrix, and is different from what you mention. – Bertrand Sep 10 at 19:22
• @Bertrand Thank you! Doing so, I am getting 0 covariance. Is it right? I mean why does it be so? – Elina Gilbert Sep 10 at 23:48
• The 0 answer seems to be right. You may also want to prove it using projections matrices $P_X$ and $M_X$. – Bertrand Sep 11 at 6:41