Given regression equation Y=Xβ + ε which satisfies all classical assumptions including those of homoscadasticity

Estimated regression equation using OLS is the following -

Y=Xb + e

Where, b is the OLS estimator for β and e is the estimated residual term for equation where E(e|x) =0

Note: b=(X'X)$^{-1}$X'Y

We can find, Cov(b,e|x) = E(b.e|x) - E(b).E(e|x)

=E(b.e|x) = E((X'X)$^{-1}$X'Y.e|x)=E((X'X)$^{-1}$X'(Xβ + ε).e|x)

=E((β.e + (X'X)$^{-1}$X'ε.e)|x)


Solving for E(ε.e) gives σ²(n-k) [I wrote e= Y-Xb, substituting for b and some manipulation gave me e= ε + X(X'X)$^{-1}$X'ε]

This gives Cov(b,e|x)=(X'X)$^{-1}$X'σ²(n-k)

I think in this expression - (X'X)$^{-1}$X' - I shouldn't have X' in the right (as the formula of var(b) suggests). What am I missing?


  • 1
    $\begingroup$ In your last equality E(β.e|x + (X'X)−1X'ε.e|x) is kind of vaudou for most humans. $\endgroup$ – Bertrand Sep 10 '19 at 13:48
  • $\begingroup$ @Bertrand Please have a look at the edited question. Does this make sense now? $\endgroup$ – Elina Gilbert Sep 10 '19 at 14:24
  • $\begingroup$ i) You should be careful with the dimensions of your vectors: $b$ is $K \times 1$ and $e$ is $N \times 1$, so the covariance is $K \times N$. It is better to transpose $e$ in $E[be^T]$. ii) After taking carefully into account the dimensions, it turns out that $E[\varepsilon e^T]$ is a $N \times N$ matrix, and is different from what you mention. $\endgroup$ – Bertrand Sep 10 '19 at 19:22
  • $\begingroup$ @Bertrand Thank you! Doing so, I am getting 0 covariance. Is it right? I mean why does it be so? $\endgroup$ – Elina Gilbert Sep 10 '19 at 23:48
  • $\begingroup$ The 0 answer seems to be right. You may also want to prove it using projections matrices $P_X$ and $M_X$. $\endgroup$ – Bertrand Sep 11 '19 at 6:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.