1
$\begingroup$

If all assumptions hold (linear model, mean-zero error, homoskedasticity, no serial correlation and normal distribution), will the covariance between $\epsilon_{n+1}$ and $\hat{\beta}_{n}$ (the estimator) be zero?

I know that the $cov(x_{i},\epsilon_{i})=0$ by the mean-zero error assumption and law of iterated expectations. And I think that the error term and the estimator need to be uncorrelated as well. Can somebody explain me why or maybe derive it mathematically?

Thank you in advance

$\endgroup$
2

1 Answer 1

2
$\begingroup$

I would like to show you some intuition behind it.

Well, first think about what does the regression exactly do?

Figuratively speaking, it filters out the variability ("information") from the random component of some random variable. It does it by finding such object (e.g. line, plane, hyperplane, etc...) which minimizes the sum of squared residuals , which are a distance between dependent variable and this object, measured exactly in units of the dependent variable. In regression, this object is defined parametrically.

Now, what does that mean? You have your parameter $\hat{\beta_i}$ which is a random variable, meaning that by selecting a little different data, the result would be different. But here's the catch... For each realization of $\hat{\beta_i}$ you would have a set of realizations of residuals $\hat{e}$. If you take new data from some population, you change only 1 value of $\hat{\beta_i}$ but as many values of residuals as the number of data you use. Thus if you change the data you change the residuals and the parameter! Are they therefore correlated because of coming from the same generation process? Well, not exactly...

Illustration: Consider the setup in which you have simple regression model $y = \beta_0 + \epsilon$ and you repeatedly take the sample of 2 data from population containing $n$. And on each sample you perform the regression. See below:

independence

As you can see, for two arbitrarily given parameters $\hat{\beta_i}$ you can get similar residuals (see two pairs of red dots or two pairs of blue dots). They change randomly as the parameter changes. Thus $\hat{\beta_i}$ and $\hat{e}$ are independent

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.