If we have a normal OLS estimator, then the probability limit of Beta_1 hat as n -> infinity equals beta_1 plus (cov(x, error) / var(x)).

How is this calculated for a vector of betas?


1 Answer 1


For multivariate regressions, we can utilize vector notations and write the regression equation for observation $i$ as $y_i = x_i' \beta + u_i$ where $y_i$ is of dimension $1 \times 1$, $x_i$ is of dimension $K \times 1$, $\beta$ is of dimension $K \times 1$, and $e_i$ is of dimension $1 \times 1$.

We can then stack the $n$ equations ($n$ is the number of observations) to obtain the matrix form $y = X \beta + e$, where $Y = (y_1, y_2, ..., y_n)'$, $X = (x_1', x_2', ..., x_n')'$, and $U = (u_1, u_2, ..., u_n)'$.

Now the OLS estimator expressed in matrix notation is just $\hat{\beta} = (X'X)^{-1} X'Y$. Using the fact that $Y = X \beta + U$, we can rewrite that as:

$$ (X'X)^{-1} X'(X \beta + U) = \beta + (X'X)^{-1} X'U $$

For consistency, by Law of Large Numbers, we have $n^{-1} X'X = n^{-1}\sum{x_i x_i'} \rightarrow E(x x')$ in probability, and $n^{-1} X'U = n^{-1}\sum{x_i' y_i} \rightarrow E(x u)$ in probability.

Now by Continuous Mapping Theorem, we have $\hat{\beta} = \beta + (X'X)^{-1} X'U = \beta + (n^{-1} X'X)^{-1} n^{-1} X'U \rightarrow (E(x x'))^{-1} E(x u)$ in probability. As $E(xu) = 0$ (this is generally assumed for OLS, as otherwise you have inconsistency, as we just showed) we have that $\hat{\beta} \to \beta$ in probability

For bias, look at: $$ E(\hat{\beta} | X) $$ and try to prove that it is equal to $\beta$ no matter your choice of $X$

  • $\begingroup$ Substitute $y=X\beta+e$ so that $\hat\beta = \beta + (X'X)^{-1} X'e$ and then follow @easyliving $\endgroup$
    – chan1142
    Dec 4, 2022 at 11:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.