How is bias and consistency calculated for multiple regression?

Question

If we have a normal OLS estimator, then the probability limit of Beta_1 hat as n -> infinity equals beta_1 plus (cov(x, error) / var(x)).

How is this calculated for a vector of betas?

Answer 1

For multivariate regressions, we can utilize vector notations and write the regression equation for observation $i$ as $y_i = x_i' \beta + u_i$ where $y_i$ is of dimension $1 \times 1$, $x_i$ is of dimension $K \times 1$, $\beta$ is of dimension $K \times 1$, and $e_i$ is of dimension $1 \times 1$.

We can then stack the $n$ equations ($n$ is the number of observations) to obtain the matrix form $y = X \beta + e$, where $Y = (y_1, y_2, ..., y_n)'$, $X = (x_1', x_2', ..., x_n')'$, and $U = (u_1, u_2, ..., u_n)'$.

Now the OLS estimator expressed in matrix notation is just $\hat{\beta} = (X'X)^{-1} X'Y$. Using the fact that $Y = X \beta + U$, we can rewrite that as:

$$ (X'X)^{-1} X'(X \beta + U) = \beta + (X'X)^{-1} X'U $$

For consistency, by Law of Large Numbers, we have $n^{-1} X'X = n^{-1}\sum{x_i x_i'} \rightarrow E(x x')$ in probability, and $n^{-1} X'U = n^{-1}\sum{x_i' y_i} \rightarrow E(x u)$ in probability.

Now by Continuous Mapping Theorem, we have $\hat{\beta} = \beta + (X'X)^{-1} X'U = \beta + (n^{-1} X'X)^{-1} n^{-1} X'U \rightarrow (E(x x'))^{-1} E(x u)$ in probability. As $E(xu) = 0$ (this is generally assumed for OLS, as otherwise you have inconsistency, as we just showed) we have that $\hat{\beta} \to \beta$ in probability

For bias, look at: $$ E(\hat{\beta} | X) $$ and try to prove that it is equal to $\beta$ no matter your choice of $X$