# How is bias and consistency calculated for multiple regression?

If we have a normal OLS estimator, then the probability limit of Beta_1 hat as n -> infinity equals beta_1 plus (cov(x, error) / var(x)).

How is this calculated for a vector of betas?

For multivariate regressions, we can utilize vector notations and write the regression equation for observation $$i$$ as $$y_i = x_i' \beta + u_i$$ where $$y_i$$ is of dimension $$1 \times 1$$, $$x_i$$ is of dimension $$K \times 1$$, $$\beta$$ is of dimension $$K \times 1$$, and $$e_i$$ is of dimension $$1 \times 1$$.

We can then stack the $$n$$ equations ($$n$$ is the number of observations) to obtain the matrix form $$y = X \beta + e$$, where $$Y = (y_1, y_2, ..., y_n)'$$, $$X = (x_1', x_2', ..., x_n')'$$, and $$U = (u_1, u_2, ..., u_n)'$$.

Now the OLS estimator expressed in matrix notation is just $$\hat{\beta} = (X'X)^{-1} X'Y$$. Using the fact that $$Y = X \beta + U$$, we can rewrite that as:

$$(X'X)^{-1} X'(X \beta + U) = \beta + (X'X)^{-1} X'U$$

For consistency, by Law of Large Numbers, we have $$n^{-1} X'X = n^{-1}\sum{x_i x_i'} \rightarrow E(x x')$$ in probability, and $$n^{-1} X'U = n^{-1}\sum{x_i' y_i} \rightarrow E(x u)$$ in probability.

Now by Continuous Mapping Theorem, we have $$\hat{\beta} = \beta + (X'X)^{-1} X'U = \beta + (n^{-1} X'X)^{-1} n^{-1} X'U \rightarrow (E(x x'))^{-1} E(x u)$$ in probability. As $$E(xu) = 0$$ (this is generally assumed for OLS, as otherwise you have inconsistency, as we just showed) we have that $$\hat{\beta} \to \beta$$ in probability

For bias, look at: $$E(\hat{\beta} | X)$$ and try to prove that it is equal to $$\beta$$ no matter your choice of $$X$$

• Substitute $y=X\beta+e$ so that $\hat\beta = \beta + (X'X)^{-1} X'e$ and then follow @easyliving Dec 4, 2022 at 11:48