# OLS and 2SLS normal equations

For a system of equations with $$M=2$$ endogenous variables, $$Y=\begin{bmatrix} y_1 & y_2 \end{bmatrix}$$ and $$K=3$$ exogenous variables, $$X=\begin{bmatrix} x_1 & x_2 & x_3 \end{bmatrix}$$. The first equation of the system is given by: $$y_{1i}=\gamma_{12}y_{2i}+\beta_{11}x_{1i}+\epsilon_{1i}$$. The data matrices yield, $$X'X$$, $$X'Y$$ and $$Y'Y$$, which are of dimension, $$3*3$$, $$3*2$$ and $$2*2$$ respectively

I need to

• Write the OLS and 2SLS normal equations in terms of cross products of the data matrices.

I tried using the standard procedure for deriving normal equations, that is, by minimizing sum of squared residuals, but I fail to understand how the equations can be modelled in form of the cross products of the given data matrices. I think I'm missing the point of the exercise. Please help me in understanding what the question demands. Thanks!

• Note that $x_1, x_2, x_3, y_1$ and $y_2$ are all $n\times 1$ vectors. – ahorn Oct 12 '19 at 6:20

OLS

The OLS part is clear: $$\sum_{i=1}^n y_{2i} (y_{1i} - \hat\gamma_{12} y_{2i} - \hat\beta_{11} x_{1i}) = 0$$ and $$\sum_{i=1}^n x_{1i} (y_{1i} - \hat\gamma_{12} y_{2i} - \hat\beta_{11} x_{1i}) = 0$$, where $$\hat\gamma_{12}$$ and $$\hat\beta_{11}$$ are the OLS estimators.

2SLS

For 2SLS, I will assume that $$(x_{1i}, x_{2i}, x_{3i})$$ is used as IV. The corresponding 2SLS is identical to the IV estimator using $$\hat{y}_{2i}$$ and $$x_{1i}$$ as IV. Thus, (I think) the "normal equations" are \begin{align} \sum_{i=1}^n \hat{y}_{2i} (y_{1i} - \tilde\gamma_{12} y_{2i} - \tilde\beta_{11} x_{1i}) &= 0,\\ \sum_{i=1}^n x_{1i} (y_{1i} - \tilde\gamma_{12} y_{2i} - \tilde\beta_{11} x_{1i}) &= 0, \end{align} where $$\tilde\gamma_{12}$$ and $$\tilde\beta_{11}$$ are the 2SLS estimators using $$x_{1i}, x_{2i}, x_{3i}$$ as IV, and $$\hat{y}_{2i}$$ is the fitted value obtained by regressing $$y_{2i}$$ on the instruments $$x_{1i}$$, $$x_{2i}$$ and $$x_{3i}$$.

In the above, I understood 2SLS as the IV estimator using $$(\hat{y}_{2i}, x_{1i})$$ as IV. We can also understand 2SLS as the OLS estimator of $$y_{1i}$$ on $$(\hat{y}_{2i}, x_{1i})$$. In that case, the normal equations can be written as \begin{align} \sum_{i=1}^n \hat{y}_{2i} (y_{1i} - \tilde\gamma_{12} \hat{y}_{2i} - \tilde\beta_{11} x_{1i}) &= 0,\\ \sum_{i=1}^n x_{1i} (y_{1i} - \tilde\gamma_{12} \hat{y}_{2i} - \tilde\beta_{11} x_{1i}) &= 0, \end{align} where $$y_{2i}$$ is replaced with $$\hat{y}_{2i}$$. The two sets of normal equations for 2SLS (one in terms of $$y_{2i}$$ and the other in terms of $$\hat{y}_{2i}$$) are identical.

Intercept

I have noticed that the intercept is excluded in the question. If the model is $$y_{1i} = \beta_{10} + \gamma_{12} y_{2i} + \beta_{11} x_{1i} + \epsilon_i$$ instead, you need equations for the intercept as well.

Matrix notations

Using matrix notations, let $$y$$ be the $$n\times 1$$ matrix of the LHS variable, $$X$$ the $$n\times 2$$ matrix of $$(y_{2i}, x_{1i})$$ and $$Z$$ the $$n\times 3$$ matrix of $$(x_{i1}, x_{i2}, x_{i3})$$. Let $$\beta = (\gamma_{12}, \beta_{11})'$$. (I am considering the model without the intercept. For the model with the intercept, $$X$$ and $$Z$$ contain a column of ones.) Then the OLS estimator is $$\hat\beta = (X'X)^{-1} X'y$$, which solves the normal equations $$X'(y-X\hat\beta)=\boldsymbol 0$$. To see this, note that $$X'(y-X\hat\beta)=\boldsymbol 0 \\ X'y - X'X\hat\beta = \boldsymbol 0\\ X'X\hat\beta = X'y\\ \hat\beta = (X'X)^{-1}X'y$$

The 2SLS (using $$Z$$ as instruments) estimator is $$\tilde\beta = [X'Z(Z'Z)^{-1} Z'X]^{-1} X'Z(Z'Z)^{-1} Z'y = (\hat{X}'X)^{-1} \hat{X}'y$$ where $$\hat{X} = Z(Z'Z)^{-1} Z'X$$. This 2SLS solves $$\hat{X}' (y-X\tilde\beta)=\boldsymbol 0$$, the normal equations for this 2SLS. Again, to see this, note that $$\hat{X}' (y-X\tilde\beta)=\boldsymbol 0\\ \hat X'y - \hat X'X\tilde\beta = \boldsymbol 0\\ \hat X'X\tilde\beta = \hat X' y\\ \tilde\beta = (\hat X'X)^{-1} \hat X'y$$ You will see that $$\hat{X}$$ is the $$n\times 2$$ matrix of $$(\hat{y}_{2i}, x_{1i})$$. It is also true that $$\tilde\beta = (\hat{X}'\hat{X})^{-1} \hat{X}'y$$, and the normal equations are also written as $$\hat{X}' (y-\hat{X}\tilde\beta)=0$$. The two different expressions are identical because $$\hat{X}'X = \hat{X}'\hat{X}$$.

• Note that $\boldsymbol {X'}(\boldsymbol y-\boldsymbol{X\hat\beta})=\boldsymbol 0$ is a $K\times 1$ vector. This corresponds to the equations at the beginning of the answer. – ahorn Oct 12 '19 at 6:23
• I've just found that my $X$ is different from OP's $X$. Sorry. – chan1142 Oct 12 '19 at 6:53
• Note that $(Z'Z)^{-1}Z'X$ is a matrix of coefficients, so $\hat X = Z(Z'Z)^{-1}Z'X$ is a $N\times 2$ matrix of the predicted values of the regressions of $\boldsymbol {y_2}$ and $\boldsymbol {x_1}$ on $\boldsymbol {x_1}, \boldsymbol {x_2}$ and $\boldsymbol {x_3}$ (known as the "projection"). You can see that $\boldsymbol{\hat x_1} = \boldsymbol {x_1}$ since $\boldsymbol {x_1}, \boldsymbol {x_2}$ and $\boldsymbol {x_3}$ are independent. However, $\boldsymbol {\hat y_2}\neq\boldsymbol {y_2}$. – ahorn Oct 12 '19 at 6:54
• no, I didn't pick up any mistakes in your answer. – ahorn Oct 12 '19 at 6:55
• You re-defined $X$, which was the correct thing to do (in order to maintain conventional notation). – ahorn Oct 12 '19 at 7:16