For a system of equations with $M=2$ endogenous variables, $ Y=\begin{bmatrix} y_1 & y_2 \end{bmatrix}$ and $K=3$ exogenous variables, $X=\begin{bmatrix} x_1 & x_2 & x_3 \end{bmatrix}$. The first equation of the system is given by: $y_{1i}=\gamma_{12}y_{2i}+\beta_{11}x_{1i}+\epsilon_{1i}$. The data matrices yield, $X'X$, $X'Y$ and $Y'Y$, which are of dimension, $3*3$, $3*2$ and $2*2$ respectively

I need to

  • Write the OLS and 2SLS normal equations in terms of cross products of the data matrices.

I tried using the standard procedure for deriving normal equations, that is, by minimizing sum of squared residuals, but I fail to understand how the equations can be modelled in form of the cross products of the given data matrices. I think I'm missing the point of the exercise. Please help me in understanding what the question demands. Thanks!

  • $\begingroup$ Note that $x_1, x_2, x_3, y_1$ and $y_2$ are all $n\times 1$ vectors. $\endgroup$
    – ahorn
    Oct 12, 2019 at 6:20

1 Answer 1



The OLS part is clear: $\sum_{i=1}^n y_{2i} (y_{1i} - \hat\gamma_{12} y_{2i} - \hat\beta_{11} x_{1i}) = 0$ and $\sum_{i=1}^n x_{1i} (y_{1i} - \hat\gamma_{12} y_{2i} - \hat\beta_{11} x_{1i}) = 0$, where $\hat\gamma_{12}$ and $\hat\beta_{11}$ are the OLS estimators.


For 2SLS, I will assume that $(x_{1i}, x_{2i}, x_{3i})$ is used as IV. The corresponding 2SLS is identical to the IV estimator using $\hat{y}_{2i}$ and $x_{1i}$ as IV. Thus, (I think) the "normal equations" are \begin{align} \sum_{i=1}^n \hat{y}_{2i} (y_{1i} - \tilde\gamma_{12} y_{2i} - \tilde\beta_{11} x_{1i}) &= 0,\\ \sum_{i=1}^n x_{1i} (y_{1i} - \tilde\gamma_{12} y_{2i} - \tilde\beta_{11} x_{1i}) &= 0, \end{align} where $\tilde\gamma_{12}$ and $\tilde\beta_{11}$ are the 2SLS estimators using $x_{1i}, x_{2i}, x_{3i}$ as IV, and $\hat{y}_{2i}$ is the fitted value obtained by regressing $y_{2i}$ on the instruments $x_{1i}$, $x_{2i}$ and $x_{3i}$.

In the above, I understood 2SLS as the IV estimator using $(\hat{y}_{2i}, x_{1i})$ as IV. We can also understand 2SLS as the OLS estimator of $y_{1i}$ on $(\hat{y}_{2i}, x_{1i})$. In that case, the normal equations can be written as \begin{align} \sum_{i=1}^n \hat{y}_{2i} (y_{1i} - \tilde\gamma_{12} \hat{y}_{2i} - \tilde\beta_{11} x_{1i}) &= 0,\\ \sum_{i=1}^n x_{1i} (y_{1i} - \tilde\gamma_{12} \hat{y}_{2i} - \tilde\beta_{11} x_{1i}) &= 0, \end{align} where $y_{2i}$ is replaced with $\hat{y}_{2i}$. The two sets of normal equations for 2SLS (one in terms of $y_{2i}$ and the other in terms of $\hat{y}_{2i}$) are identical.


I have noticed that the intercept is excluded in the question. If the model is $y_{1i} = \beta_{10} + \gamma_{12} y_{2i} + \beta_{11} x_{1i} + \epsilon_i$ instead, you need equations for the intercept as well.

Matrix notations

Using matrix notations, let $y$ be the $n\times 1$ matrix of the LHS variable, $X$ the $n\times 2$ matrix of $(y_{2i}, x_{1i})$ and $Z$ the $n\times 3$ matrix of $(x_{i1}, x_{i2}, x_{i3})$. Let $\beta = (\gamma_{12}, \beta_{11})'$. (I am considering the model without the intercept. For the model with the intercept, $X$ and $Z$ contain a column of ones.) Then the OLS estimator is $\hat\beta = (X'X)^{-1} X'y$, which solves the normal equations $X'(y-X\hat\beta)=\boldsymbol 0$. To see this, note that $$X'(y-X\hat\beta)=\boldsymbol 0 \\ X'y - X'X\hat\beta = \boldsymbol 0\\ X'X\hat\beta = X'y\\ \hat\beta = (X'X)^{-1}X'y$$

The 2SLS (using $Z$ as instruments) estimator is $$\tilde\beta = [X'Z(Z'Z)^{-1} Z'X]^{-1} X'Z(Z'Z)^{-1} Z'y = (\hat{X}'X)^{-1} \hat{X}'y$$ where $\hat{X} = Z(Z'Z)^{-1} Z'X$. This 2SLS solves $\hat{X}' (y-X\tilde\beta)=\boldsymbol 0$, the normal equations for this 2SLS. Again, to see this, note that $$\hat{X}' (y-X\tilde\beta)=\boldsymbol 0\\ \hat X'y - \hat X'X\tilde\beta = \boldsymbol 0\\ \hat X'X\tilde\beta = \hat X' y\\ \tilde\beta = (\hat X'X)^{-1} \hat X'y $$ You will see that $\hat{X}$ is the $n\times 2$ matrix of $(\hat{y}_{2i}, x_{1i})$. It is also true that $\tilde\beta = (\hat{X}'\hat{X})^{-1} \hat{X}'y$, and the normal equations are also written as $\hat{X}' (y-\hat{X}\tilde\beta)=0$. The two different expressions are identical because $\hat{X}'X = \hat{X}'\hat{X}$.

  • $\begingroup$ Note that $\boldsymbol {X'}(\boldsymbol y-\boldsymbol{X\hat\beta})=\boldsymbol 0$ is a $K\times 1$ vector. This corresponds to the equations at the beginning of the answer. $\endgroup$
    – ahorn
    Oct 12, 2019 at 6:23
  • $\begingroup$ I've just found that my $X$ is different from OP's $X$. Sorry. $\endgroup$
    – chan1142
    Oct 12, 2019 at 6:53
  • 1
    $\begingroup$ Note that $(Z'Z)^{-1}Z'X$ is a matrix of coefficients, so $\hat X = Z(Z'Z)^{-1}Z'X$ is a $N\times 2$ matrix of the predicted values of the regressions of $\boldsymbol {y_2}$ and $\boldsymbol {x_1}$ on $\boldsymbol {x_1}, \boldsymbol {x_2}$ and $\boldsymbol {x_3}$ (known as the "projection"). You can see that $\boldsymbol{\hat x_1} = \boldsymbol {x_1}$ since $\boldsymbol {x_1}, \boldsymbol {x_2}$ and $\boldsymbol {x_3}$ are independent. However, $\boldsymbol {\hat y_2}\neq\boldsymbol {y_2}$. $\endgroup$
    – ahorn
    Oct 12, 2019 at 6:54
  • $\begingroup$ no, I didn't pick up any mistakes in your answer. $\endgroup$
    – ahorn
    Oct 12, 2019 at 6:55
  • $\begingroup$ You re-defined $X$, which was the correct thing to do (in order to maintain conventional notation). $\endgroup$
    – ahorn
    Oct 12, 2019 at 7:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.