1
$\begingroup$

For a system of equations with $M=2$ endogenous variables, $ Y=\begin{bmatrix} y_1 & y_2 \end{bmatrix}$ and $K=3$ exogenous variables, $X=\begin{bmatrix} x_1 & x_2 & x_3 \end{bmatrix}$. The first equation of the system is given by: $y_{1i}=\gamma_{12}y_{2i}+\beta_{11}x_{1i}+\epsilon_{1i}$. The data matrices yield, $X'X$, $X'Y$ and $Y'Y$, which are of dimension, $3*3$, $3*2$ and $2*2$ respectively

I need to

  • Write the OLS and 2SLS normal equations in terms of cross products of the data matrices.

I tried using the standard procedure for deriving normal equations, that is, by minimizing sum of squared residuals, but I fail to understand how the equations can be modelled in form of the cross products of the given data matrices. I think I'm missing the point of the exercise. Please help me in understanding what the question demands. Thanks!

$\endgroup$
  • $\begingroup$ Note that $x_1, x_2, x_3, y_1$ and $y_2$ are all $n\times 1$ vectors. $\endgroup$ – ahorn Oct 12 '19 at 6:20
1
$\begingroup$

OLS

The OLS part is clear: $\sum_{i=1}^n y_{2i} (y_{1i} - \hat\gamma_{12} y_{2i} - \hat\beta_{11} x_{1i}) = 0$ and $\sum_{i=1}^n x_{1i} (y_{1i} - \hat\gamma_{12} y_{2i} - \hat\beta_{11} x_{1i}) = 0$, where $\hat\gamma_{12}$ and $\hat\beta_{11}$ are the OLS estimators.

2SLS

For 2SLS, I will assume that $(x_{1i}, x_{2i}, x_{3i})$ is used as IV. The corresponding 2SLS is identical to the IV estimator using $\hat{y}_{2i}$ and $x_{1i}$ as IV. Thus, (I think) the "normal equations" are \begin{align} \sum_{i=1}^n \hat{y}_{2i} (y_{1i} - \tilde\gamma_{12} y_{2i} - \tilde\beta_{11} x_{1i}) &= 0,\\ \sum_{i=1}^n x_{1i} (y_{1i} - \tilde\gamma_{12} y_{2i} - \tilde\beta_{11} x_{1i}) &= 0, \end{align} where $\tilde\gamma_{12}$ and $\tilde\beta_{11}$ are the 2SLS estimators using $x_{1i}, x_{2i}, x_{3i}$ as IV, and $\hat{y}_{2i}$ is the fitted value obtained by regressing $y_{2i}$ on the instruments $x_{1i}$, $x_{2i}$ and $x_{3i}$.

In the above, I understood 2SLS as the IV estimator using $(\hat{y}_{2i}, x_{1i})$ as IV. We can also understand 2SLS as the OLS estimator of $y_{1i}$ on $(\hat{y}_{2i}, x_{1i})$. In that case, the normal equations can be written as \begin{align} \sum_{i=1}^n \hat{y}_{2i} (y_{1i} - \tilde\gamma_{12} \hat{y}_{2i} - \tilde\beta_{11} x_{1i}) &= 0,\\ \sum_{i=1}^n x_{1i} (y_{1i} - \tilde\gamma_{12} \hat{y}_{2i} - \tilde\beta_{11} x_{1i}) &= 0, \end{align} where $y_{2i}$ is replaced with $\hat{y}_{2i}$. The two sets of normal equations for 2SLS (one in terms of $y_{2i}$ and the other in terms of $\hat{y}_{2i}$) are identical.

Intercept

I have noticed that the intercept is excluded in the question. If the model is $y_{1i} = \beta_{10} + \gamma_{12} y_{2i} + \beta_{11} x_{1i} + \epsilon_i$ instead, you need equations for the intercept as well.

Matrix notations

Using matrix notations, let $y$ be the $n\times 1$ matrix of the LHS variable, $X$ the $n\times 2$ matrix of $(y_{2i}, x_{1i})$ and $Z$ the $n\times 3$ matrix of $(x_{i1}, x_{i2}, x_{i3})$. Let $\beta = (\gamma_{12}, \beta_{11})'$. (I am considering the model without the intercept. For the model with the intercept, $X$ and $Z$ contain a column of ones.) Then the OLS estimator is $\hat\beta = (X'X)^{-1} X'y$, which solves the normal equations $X'(y-X\hat\beta)=\boldsymbol 0$. To see this, note that $$X'(y-X\hat\beta)=\boldsymbol 0 \\ X'y - X'X\hat\beta = \boldsymbol 0\\ X'X\hat\beta = X'y\\ \hat\beta = (X'X)^{-1}X'y$$

The 2SLS (using $Z$ as instruments) estimator is $$\tilde\beta = [X'Z(Z'Z)^{-1} Z'X]^{-1} X'Z(Z'Z)^{-1} Z'y = (\hat{X}'X)^{-1} \hat{X}'y$$ where $\hat{X} = Z(Z'Z)^{-1} Z'X$. This 2SLS solves $\hat{X}' (y-X\tilde\beta)=\boldsymbol 0$, the normal equations for this 2SLS. Again, to see this, note that $$\hat{X}' (y-X\tilde\beta)=\boldsymbol 0\\ \hat X'y - \hat X'X\tilde\beta = \boldsymbol 0\\ \hat X'X\tilde\beta = \hat X' y\\ \tilde\beta = (\hat X'X)^{-1} \hat X'y $$ You will see that $\hat{X}$ is the $n\times 2$ matrix of $(\hat{y}_{2i}, x_{1i})$. It is also true that $\tilde\beta = (\hat{X}'\hat{X})^{-1} \hat{X}'y$, and the normal equations are also written as $\hat{X}' (y-\hat{X}\tilde\beta)=0$. The two different expressions are identical because $\hat{X}'X = \hat{X}'\hat{X}$.

|improve this answer|||||
$\endgroup$
  • $\begingroup$ Note that $\boldsymbol {X'}(\boldsymbol y-\boldsymbol{X\hat\beta})=\boldsymbol 0$ is a $K\times 1$ vector. This corresponds to the equations at the beginning of the answer. $\endgroup$ – ahorn Oct 12 '19 at 6:23
  • $\begingroup$ I've just found that my $X$ is different from OP's $X$. Sorry. $\endgroup$ – chan1142 Oct 12 '19 at 6:53
  • 1
    $\begingroup$ Note that $(Z'Z)^{-1}Z'X$ is a matrix of coefficients, so $\hat X = Z(Z'Z)^{-1}Z'X$ is a $N\times 2$ matrix of the predicted values of the regressions of $\boldsymbol {y_2}$ and $\boldsymbol {x_1}$ on $\boldsymbol {x_1}, \boldsymbol {x_2}$ and $\boldsymbol {x_3}$ (known as the "projection"). You can see that $\boldsymbol{\hat x_1} = \boldsymbol {x_1}$ since $\boldsymbol {x_1}, \boldsymbol {x_2}$ and $\boldsymbol {x_3}$ are independent. However, $\boldsymbol {\hat y_2}\neq\boldsymbol {y_2}$. $\endgroup$ – ahorn Oct 12 '19 at 6:54
  • $\begingroup$ no, I didn't pick up any mistakes in your answer. $\endgroup$ – ahorn Oct 12 '19 at 6:55
  • $\begingroup$ You re-defined $X$, which was the correct thing to do (in order to maintain conventional notation). $\endgroup$ – ahorn Oct 12 '19 at 7:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.