7
$\begingroup$

In Greene's Econometric Analysis there is a derivation regarding the F stat. The setup is a null hypothesis of the form: $H_0: R\beta =q$ where $\beta$ is a $k\times 1$ vector of parameters, $R$ is a $J\times k$ matrix and $q$ is a $J\times 1$ vector. $R$ and $q$ together define restrictions that are hypothesized. $e_*$ are residuals that result from OLS when imposing the restrictions and $e$ are residuals when estimating OLS without restrictions. $b_*$ and $b$ are respective parameter estimates. (Note that $Rb_*=q$).

Greene's derivation has a portion in which this line is shown,

$$(1) \hspace{3.2cm} e_*'e_*=e'e +(b-b_*)'X'X(b-b_*) $$ The next line shows,

$$(2) \hspace{0.5cm}e_*'e_*-e'e =(Rb-q)'[R(X'X)^{-1}R']^{-1}(Rb-q) $$

I am curious how to go between these lines. If $R$ were an invertible square matrix, it is immediate because $e'e +(b-b_*)'X'X(b-b_*) =e'e +(b-b_*)'R'(R')^{-1}X'XR^{-1}R(b-b_*)$ then we simplify further.

$R$ is not an invertible square matrix however, so this can't be the method. Futhermore, appealing to the pseudoinverse does not help, because $R$ does not have full column rank, thus $R^+R\ne I$ and we can't proceed with $e'e +(b-b_*)'X'X(b-b_*) =e'e +(b-b_*)'R'(R')^{+}X'XR^{+}R(b-b_*)$.

In short, there must be an algebraic trick to go from equation (1) to (2). I am curious what that trick is.

$\endgroup$
4
  • $\begingroup$ This looks like a good candidate for Cross Validated Stack Exchange. $\endgroup$ Commented Oct 18, 2023 at 16:14
  • 2
    $\begingroup$ Michael: Unfortunately, the proof at the link below uses different notation that you used but I'm quite confident that you can follow it and map it to your case. I did find a textbook derivation that uses your notation ( a introduction to econometric theory by james davidson ) but it's a fairly new text and probably not worth purchasing for one derivation. web.vu.lt/mif/a.buteikis/wp-content/uploads/PE_Book/… $\endgroup$
    – mark leeds
    Commented Oct 19, 2023 at 5:30
  • $\begingroup$ Mark Leeds this is great. Thank you. You could consider making this an answer rather than a comment. $\endgroup$ Commented Oct 20, 2023 at 8:54
  • $\begingroup$ Hi Michael: I'm glad it helps. I'm running out now but I'll try to remember later. Note that I re-read my comment in above and I don't mean to denigrate Davidson's book. It's quite a nice intro book. If someone is interested in that type of book, I highly recommend it. $\endgroup$
    – mark leeds
    Commented Oct 20, 2023 at 16:23

1 Answer 1

4
$\begingroup$

Consider the restricted least squares problem: $$ \min \underbrace{(Y - Xb_\ast)'}_{(1\times n)}\underbrace{(Y - Xb_\ast)}_{n\times 1} \text{ s.t. } \underbrace{R}_{(j \times k)}\underbrace{b_\ast}_{(k \times 1)} - \underbrace{q}_{(j \times 1)} $$ We set up the Lagrangian $$ L(b_\ast,\lambda) = (Y - Xb_\ast)'(Y - Xb_\ast) - 2 \underbrace{\lambda'}_{(1 \times j)}(Rb_\ast-q). $$ The first order conditions give: $$ \underbrace{X'(Y - X b_\ast)}_{(k \times k)} + \underbrace{R'\lambda}_{(k \times k)} = 0. $$ This gives: $$ b_\ast = \underbrace{(X'X)^{-1}X'Y}_{b} - (X'X)^{-1}R'\lambda. $$ The first term is the usual (unrestricted) OLS estimator, say $b$. Multiplying both sides with $R$ gives: $$ \underbrace{Rb_\ast}_{q} = \underbrace{R b}_{(j \times 1)} - \underbrace{R(X'X)^{-1}R'}_{(j \times j)} \underbrace{\lambda}_{j \times j} $$ If $R(X'X)^{-1}R'$ is of full rank, this can be solved for $\lambda$: $$ \lambda = (R(X'X)^{-1}R')^{-1}(Rb - q). $$ Subsituting into the expression for $b_\ast$ gives: $$ b_\ast - b = \underbrace{(X'X)^{-1}}_{(k \times k)} \underbrace{R'}_{(k \times j)}\underbrace{(R(X'X)^{-1}R')^{-1}}_{(j \times j)} \underbrace{(q - Rb)}_{(j \times 1)} $$

Now, substitute this into the expression $(b-b_\ast)'X'X(b - b_ast)$

$$ (q - Rb)' (R(X'X)^{-1}R')^{-1}R(X'X)^{-1} X'X (X'X)^{-1}R'(R(X'X)^{-1}R')^{-1}(q - Rb) $$

We can simplify the middle part as $X'X (X'X)^{-1} = I$. $$ (q - Rb)' (R(X'X)^{-1}R')^{-1}R(X'X)^{-1} R'(R(X'X)^{-1}R')^{-1}(q - Rb) $$ Next, also $R(X'X)^{-1}R'(R(X'X)^{-1}R')^{-1} = I$ so we get that: $$ (b- b_\ast) X'X (b - b_\ast) = (q - Rb)' (R(X'X)^{-1}R')^{-1}(q - Rb) $$ This is expression (2).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.