0
$\begingroup$

While constructing the F-test with matrix notation,

For the Unrestricted model, we have:

$$ y=X_{1}\cdot b_{1}+X_{2}\cdot b_{2}+e $$

For the Restricted model;

$$ y=\widehat{X}_{1}.b_{1}+e_{R} $$

To create a linear relationship between $ e_{R} $ AND $ e $ ,we multiply the unrestricted model with $M_{1}$ such that:

$$ \underbrace{M_{1}\cdot y}_{\text{$e_{R}$}} = \underbrace{M_{1}\cdot X_{1}}_{\text{0}} b_{1}+M_{1}\cdot X_{2}\cdot b_{2}+ \underbrace{M_{1}\cdot e}_{\text{e}} $$

Therefore,

$$ e_{R}^{T}\cdot e_{R} = \left( e+M_{1}\cdot X_{2}\cdot b_{2}\right) ^{T} \cdot \left( e+M_{1}\cdot X_{2}\cdot b_{2}\right) $$

After multiplication and by using Normal Equations, we get:

$$ e_{R}^{T}\cdot e_{R} -e^{T}\cdot e = b_{2}^{T}\cdot X_{2}^{T}.M_{1}\cdot X_{2}\cdot b_{2} $$

And here after solving a partitioned regression we have the value of $b_{2} $ such that:

$$ b_{2} = (X_{2}^{T}.M_{1}\cdot X_{2})^{-1} \cdot X_{2}^{T} \cdot M_{1} \cdot y $$

Inserting $ y = X_{1}\cdot \beta_{1} + X_{2}\cdot \beta_{2} $ , and after multiptication, we have such a $b_{2}$ :

  • $$ b_{2} = \underbrace{(X_{2}^{T}.M_{1}\cdot X_{2})^{-1} \cdot X_{2}^{T} \cdot X_{1} \cdot \beta_{1}}_{\text{$0$}} + \underbrace{(X_{2}^{T}.M_{1}\cdot X_{2})^{-1} \cdot X_{2}^{T} \cdot X_{2} \cdot \beta_{2}}_{\text{$0$}} + ... $$

and it continues actually but here in the part $ \underbrace{(X_{2}^{T}.M_{1}\cdot X_{2})^{-1} \cdot X_{2}^{T} \cdot X_{2} \cdot \beta_{2}}_{\text{$0$}}$ for this expression to be $0$ we need to the null hypothesis $H_{0}:\beta_{2} = 0$

My question is why do we need to impose this? and what happens if we don't?

$\endgroup$
16
  • 1
    $\begingroup$ Hi: It's because that's the restriction that the null hypothesis is imposing. It's analogous with a t-test for testing $\mu =0$. When we calculate the t-statistic, we use $\mu = 0$ in the expression for the t-statistic. So, we end up with $ T = \frac{\bar{x}}{\frac{\sigma}{\sqrt{n}}}$. $\endgroup$
    – mark leeds
    Nov 26, 2022 at 20:40
  • 1
    $\begingroup$ Hi: What happens is that then the statistic you obtain will be incorrect because it's not assuming that the null hypothesis is true. The steps of a hypothesis test ( neyman pearson ) require that the test- statistic is such that the null hypothesis is true when it is calculated. The idea is to create a pivotal quantity that has a known distribution when the null is true. Then, if this pivotal quantity ends up being extreme in terms of where it falls in the distribution, then this is evidence that the null is not true. $\endgroup$
    – mark leeds
    Nov 27, 2022 at 21:11
  • 1
    $\begingroup$ You'll understand the answer more clearly if you take a class or look at book on statistical hypothesis testing. $\endgroup$
    – mark leeds
    Nov 27, 2022 at 21:11
  • 1
    $\begingroup$ I hope that it made sense. Lehmann is one book for statistical hypothesis testing. Another one might be Casella and Berger. $\endgroup$
    – mark leeds
    Nov 28, 2022 at 18:14
  • 1
    $\begingroup$ Hi: I'm not sure that your question is so much about projection matrices but it won't hurt to understand projection in regression. I think a more key issue is what is termed "the pivotal quantity". This is what we want to use as a part of the test statistic, because, when the null hypothesis is true, the distribution of the test-statistic is then independent of the value of the estimate being used to test the hypothesis. ( in a t-test, it's $\bar{x}$ ). $\endgroup$
    – mark leeds
    Nov 30, 2022 at 17:02

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.