1
$\begingroup$

Today I read a document about multicollinearity here.

In the first page, the author runs a simple regression that

$y = \beta_1x1+ \beta_2x2$

the p-value( or t-statistic) of coefficients of $x1$ and $x2$ are insignificant but the overall F-test is significant. And he said that it is an indicator of multicollinearity.

I do not understand what is the reason for this conclusion, would you please help me to understand it intuitively?

Much appreciated.

$\endgroup$
2
  • $\begingroup$ If $y = x_1+ x_2$ there is no need for a regression. $\endgroup$
    – Bertrand
    Oct 27, 2021 at 8:25
  • $\begingroup$ @Betrand sorry, I forget to add the coefficient, it is a regression $\endgroup$ Oct 27, 2021 at 8:40

1 Answer 1

1
$\begingroup$

The reasoning behind it is simple:

  1. Multicolinearity distorts t-test and consequently p-values because it inflates standard errors of the $\beta$ coefficients. Since in multivariate regression the variance of estimated coefficients depends on correlation between independent variables.

  2. F-test is not affected by multicolinearity.

  3. the t-test tests whether coefficients are individually significant (with null $\beta_i=0$ and alternative hypothesis $\beta_i\neq0$

  4. The F-test tests the null of joint insignificance $\beta_1 = \beta_2 = … = \beta_k=0$ against alternative that at least one of the coefficients is not equal to zero.

Given 1-4 it is very strange to observe situation where using F-test we cannot reject the hypothesis that at least one coefficient is non-zero, while individual t-test we cannot reject the hypothesis that both coefficients are individually equal to zero.

In that case results become a bit suspicious which warrants further investigation. One likely explanation is that there is a multicolinearity present because as mentioned above multicolinearity distorts t-tests but not F-test.

However, this itself is not a test for multicolinearity just common symptom. If you want to make sure that multicolinearity is present you should make further investigation by for example calculating variance inflation factors. VIF>5 (some sources say VIF>10 these values are just rule of thumbs) tell you that multicolinearity is indeed present.

$\endgroup$
4
  • $\begingroup$ Thanks @1muflon1, from the calculation of p-value based on std.err (bmj.com/content/343/bmj.d2304), I know that the higher the std.err, the lower p-value. But I don't understand why multicollinearity cause the std.err inflation $\endgroup$ Oct 26, 2021 at 20:38
  • $\begingroup$ and can I ask why small N is also an indicator of multicollinearity in this case if it is convenient to you $\endgroup$ Oct 26, 2021 at 20:47
  • 1
    $\begingroup$ @Louise 1. Small N is not an indicator of multicolinearity. 2. It is because in formula for se in a multivariate regression there is also term $(1/(1-r_{x1x_2}^2$ $\endgroup$
    – 1muflon1
    Oct 27, 2021 at 18:19
  • $\begingroup$ Thanks @1muflon1 $\endgroup$ Oct 27, 2021 at 19:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.