1
$\begingroup$

I am wondering when the sign and magnitude of coefficients of a variable of interest matter if it is statistically insignificant. Normally, I am concerning a coefficient of variable if it is significant at 5% level or lower. I am wondering when we should care about the magnitude and sign of coefficient if it is insignificant? Saying for example, my variable of interest saying about the impact of anticorruption laws on asset growth in Difference - in -Difference setting.

Please guide me to clarify my question more if it is not yet clear. Sorry @1muflon1 that I did not clarify my question that confused you

$\endgroup$
1
$\begingroup$

answer to new edit

If the coefficient is not significant then you cannot reject the hypothesis that true coefficient is zero. In that case, magnitude or sign of the coefficient is not very relevant.

You could still care about it a bit because if you find large coefficient with sign you would expect to find, it might be that it is insignificant only because there is a lot of noise in your data (remember test statistics depends not just on coefficient size but also standard errors $\hat{\beta}/se(\hat{\beta})$. So finding large coefficient with expected sign might motivate you to perhaps find larger dataset where there is less noise, but other than this it would not be very relevant.

Answer to original question:

Of course you should care about sign and magnitude of the coefficient. This is especially true when it comes to policy analysis.

I am not familiar with the anti-corruption laws research, so let me give you another example. Consider effect of minimum wage laws on employment.

Sign of the treatment dummy clearly matters as it would be a whole world of difference if research would show that minimum wages have positive impact on employment, to case where they have negative impact on employment (which implies difficult trade-off between higher wages for low income people vs their employment).

Second, magnitude of the coefficient matters as well as again if the relationship between minimum wage and employment is such that 1% increase in minimum wage leads to 10% decrease in employment that implies the trade-off would be very severe. However, if 1% increase in minimum wages increases unemployment just by 0.0001% then no matter whether the coefficient is statistically significant or not the effect is so small it could be safely ignored and you do not even need to worry about it.

$\endgroup$
1
  • $\begingroup$ I mean, when should we care about the magnitude and sign of the coefficient if it is insignificant. Sorry i did not explain my concern well. $\endgroup$
    – Nguyen Lis
    Oct 12 at 21:46
1
$\begingroup$

When insignificant (not significantly different from $0$), we should never interprete the sign and magnitude of the estimated coefficient. Because, with high probability the sign of the coefficient could be the opposite to the estimated one, and the magnitude of the coefficient either much smaller or much higher. The estimations are specific values taken by random variables which could be fully different for other $X$ values.
More precisely, the only interpretation which makes sense in a linear model, is that the impact of the variable $X_j$ is not significantly different from zero, the anti-corruption law is not an effective tool to fight against corruption, because in the treated sample there is comparable corruption as in the untreated sample for instance (ceteris paribus, and conditionally to the chosen model specification).

$\endgroup$
2
  • $\begingroup$ "high probability the sign of the coefficient could be the opposite to the estimated one" can I ask the reason why? I am wondering why it is high probability opposite to the estimated one? $\endgroup$
    – Nguyen Lis
    Oct 13 at 10:03
  • 1
    $\begingroup$ Assume that $\widehat{\beta_j}=1.23$ and that the p-value is $Pr[|T|>|t||\beta_j=0]=0.37$. In this case $\beta_j=0$ cannot be rejected at the 5% threshold. This means that if the null hypothesis is true there is 37/2% chance to draw a value such that $\widehat{\beta_j}=1.23$ or higher. And as the distribution is symmetric wrt 0, there is also 37/2 % chance to draw a value lower than $\widehat{\beta_j}=-1.23$, $\endgroup$
    – Bertrand
    Oct 13 at 12:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.