0
$\begingroup$

I have completed an F-test on the variable which gives p = 0.6, should I remove it from the model?

$\endgroup$
3
  • $\begingroup$ Yes you can. If it is just a simple dummy variable for a single observation then it is orthogonal to other variables in the model and is just eating up one degree of freedom. So unless there is a theoretical reason for including I would remove. $\endgroup$
    – Andrew M
    Apr 29, 2020 at 13:50
  • 1
    $\begingroup$ Can you present the model? It would be interesting to know what the variable is. Also what F-test deals with a single variable, there is more going on than a simple test of significance. Please provide your results! $\endgroup$
    – Brennan
    Apr 29, 2020 at 18:31
  • $\begingroup$ If you are interested in causal relationship, pretesting is dangerous. $\endgroup$
    – chan1142
    Apr 30, 2020 at 0:37

1 Answer 1

3
$\begingroup$

Statisticians object to this kind of pre-testing. This is a form of stepwise regression, a methodology known to be problematic.

“Stepwise regression model-building is…pants.”

Two issues particularly bother me, both of which are discussed further in the links.

  1. If you do any kind of testing (keep in mind that confidence intervals are just inverses of tests), the p-values are lying, since you also should condition on having performed a pre-test (but the usual p-values do not do this). This corresponds to criticisms 2 and 4 in the Harrell article I link at the end of this post.

  2. Even if you just want to predict, your measures of performance are likely to be inflated. In particular, $R^2_{adj}$ and the usual “unbiased” estimator of error term variance will use the degrees of freedom from the model fitted after the pre-test. However, you did consider that additional variable, and it should count towards the degrees of freedom. This corresponds to criticism #1 in the Harrell article I link at the end of this post.

Frank Harrell and Andrew Gelman have written some thoughts on stepwise regression that expand on my comments here.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.