4
$\begingroup$

I wanted to construct a logit model for determining the probability a recession will be determined for any given month using the usual Macro indicators; however, I noticed that 90% of the months in my range were expansionary and thought that might bias the model.

I thought that it might be more robust to bootstrap sample from the expansionary and recessionary months separately such that the new samples are balanced between the two. The final model would then be the average of the bootstrapped models.

I can see arguments for why this could work, actually introduce bias, or have no effect.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

Imbalanced classes present minimal problem to proper statistical methods.

A standard criticism of class imbalance is that it can result in models always or often classifying as the majority class. However, most models don’t actually do classification but output scores on a continuum, such as a logistic regression outputting probability. Software packages default to a cutoff threshold of $0.5$ probability, but this might be wildly inappropriate for your task (whether the classes are balanced or not). A simple approach is just to adjust the threshold. A more sophisticated approach would directly evaluate the probability outputs with a metric like log loss or Brier score (two examples of so-called “strictly proper scoring rules” that are uniquely minimized in expected value by the true probabilities, so they seek out the true probabilities of events).

This topic comes up so often on the statistics Stack, Cross Validated, that I have compiled a list of links to further reading about class imbalance and proper statistical methods that handle class imbalance.

https://stats.stackexchange.com/questions/357466

https://www.fharrell.com/post/class-damage/

https://www.fharrell.com/post/classification/

https://stats.stackexchange.com/a/359936/247274

https://stats.stackexchange.com/questions/464636/

https://stats.stackexchange.com/questions/558942/

https://stats.stackexchange.com/a/316114/247274

https://twitter.com/f2harrell/status/1062424969366462473?lang=en

(For those who do not know, Frank Harrell was the founding chair of Biostatistics at Vanderbilt University.)

We also have a Cross Validated Meta post that deals with this topic and links to other material (and a lot of the same material).

Improve this answer
$\endgroup$
2
  • $\begingroup$ Just to check my understanding, are you suggesting that I try changing the cut-off such that it's closer to proportion of the majority class (I have 90% of obs coded as 1, so I should try setting the threshold to 0.9)? Generally, when I build these models by hand--as opposed to using a library--I'll use the Brier score as the objective function and optimize over the whole sample. $\endgroup$
    – Forgot the Jacobian
    Sep 16, 2022 at 20:33
  • $\begingroup$ Adjusting the threshold is a simple approach. The more sophisticated approach would be to work with the probability values, not with categories; Brier score is one such approach. Two links worth perusing: stats.stackexchange.com/q/312119/247274 stats.stackexchange.com/questions/478494/… $\endgroup$
    – Dave
    Sep 16, 2022 at 21:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.