0
$\begingroup$

My dataset contains two numerical variables (n1, n2) and six indicator variables. The first three indicator variables specify the location of a property (i1=north, i2=center, i3=south), the next three indicator variables specify the color of walls (j1=blue, j2=red, j3=other). Let's say I want to predict the property price (y). I want to run an ordinary least squares regression. I have two questions:

  1. Should I include all indicator variables or should I exclude one per group? And why?

y = b1 * n1 + b2 * n2 + b3 * i1 + b4 * i2 + b5 * i3 + b6 * j1 + b7 * j2 + b8 * j3

vs

y = b1 * n1 + b2 * n2 + b3 * i1 + b4 * i2 + b6 * j1 + b7 * j2

  1. Should I include an intercept? And how does it affect the interpretation of the indicator variables?

y = b0 + b1 * n1 + b2 * n2 + b3 * i1 + b4 * i2 + b5 * i3 + b6 * j1 + b7 * j2 + b8 * j3

and

y = b0 + b1 * n1 + b2 * n2 + b3 * i1 + b4 * i2 + b6 * j1 + b7 * j2

Improve this question
$\endgroup$
4
  • $\begingroup$ Why do you suspect there might be a case for excluding some variables? What do your regressions tell you? $\endgroup$
    – BrsG
    Commented Sep 15, 2022 at 9:33
  • $\begingroup$ Otherwise you overfit the model. The meaning of the regressors is already in the question $\endgroup$
    – NC520
    Commented Sep 15, 2022 at 10:05
  • $\begingroup$ I don't understand whether I should exclude one indicator variable from each group or just one overall. $\endgroup$
    – NC520
    Commented Sep 15, 2022 at 10:18
  • $\begingroup$ This is a common and well-answered question in stats.stackexchange.com. $\endgroup$
    – Alalalalaki
    Commented Sep 15, 2022 at 10:36

1 Answer 1

Reset to default
1
$\begingroup$

This seems like a question about perfect collinearity. Does $i1 +i2+i3 = 1$ always? If so, then you must exclude one of them if you want to include a constant. This would be the case if all the homes are either in the north, south, or center. If you have some homes in another region (east or west), then you can include all three of your variables as well as a constant because it is not always the case that $i1 +i2 +i3 = 1$.

It's analogous for the second set of 3 indicator variables.

Suppose $i1+i2+i3 = 1$, that is, all homes are either in north, south, or center. Then, if you include $i1$ and $i2$ as well as a constant, the constant is interpreted as "the average price for the omitted group (south)" while the coefficient for $i1$ is "the average difference in price for homes in the north compared to the south".

You could instead include $i1$, $i2$, and $i3$ but exclude the constant, in which case coefficients directly estimate the average for each group.

If you include other covariates (such as dummy variables for color of the walls), the above interpretation only changes in that we must add "holding fixed all other covariates".

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.