Is there any explanation if running without intercept caused the change in coefficients of other variables ?
I am talking about a panel data regression.
xtreg y x
and
xtreg y x, nocons
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
4
Running an (OLS) regression with or without intercept will not change the other coefficients of the other covariates if the means of (all) these covariates are zero.
For simplicity, consider the case of one covariate. We have two regressions, the first with and the second without an intercept. $$ \begin{align*} &y_i = \alpha + \beta x_{i} + \varepsilon_i, \tag{1}\\ &y_i = \gamma x_i + \delta_i \tag{2} \end{align*} $$ For the first regression, you will estimate the coefficients by setting the following sample moment conditions: $$ \begin{align*} &\frac{1}{n} \sum_i \varepsilon_i = 0,\\ &\frac{1}{n} \sum_i \varepsilon_i x_i = 0 \end{align*} $$ So: $$ \begin{align*} &\frac{1}{n} \sum_i y_i = \hat \alpha + \hat \beta \left(\frac{1}{n} \sum_i x_i\right),\\ &\frac{1}{n} \sum_i y_ix_i = \hat \alpha \left(\frac{1}{n} \sum_i x_i\right) + \hat \beta \left(\frac{1}{n} \sum_i (x_i)^2\right) \end{align*} $$ Then: $$ \begin{align*} &\hat \alpha = \left(\frac{1}{n} \sum_i y_i\right) - \hat \beta \left(\frac{1}{n} \sum_i x_i\right),\\ &\frac{1}{n} \sum_i y_i x_i = \left(\frac{1}{n} \sum_i y_i\right) \left( \frac{1}{n} \sum_i x_i\right) - \hat \beta \left(\frac{1}{n} \sum_i x_i\right)^2 + \hat \beta \left(\frac{1}{n} \sum_i (x_i)^2\right), \end{align*} $$ So using $\widehat{cov}(y_i, x_i) = \left(\frac{1}{n} \sum_i y_i x_i\right) - \left(\frac{1}{n} \sum_i y_i\right) \left( \frac{1}{n} \sum_i x_i\right)$ and $\widehat{var}(x_i) = \left(\frac{1}{n} \sum_i (x_i)^2\right) - \left(\frac{1}{n} \sum_i x_i\right)^2$: $$ \hat \beta = \frac{\widehat{cov}(y_i, x_i)}{\widehat{var}(x_i)}. \tag{3} $$ For the regression $(2)$ without intercept, we only have the second moment condition: $$ \begin{align*} &\frac{1}{n} \sum_i \delta_i x_i = 0,\\ \to &\frac{1}{n} \sum_i y_i x_i = \hat \gamma \left(\frac{1}{n} \sum_i (x_i)^2\right) \end{align*} $$ So: $$ \hat \gamma = \frac{\frac{1}{n} \sum_i y_i x_i}{\frac{1}{n} \sum_i (x_i)^2} \tag{4} $$ Comparing $(3)$ and $(4)$, we see that they are the same if the mean of $x$ is zero: $\frac{1}{n} \sum_i x_i = 0$.
The figure below demonstrates. We have 5 data points with $\frac{1}{n} \sum_i x_i = 0$. We see that the two regression lines, the black with intercept and the red without intercept have the same slope.
-
1$\begingroup$ That is a rather special case. If the mean of any of the covariates is not zero you would typically see all the coefficients change by removing the intercept. As your chart shows, if the means of all of the covariates is zero but the mean of the variate is not zero then the no-intercept regression can be so seriously biased as to be largely meaningless $\endgroup$– HenryJun 17, 2021 at 8:03
-
2$\begingroup$ @Henry Indeed, the intercept will be biased (as you fix it to zero). However, the slope will still be unbiased. The whole derivation is in fact a special case of the Firsh-Waugh-Lovell theorem. First de-mean the covariates by regressing on the constant term and taking the residuals. Then regress $y$ on these de-meaned variables. $\endgroup$– tdmJun 17, 2021 at 8:49
-
$\begingroup$ Hi @tdm, a novice question, I can explain and rpvoe that (3) equalling to (4) based on this condition. However, can I ask why for the (1) regression ,For the first regression, I can estimate the coefficients by setting the following sample moment conditions: $$ \begin{align*} &\frac{1}{n} \sum_i \varepsilon_i = 0,\\ &\frac{1}{n} \sum_i \varepsilon_i x_i = 0 \end{align*} $$ $\endgroup$ Jun 17, 2021 at 23:20
-
2$\begingroup$ @BeautifulMindset One way to see this is to notice that OLS minimizes $\frac{1}{n} \sum_i (y_i - \alpha - \beta x)^2$ with respect to $\alpha$ and $\beta$. This gives the conditions $\frac{1}{n} \sum_i (y_i - \alpha - \beta x) = 0$ and $\frac{1}{n} \sum_i (y_i - \alpha - \beta x) x = 0$. Subsituting back $\varepsilon = (y - \alpha - \beta x)$ gives you the two moment conditions. Alternatively, you can also see the conditions as the method of moments estimator (see for example here). $\endgroup$– tdmJun 18, 2021 at 5:31
$\begingroup$
$\endgroup$
While the previous answer was very instructive, the correct answer for Stata is that xtreg y x, nocons will result in error r(198) option noconstant not allowed. I don't know why Stata programmers did not allow noconstant option for random effects model, but that's a different question.
