How clustering affects standard error?

Question

In panel data, when adding covariates, the standard error normally decreased, I am wondering whether standard error when clustering higher or lower comparing to without clustering.

Answer 1

It can either increase or decrease when clustering, it depends on the covariance structure of the error terms. It is more frequent to observe standard errors increase with clustering.

Answer 2

To further elaborate on the answer of @Michael Gmeiner, the difference in the estimate of the variance between the model that takes into account clustering and the model without cluster is usually expressed with what is called, Moulton factor. See also Moulton (1986), Random group effects and the precission of regression estimates.

Consider the basic case with $G$ groups (clusters) indexed by $g$. Assume that there are in total $N$ observations with $N_g$ observations in group $g$. Individual observations are indexed by $i$. Consider the following simple regression with a single covariate: $$ y_{i,g} = \alpha + \beta x_{i,g} + \varepsilon_{i,g}. $$ We would like to see how the estimated variance of the OLS estimate of $\beta$ differs when we take intra group correlation (clustering) into account and when not.

To model intra-group correlation, take the case where for $i \ne j$ and $g \ne g'$, $$ \begin{align*} &\mathbb{E}(\varepsilon_{i,g}) = 0,\\ &\mathbb{E}(\varepsilon_{i,g}^2) = \sigma^2,\\ &\mathbb{E}(\varepsilon_{i,g}\varepsilon_{j,g'}) = 0,\\ &\mathbb{E}(\varepsilon_{i,g}\varepsilon_{j,g}) = \sigma^2 \rho. \end{align*} $$ So there is no correlation of the residuals across groups $(g \ne g')$ but there is correlation of the residuals within each group, denoted by $\rho$. We assume that $\sigma^2$ and $\rho$ is the same for all groups. Given the residuals $\hat \varepsilon_{i,g}$ of the OLS, we can estimate these using: $$ \begin{align*} &\hat \sigma^2 = \sum_{g} \sum_{i \in g} \hat \varepsilon_{i,g}^2,\\ &\hat \rho = \frac{\sum_{g} \sum_{i \in g} \sum_{j \in g, j \ne i} \varepsilon_{i,g} \varepsilon_{j,g}}{\hat \sigma^2 \sum_g N_g(N_g - 1)} \end{align*} $$ The OLS estimate $\hat \beta$ of $\beta$, is given as follows: $$ \hat \beta = \frac{\sum_{g} \sum_{i \in g} (y_{i,g}- \overline{y})(x_{i,g}-\overline{x})}{\sum_g \sum_{i} (x_{i,g} - \overline{x})^2} $$ Here $\overline{x}$ and $\overline{y}$ are the sample averages of $x$ and $y$. Substituting $y_{i,g} = \alpha + \beta x_{i,g} + \varepsilon_{i,g}$ gives: $$ \hat \beta - \beta = \frac{\sum_g \sum_{i \in g}(\varepsilon_{i,g})(x_{i,g} - \overline{x})}{\sum_g\sum_i (x_{i,g} - \overline{x})^2} $$ Notice that: $$ var(\hat \beta) = \mathbb{E}(\hat \beta -\beta)^2 = \mathbb{E}\left(\frac{\left(\sum_{g} \sum_{i \in g} \varepsilon_{i,g} (x_{i,g}- \overline{x})\right)^2}{\left(\sum_g \sum_{i \in g} (x_{i,g} - \overline{x})^2\right)^2}\right) \tag{1} $$ We would like to estimate this expression. Define the inter-group (sample) correlation of $x$: $$ \hat \rho_x = \frac{\sum_g \sum_{i \in g} \sum_{j \in g, i \ne j} (x_{i,g} - \overline{x})(x_{j,g} - \overline{x})}{\sum_g N_g (N_g - 1)} \frac{1}{\frac{1}{N}\sum_g \sum_i (x_{i,g} - \overline{x})^2 },\\ \to \sum_g \sum_{i \ne j} (x_{i,g} - \overline{x})(x_{j,g} - \overline{x}) = \hat \rho_x \sum_g \sum_i (x_{i,g} - \overline{x})^2 \frac{1}{N} \sum_g N_g(N_g -1) $$

Which is the sample estimator of $corr(x_{i,g}, x_{j,g})$. The numerator of the expression $(1)$ can then be estimated by: $$ \begin{align*} &\sum_g \sum_{g'} \sum_{i \in g} \sum_{j \in g'} \hat \varepsilon_{i,g} \hat \varepsilon_{j,g'} (x_{i,g}-\overline{x}) (x_{j,g'}-\overline{x})\\ &\approx \hat \sigma^2 \sum_g \sum_{i \in g} (x_{i,g}- \overline{x})^2 + \hat \sigma^2 \hat \rho \sum_{g} \sum_{i \in g} \sum_{j \in g, i \ne j} (x_{i,g}-\overline{x})(x_{j,g} - \overline{x}),\\ &= \hat \sigma^2 \sum_g \sum_{i \in g} (x_{i,g} - \overline{x})^2 + \hat \sigma^2 \hat \rho \hat \rho_x \frac{1}{N} \sum_{i}(x_{i,g} - \overline{x})^2 \sum_g N_g(N_g-1),\\ &=\hat \sigma^2 \sum_g \sum_i (x_{i,g} - \overline{x})^2\left[1 + \left(\frac{1}{N} \sum_g N_g^2 - 1\right)\hat \rho \hat \rho_x \right] \end{align*} $$ Next, observe that: $$ \begin{align*} var(N_g) &= \frac{1}{G}\sum_g N_g^2 - \frac{N^2}{G^2},\\ &\to \frac{1}{N}\sum_g N_g^2 = \frac{var(N_g)}{\overline{m}} + \overline{m} \end{align*} $$ where $\overline{m}$ is the average group size. Then we have that: $$ \mathbb{E}(\hat \beta - \beta)^2 \approx \frac{\hat \sigma^2 \sum_g \sum_{i \in g} (x_{i,g} - \overline{x})^2 \left(1 + \hat \rho\hat \rho_x \left(\frac{var(N_g)}{\overline{m}} + \overline{m} -1\right) \right)}{\left(\sum_g \sum_i (x_i - \overline{x})^2\right)^2} \tag{2} $$ If we would disregard the clustering, i.e. set $\hat \rho = 0$, we would obtain the estimator: $$ \mathbb{E}(\hat \beta - \beta)^2 \approx \frac{\hat \sigma^2 \sum_g \sum_i (x_{i,g} - \overline{x})^2}{\left(\sum_g \sum_i (x_{i,g} - \overline{x})^2\right)^2} \tag{3} $$ The ratio of these two estimators $(2)$ and $(3)$ is then: $$ \tau = 1 + \left[\frac{var(N_g)}{\overline{m}} + \overline{m} - 1\right]\hat \rho \hat \rho_x $$ This is called the Moulton factor. So if $V(\hat \beta)$ is the estimated variance for the clustered model and $\hat V(\hat \beta)$ is the one for the model that disregards clustering, we have: $$ V(\hat \beta) = \tau\,\, \hat V(\hat \beta). $$ We see that $\tau > 1$ if $\hat \rho, \hat \rho_x > 0$ which will be the case if there is positive intra-group correlation. If the intra-group correlation is negative, but $\rho_x > 0$, we have that $\tau < 1$. so clustering will decrease the variance as menioned by @Michael Gmeiner

Now, consider the special case where $x_{i,g}$ only varies over groups. So $x_{i,g} = x_g$ for all $i \in g$ and where all groups have the same size. Then we have that $\hat \rho_x = 1$, $var(N_g) = 0$ and $\overline{m} = N_g$. Then: $$ \tau = 1 + (N_g - 1)\hat \rho. $$ So the increase in the estimated variance will be increasing in the intra-group correlation and the number of observations in each group. In general we can expect that $\hat \rho > 0$ so clustered variances will be higher than the standard estimate that does not take into account clustering.