How to get the bias-adjusted variance estimates for high-dimensional fixed effects in a linear model?

Question

I want to do a linear regression: $y_{i}=\psi_{j(i)}+X_{i}^{\prime} \xi+\varepsilon_{i}$, where $\psi_{j(i)}$ is a high dimensional fixed effect on group $j$, $X_i$ are the covariables of each observation $i$, and $\varepsilon_{i}$ is error assumed to be exogenous.

This linear regression can be done by many statistic packages through LSMR method avoids inverting the regressor matrix.

However after this regression I want to calculate the size weighted variance of the fixed effects $$\theta_{\psi}=\sum_{j=1}^{J} s_{j}\left(\psi_{j}-\bar{\psi}\right)^{2}$$, where $s_j$ is the share of the group. However the simple "plug-in" estimates $$\begin{aligned} \hat{\theta}_{\psi} =\sum_{j=1}^{J} s_{j}\left(\hat{\psi}_{j}-\hat{\bar{\psi}}\right)^{2} =\sum_{j=1}^{J} s_{j}\left(\hat{\psi}_{j}\right)^{2}-(\hat{\bar{\psi}})^{2} \end{aligned}$$ is upward biased because $$\begin{aligned} \mathbb{E}\left[\hat{\theta}_{\psi}\right] &=\sum_{j=1}^{J} s_{j} \mathbb{E}\left[\left(\hat{\psi}_{j}\right)^{2}\right]-\mathbb{E}\left[(\hat{\psi})^{2}\right] \\ &=\sum_{j=1}^{J} s_{j}\left\{\psi_{j}^{2}+\mathbb{V}\left[\hat{\psi}_{j}\right]\right\}-(\bar{\psi})^{2}-\mathbb{V}[\hat{\bar{\psi}}] \\ &=\theta_{\psi}+\underbrace{\sum_{j=1}^{J} s_{j} \mathbb{V}\left[\hat{\psi}_{j}\right]-\mathbb{V}[\hat{\bar{\psi}}]}_{\text {bias }} \end{aligned}$$.

A simple correction is to use the conventional HC standard error: $$\theta_{\psi} \approx \hat{\theta}_{\psi} - \sum_{j=1}^{J} s_{j} \hat{\mathbb{V}}_{H C}\left[\hat{\psi}_{j}\right]$$.

However the high dimensional effects estimated from LSMR method does not provide std errors. And thus I have no idea how to do this correction.

Another way to do correction is to assume $\mathbb{V}[\varepsilon]=I \sigma^{2}$ and then direct calculate $$\mathbb{V}[\hat{\psi}]=\left(\tilde{F}^{\prime} \tilde{F}\right)^{-1} \sigma^{2}$$, where $\tilde{F}$ is residualized version of $F$, and $F$ is the dummy matrix of ${\psi}_{j(i)}$.

However I have no idea how to regress $F$ on $X$ to get the residual $\tilde{F}$ (how can I regress a dummy matrix on 𝑋?) and also how to inverse the $\left(\tilde{F}^{\prime} \tilde{F}\right)$.

Answer 1

I guess this follows from the Frish-Waugh Lovell theorem.

If you have $K$ dummies and say $n$ observations, so $F$ is $n \times K$, you need to regress every dummy in $F$ on all variables in $X$ separately ($K$ regressions). The residuals make up the matrix $\bar F$ which is again of dimension $n \times K$.