# How to get the bias-adjusted variance estimates for high-dimensional fixed effects in a linear model?

I want to do a linear regression: $$y_{i}=\psi_{j(i)}+X_{i}^{\prime} \xi+\varepsilon_{i}$$, where $$\psi_{j(i)}$$ is a high dimensional fixed effect on group $$j$$, $$X_i$$ are the covariables of each observation $$i$$, and $$\varepsilon_{i}$$ is error assumed to be exogenous.

This linear regression can be done by many statistic packages through LSMR method avoids inverting the regressor matrix.

However after this regression I want to calculate the size weighted variance of the fixed effects $$\theta_{\psi}=\sum_{j=1}^{J} s_{j}\left(\psi_{j}-\bar{\psi}\right)^{2}$$, where $$s_j$$ is the share of the group. However the simple "plug-in" estimates \begin{aligned} \hat{\theta}_{\psi} =\sum_{j=1}^{J} s_{j}\left(\hat{\psi}_{j}-\hat{\bar{\psi}}\right)^{2} =\sum_{j=1}^{J} s_{j}\left(\hat{\psi}_{j}\right)^{2}-(\hat{\bar{\psi}})^{2} \end{aligned} is upward biased because \begin{aligned} \mathbb{E}\left[\hat{\theta}_{\psi}\right] &=\sum_{j=1}^{J} s_{j} \mathbb{E}\left[\left(\hat{\psi}_{j}\right)^{2}\right]-\mathbb{E}\left[(\hat{\psi})^{2}\right] \\ &=\sum_{j=1}^{J} s_{j}\left\{\psi_{j}^{2}+\mathbb{V}\left[\hat{\psi}_{j}\right]\right\}-(\bar{\psi})^{2}-\mathbb{V}[\hat{\bar{\psi}}] \\ &=\theta_{\psi}+\underbrace{\sum_{j=1}^{J} s_{j} \mathbb{V}\left[\hat{\psi}_{j}\right]-\mathbb{V}[\hat{\bar{\psi}}]}_{\text {bias }} \end{aligned}.

A simple correction is to use the conventional HC standard error: $$\theta_{\psi} \approx \hat{\theta}_{\psi} - \sum_{j=1}^{J} s_{j} \hat{\mathbb{V}}_{H C}\left[\hat{\psi}_{j}\right]$$.

However the high dimensional effects estimated from LSMR method does not provide std errors. And thus I have no idea how to do this correction.

Another way to do correction is to assume $$\mathbb{V}[\varepsilon]=I \sigma^{2}$$ and then direct calculate $$\mathbb{V}[\hat{\psi}]=\left(\tilde{F}^{\prime} \tilde{F}\right)^{-1} \sigma^{2}$$, where $$\tilde{F}$$ is residualized version of $$F$$, and $$F$$ is the dummy matrix of $${\psi}_{j(i)}$$.

However I have no idea how to regress $$F$$ on $$X$$ to get the residual $$\tilde{F}$$ (how can I regress a dummy matrix on 𝑋?) and also how to inverse the $$\left(\tilde{F}^{\prime} \tilde{F}\right)$$.

If you have $$K$$ dummies and say $$n$$ observations, so $$F$$ is $$n \times K$$, you need to regress every dummy in $$F$$ on all variables in $$X$$ separately ($$K$$ regressions). The residuals make up the matrix $$\bar F$$ which is again of dimension $$n \times K$$.