0
$\begingroup$

I have two panels. One panel that consists of an economic indicator variable across firms and years. The second panel consists of the same economic indicator variable across the same firms and years. However, the indicator variable in the second panel is calculated on the basis of different data than the one in the first panel, I call that data "adjusted". Hence, the values of the variable in the second panel data differ from that in the first panel. Put differently, I have two panels of paired observations of a single variable.

I now want to assess wether the indicators based on the adjusted data are significantly larger than the ones based on the unadjusted data.

Or simply: Are the values in one panel larger than the values in the other panel that describes an alternative set of values of the same variable for the same firms and years.

One issue to keep in mind is that the indicator variables are autocorrelated by construction, since they are computed on a rolling (overlapping) window basis. What I have come up with is as follows:

Step 1: I calculate the paired differences, i.e. $\delta_{i,t} = x_{i,t} - y_{i,t}$, where $x_{i,t}$ is the economic indicator based on adjusted data and $y_{i,t}$ is the economic indicator based on unadjusted data.

Step 2: I estimate a fixed effects model on a constant only, i.e. $\delta_{i,t} = \alpha + \gamma_{i} + \epsilon_{i,t}$, where $\alpha$ is the constant and $\gamma_{i}$ is a firm fixed effect.

I test the hypothesis that $\alpha > 0$ against the null hypothesis $\alpha = 0$ using the usual t-test. I employ robust standard errors to account for the autocorrelation that I expect in the $\delta_{i,t}$.

If the $\alpha$ is significantly larger than $0$, I assume that the $x_{i,t}$ is, on average, larger than $y_{i,t}$.

I am not an expert in panel regression, so my question is: Is this model design reasonable or flawed? Moreover, are there "better" ways to assess my question of interest?

--------------------------------------------------------------------------------

EDIT 1: What I mean by paired samples is that the observations are paired, i.e. they are somewhat linked because they refer to the same {i,t}. That is why I compute the differences of each "pair" (paired t-test).

EDIT 2: I actually analyse the quality of reported earnings. The one panel consists of quality metrics that are calculated based on actual reported earnings ($y_{i,t}$). The other panel consists of quality metrics that are based on "alternative" earnings values ($x_{i,t}$). What I mean by that is: For example, one could hypothetically imagine a firm in the US applying different accounting standards (e.g. German accounting standards), which leads to different values of reported earnings of the same firm in the same period. Now simply imagine that the second panel are earnings quality metrics that are based on earnings quantities reported under a set of different accounting regulations than the one in the first panel, but for the same years and firms!

EDIT 3: The panels consists of around 500 firms and 10 years each. I want to conduct the test for the entire sample. That is, I have 500 $\delta_{i,t}$ (500 firm-year differences).

$\endgroup$
  • $\begingroup$ I'm not sure what you mean by "panels are paired" and "adjusted versus unadjusted data". It might be useful to clarify it. $\endgroup$ – emeryville Apr 25 at 11:03
  • $\begingroup$ To devise a plan to solve the problem the following information is necessary: 1. How many countries do you have? 2. How many years do you have? 3. For many variables (y) do you want to do the test? Statistical power and p-value adjustments will be affected by all of the above. $\endgroup$ – Grada Gukovic Apr 25 at 11:41
  • $\begingroup$ In step 1 shouldn't it say $\delta_{i,j,t} = x_{i,jt} - y_{i,j,t}$. $i$ - country, $j$- variable, $t$ - year. $\endgroup$ – Grada Gukovic Apr 25 at 11:44
  • $\begingroup$ See the adjustments I just made to make it more clear. I made a mistake, its not countries I am referring to, its firms! So $i$ denots a firm and not a country - sorry. $\endgroup$ – shenflow Apr 25 at 11:46
  • $\begingroup$ @ EDIT 2 - the adjustments are not of statistical nature, but when you do multiple testing you have to adjust the p-value for the number of tests. The most appropriate method for controlling the family-wise error rate depends on the size of the data. $\endgroup$ – Grada Gukovic Apr 25 at 11:46
2
$\begingroup$

The constant term in your final FE model has no specific meaning without further restrictions. For Stata, it is only chosen such that the (sample) mean of the estimated individual effects add up to 0. So your testing $\alpha>0$ is in fact based on $$\frac{1}{n} \sum_{i=1}^n \hat\alpha_i,\quad \hat\alpha_i = \frac{1}{T_i} \sum_{t=1}^{T_i} (x_{it} - y_{it}),$$ where I am using the $T_i$ notation in order to allow for the panel data to be unbalanced.

For balanced panel data, the test is (almost) identical to one based on OLS if you use a cluster standard error for OLS because then $$\frac{1}{n} \sum_{i=1}^n \hat\alpha_i = \frac{1}{n} \sum_{i=1}^n \frac{1}{T} \sum_{t=1}^T (x_{it}-y_{it}) = \frac{1}{nT} \sum_{i=1}^n \sum_{t=1}^T (x_{it} - y_{it}),$$ where the LHS is a statistic from the FE regression and the RHS is the corresponding one from the OLS regression. OLS, FE and RE (and BE) will all give an identical intercept estimate, I think. Only standard errors will be different. If the panel data set is unbalanced, the method using the FE regression and that using the OLS regression differ in how the average differences ($\hat\alpha_i$) are weighted across different $i$.

The test you mention is about whether the mean of $x_{it}-y_{it}$ is bigger than 0 or not. This looks OK. What else? Which technique to use (OLS or FE or RE or BE) doesn't seem to be critical here. I would just use OLS + cluster se, as it is simple to both the researcher and the reader. Try:

xtset id year
gen d = x - y
reg d, vce(cl id)
xtreg d, fe
xtreg d, re
xtreg d, be
| improve this answer | |
$\endgroup$
  • $\begingroup$ You are right about the interpretion of the intercept I think. I think I will simply go with the OLS model and the clustered SE, as you suggested. In a more detailed analysis, do you think it is possible to somehow incorporate the fact that not every value $x_{i,t}$ will differ from $y_{i,t}$ because not every data point that they are based on is adjusted? Because as far as I understand, my estimator of $\alpha$ is biased towards zero at the moment (i.e. if I dont incorporate that detail in my analysis). $\endgroup$ – shenflow Apr 26 at 6:51
  • $\begingroup$ If whether being adjusted is determined exogenously (nonrandom), then you can do the test using only adjusted observations. But that's the same as restricting the population to those adjusted. If you want to see the whole population, and if 99% don't adjust, then the test stat will be close to 0 naturally, but it does not mean a bias. Anyway, it's all a matter of how you define your null hypothesis. Is it E(x)>E(y)? Then what's the meaning of E(), etc. Try to clarify that, before worrying about which techniques to use. $\endgroup$ – chan1142 Apr 26 at 15:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.