6
$\begingroup$

Lots of analysis in the US collapse CPS or Census data and run regressions on the group means. I wonder why not run the regression on the individual data? Absent measurement error, should we expect the regression on individual and aggregated data to be the same?

Here is a Stata example with random data where the regressions at the individual level or on the means seem to deliver different estimates.

EDIT:

Following BB King's comments, I changed the example to include a "shock" at the group level. In this case, it seems like the individual regression 1. and the collapsing on transformed data regression 3. do deliver similar results absent covariates. Is this a general result?

** make some random data
set seed 35135
clear
set obs 1000
gen state = ceil(_n/100)
gen shock = runiformint(0,1000)
bysort state (shock): replace shock = shock[_N]
gen wage = abs(int(rnormal() * 1000))
gen age = floor(abs(rlogistic()) * 15 + abs(rlogistic() * 5))


** 1. individual data
gen age2 = age^2
gen ln_wage = ln(wage)

reg ln_wage age age2 shock
reg ln_wage age age2 shock i.state

** 2. aggregated data
preserve 
    collapse (mean) wage age shock, by(state)
    gen age2 = age^2
    gen ln_wage = ln(wage)

    reg ln_wage age age2 shock
restore

** 3. transform before collapsing
collapse (mean) ln_wage age age2 shock, by(state)
reg ln_wage age age2 shock
$\endgroup$
5
  • $\begingroup$ I think the answer will depend on the research question and empirical strategy, so more context would be useful. $\endgroup$
    – BB King
    Mar 30, 2023 at 13:21
  • $\begingroup$ Not sure. I am asking in general about the difference between a regression on individual data, versus a regression on the group and whether the two give the same result. As an example, I am making up some state-level wage data to run a Mincer regression. $\endgroup$
    – Papayapap
    Mar 30, 2023 at 15:18
  • 1
    $\begingroup$ So in general they do not guarantee the same results. But if you're asking why some people do them at collapsed level instead of individual, it will depend on the context on what they want. It could be a robustness check, or it could be that the variation of their treatment is only at the group level and not individual level, so there is no meaningful individual-level variation. Suppose you are analyzing the policy that affects all people in one county- then the main variable you are interested in does not vary by person in that county- its all the same value - so you collapse by county. $\endgroup$
    – BB King
    Mar 30, 2023 at 16:28
  • $\begingroup$ That's a good point. In the literature I am thinking about, indeed, the "treatment" is at the group level. $\endgroup$
    – Papayapap
    Mar 31, 2023 at 7:13
  • $\begingroup$ Ok since this is helpful, let me put it as an answer. $\endgroup$
    – BB King
    Mar 31, 2023 at 13:06

3 Answers 3

4
$\begingroup$

In general, regressing at the individual level does not guarantee the same results as at the group level. The results may be the same, depending on how the data was collapsed, but not always.

However, there may be different reasons to regress at the grouped/collapsed level instead of individual.

One reason may be as a robustness check in addition to the individual level regression.

A very common reason, is when the variable of interest, such as the treatment, only varies at the group level and not individual level.

In that case, there is no meaningful individual-level variation for the research question in hand. Since the variable of interest would take the same value for all individuals in one group and a different value for all individuals in another group, all the variation is at the group level.

A danger in that case, an individual level regression would overestimate significance (underestimate standard errors), because you are using more observations, which drives down standard errors, but in fact those observations are meaningless duplicates as far as the variable of interest (e.g. treatment) is concerned. This could be partially addressed with clustering standard errors by group.

As an example, suppose you are analyzing a policy that affects all people in one county and want to compare it to non-treated counties. Then the main variable you are interested in - the treatment - does not vary by each person in that county. The treatment variable has the same value for each individual in any county, so you should collapse by county before regressing.

Another point to consider here is that you may want to do a weighted collapse before the regression, so your collapsed group is more representative. For example, instead of collapsing all firms in a group (e.g. county) and regressing on the average for all firms, you might want to weight the firms by size. That's because larger firms account for a larger share of economic activity, so it may make sense to give them a higher weight than smaller firms, instead of just treating all firms equal in the collapse.

$\endgroup$
2
  • $\begingroup$ Thanks! Are we overestimating significance though? We want to study the impact of a shock on the outcomes of people. It's not clear to me the unit of observation should be the aggregate. The group-level treatment may be the same, but the outcome varies within group. When we collapse we loose the within-group variation. $\endgroup$
    – Papayapap
    Apr 1, 2023 at 9:48
  • 1
    $\begingroup$ Yes but there’s no meaningful within-group variation if your variable of interest (e.g. treatment) doesn’t vary within groups. $\endgroup$
    – BB King
    Apr 1, 2023 at 14:09
4
$\begingroup$

Should we expect the regression on individual and aggregated data to be the same?

Not necessarily. One reason is that data may be aggregated in more than one way, and different aggregations may yield different results, a phenomenon known, for spatial aggregation, as the modifiable areal unit problem. If different aggregations lead to different results, then some at least must yield different results to the individual data.

$\endgroup$
4
$\begingroup$

EDIT: Clearly, the controls were the problem. Without controls, this works fine, but with controls not. We are starting to learn that including controls linearly is not as innocuous as one might think. See for example Słoczyński (2022).

Peter Hull's slides include a section on this, I guess taken from MHE, which argues that if the treatment varies only at the group-level, a weighted regression on means at the group level should give the same results as the micro-level regression (without controls!).

enter image description here

enter image description here

Maybe it has to do with the controls? I can update my code to get a count-weighted regression at the micro- and group-level to deliver the same results without controls (see 4. in the code), but not with the original example that included controls.

** make some random data
set seed 35135
clear
set obs 1000
gen state = ceil(_n/100)
gen shock = runiformint(0,1000)
bysort state (shock): replace shock = shock[_N]
gen wage = abs(int(rnormal() * 1000))
gen age = floor(abs(rlogistic()) * 15 + abs(rlogistic() * 5))


** 1. individual data
gen age2 = age^2
gen ln_wage = ln(wage)

reg ln_wage age age2 shock
reg ln_wage age age2 shock i.state

** 2. aggregated data
preserve 
    collapse (mean) wage age shock (count) count=wage, by(state)
    gen age2 = age^2
    gen ln_wage = ln(wage)

    reg ln_wage age age2 shock [aweight=count]
restore

** 3. transform before collapsing
preserve
    collapse (mean) ln_wage age age2 shock (count) count=wage, by(state)
    reg ln_wage age age2 shock [aweight=count]
restore


** 4. without contrals 
reg wage shock
collapse (mean) wage shock (count) count=wage, by(state)
reg wage shock [aweight=count]
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.