Regression on individual vs collapsed data

Question

Lots of analysis in the US collapse CPS or Census data and run regressions on the group means. I wonder why not run the regression on the individual data? Absent measurement error, should we expect the regression on individual and aggregated data to be the same?

Here is a Stata example with random data where the regressions at the individual level or on the means seem to deliver different estimates.

EDIT:

Following BB King's comments, I changed the example to include a "shock" at the group level. In this case, it seems like the individual regression 1. and the collapsing on transformed data regression 3. do deliver similar results absent covariates. Is this a general result?

** make some random data
set seed 35135
clear
set obs 1000
gen state = ceil(_n/100)
gen shock = runiformint(0,1000)
bysort state (shock): replace shock = shock[_N]
gen wage = abs(int(rnormal() * 1000))
gen age = floor(abs(rlogistic()) * 15 + abs(rlogistic() * 5))


** 1. individual data
gen age2 = age^2
gen ln_wage = ln(wage)

reg ln_wage age age2 shock
reg ln_wage age age2 shock i.state

** 2. aggregated data
preserve 
    collapse (mean) wage age shock, by(state)
    gen age2 = age^2
    gen ln_wage = ln(wage)

    reg ln_wage age age2 shock
restore

** 3. transform before collapsing
collapse (mean) ln_wage age age2 shock, by(state)
reg ln_wage age age2 shock

Answer 1

In general, regressing at the individual level does not guarantee the same results as at the group level. The results may be the same, depending on how the data was collapsed, but not always.

However, there may be different reasons to regress at the grouped/collapsed level instead of individual.

One reason may be as a robustness check in addition to the individual level regression.

A very common reason, is when the variable of interest, such as the treatment, only varies at the group level and not individual level.

In that case, there is no meaningful individual-level variation for the research question in hand. Since the variable of interest would take the same value for all individuals in one group and a different value for all individuals in another group, all the variation is at the group level.

A danger in that case, an individual level regression would overestimate significance (underestimate standard errors), because you are using more observations, which drives down standard errors, but in fact those observations are meaningless duplicates as far as the variable of interest (e.g. treatment) is concerned. This could be partially addressed with clustering standard errors by group.

As an example, suppose you are analyzing a policy that affects all people in one county and want to compare it to non-treated counties. Then the main variable you are interested in - the treatment - does not vary by each person in that county. The treatment variable has the same value for each individual in any county, so you should collapse by county before regressing.

Another point to consider here is that you may want to do a weighted collapse before the regression, so your collapsed group is more representative. For example, instead of collapsing all firms in a group (e.g. county) and regressing on the average for all firms, you might want to weight the firms by size. That's because larger firms account for a larger share of economic activity, so it may make sense to give them a higher weight than smaller firms, instead of just treating all firms equal in the collapse.

Answer 2

Should we expect the regression on individual and aggregated data to be the same?

Not necessarily. One reason is that data may be aggregated in more than one way, and different aggregations may yield different results, a phenomenon known, for spatial aggregation, as the modifiable areal unit problem. If different aggregations lead to different results, then some at least must yield different results to the individual data.

Answer 3

EDIT: Clearly, the controls were the problem. Without controls, this works fine, but with controls not. We are starting to learn that including controls linearly is not as innocuous as one might think. See for example Słoczyński (2022).

Peter Hull's slides include a section on this, I guess taken from MHE, which argues that if the treatment varies only at the group-level, a weighted regression on means at the group level should give the same results as the micro-level regression (without controls!).

Maybe it has to do with the controls? I can update my code to get a count-weighted regression at the micro- and group-level to deliver the same results without controls (see 4. in the code), but not with the original example that included controls.

** make some random data
set seed 35135
clear
set obs 1000
gen state = ceil(_n/100)
gen shock = runiformint(0,1000)
bysort state (shock): replace shock = shock[_N]
gen wage = abs(int(rnormal() * 1000))
gen age = floor(abs(rlogistic()) * 15 + abs(rlogistic() * 5))


** 1. individual data
gen age2 = age^2
gen ln_wage = ln(wage)

reg ln_wage age age2 shock
reg ln_wage age age2 shock i.state

** 2. aggregated data
preserve 
    collapse (mean) wage age shock (count) count=wage, by(state)
    gen age2 = age^2
    gen ln_wage = ln(wage)

    reg ln_wage age age2 shock [aweight=count]
restore

** 3. transform before collapsing
preserve
    collapse (mean) ln_wage age age2 shock (count) count=wage, by(state)
    reg ln_wage age age2 shock [aweight=count]
restore


** 4. without contrals 
reg wage shock
collapse (mean) wage shock (count) count=wage, by(state)
reg wage shock [aweight=count]