0
$\begingroup$

I was thinking about running a diff-in-diff with fixed effect in order to deal with a panel data experiment. The problem is that I don't know how many datapoints I need in order to the experiment be reliable.

This is the thing: I want to see how my customers react to a change in price. An AB test doesn't look like a good alternative, due to cannibalization. So I would like to change the price of a bundle of products in some cities. Let's say for cities A, B, C and D, cleaning products will be 10% cheaper. Then I would run a diff-in-diff with fixed effects using each city datapoint. The question is: how many cities do I need in order to run this experiment? How many sales each city would have to have in order to the experiment be reliable?

$\endgroup$
0
$\begingroup$

Diff-in-Diff is a parametric model. A rule of thumb for parametric models is that you should have at least 25-30 observations (different authors might disagree but it is around 30) per independent regressor used in your model (see discussion in Verbeek A Guide to Modern Econometrics pp 36). However, note this is a rule of thumb and as Verbeek points out (my comments are in []):

Unfortunately, there is no unambiguous definition of what is ‘sufficiently large’ [talking about sample sizes to justify asymptotic properties]. In simple circumstances a sample size of 30 may be sufficient, whereas in more complicated or extreme cases a sample of 1000 may still be insufficient for the asymptotic approximation to be reasonably accurate.

This being said getting precise number for every individual case would be too difficult (it would involve Monte Carlo simulations) so often people follow the rules of thumb unless they have suspicion something is awry.

Consequently, if you would just follow the rule of thumb in a diff-in-diff model with 5 independent regressors you should have at least 150 observations, in model with 7 independent regressors 210, in model with 10 independent regressors 300 and so on.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.