Diff-in-diff with different treatment years, what would be time dummy for control?

Question

I want to estimate the impact that replications have on the citations. For that, I want to make a staggered diff-in-diff, comparing replicated papers vs non-replicated ones.

In my data set I have around 80 papers that were replicated (therefore my treatment group), and 160 that were never replicated. To ensure comparability, I took only empirical papers that were published in the same journals, volumes, issues, and about the same topics or JEL code.

My date looks sth like this:

My supervisor suggested to start with a "simple" diff in diff,to see some initial effect and then proceed to do the staggered version (and probably th a Poisson regression since my dependent variable is a non-negative count number).

For the diff in diff, my treatment dummy is "replicated", which is 1 for replicated papers and 0 for the rest. And my problem/question is with my time dummy d_time, because: as you can see, my treated observations have different treatment years. In this example, one was treated in 2021 and the other one in 2018. But I have 80 papers that were replicated in total, so each was replicated in different years. So, there is a before and after for the control group, but there is no specific before and after for all the treatment so I don't know what to compare against.

Would it be ok, that my time dummy d_time, takes the values of 0 for all my control ones? However, I think is because of this that I get collinearity in my first results:

Am I doing something completly wrong? Could someone share some light for me? I'm very new at this, I apologize if this is not clear but I hope it is.

EDIT

The suggested "simple" Diff-in-Diff would look something like this:

Would it be ok for the control group (replicated=0), to have a 0 in the post_rep column? while only the treated group (replicated=1) does have 0 and 1? Would it make sense to make such an analysis?

Answer 1

Assume you have a dataset $\{Y_{it},X_{it},D_{it}\}$ where $Y_{it}$ is the dependent variable, $X_{it}$ are control variables and $D_{it}$ is an indicator which for observational unit $i$ is equal to $1$ for all $t \geq t^\star_i$ and $0$ for all $t<t^\star_i$, where $t^\star_i$ is the time period where unit $i$ receives treatment.

We then assumed that there would be no dropout of treatment after receiving it (in the diff-and-diff literature this is referred to as treatment being an absorbing state).

In the simple diff-and-diff you have a control group that never receives treatment $D_{it}=0$ for all $t$ for all $i \in \mathcal G_c$ where $\mathcal G_c$ is the group of observational units in the control group. And a group $\mathcal G_{t^\star}$ of observational units that are treated at time $t^\star$.

In this case you can simply regress

$$(1) \ \ \ Y_{it} = \delta_t + \mu_g + \lambda D_{it} + \beta X_{it} + \epsilon_{it},$$

where $\delta_t$ is time fixed effects and $\mu_g$ are group fixed effects.

It would help if you played around with these simple diff-and-diff setups. This will give you a feel about how data is behaving. Also, it will allow you to experiment with constructing $\mathcal G_c$ in different ways.

You can perfectly well estimate equation (1) with several groups that are treated at different times. There is no post_rep variable whose definition depends on the non-ambiguity of treatment timing. Instead, the specification uses time dummies.

However, estimating (1) has been criticized in a newer strand of diff-and-diff literature. The specification imposes that (a) all units have the same treatment effect, and (b) the treatment has the same effect regardless of how long it has been since treatment started. You can have a look at this review paper:

What’s Trending in Difference-in-Differences? A Synthesis of the Recent Econometrics Literature

These slides explain simple 2x2 diff and diff and then go on to the literature on staggered diff-and-diff

Diff-and-diff slides

The short story is that I cannot tell you what to do if you don't buy the assumptions (a) and (b) because in that case there are multiple ways to move on. However, the simplest way of allowing for treatment heterogeneity is simply to run simple diff-and-diff specifications for the different treated groups whose treatment you believe to be different.

You can always raise a new question once you have mastered the basics.