I have a set of active scientists and editors. I want to study the effect that becoming an editor has on the different outcomes of a scientist (such as citation count and publication rate).

I decided to do a difference in difference regression. I'm not sure though is all the steps taken are correct.

I match scientists who did not become editors (control group) to scientists who did become editors later in their career (treatment group) such that they share certain characteristics in the beginning of their career. These characteristics are gender, rank of affiliation, first year of publication (to estimate "era" and "experience"), and discipline.

I define the year in which a scientist becomes an editor as year0. Then I look at the effect becoming an editor has on different outcomes over different years post editorship (year1, year2, ... , year10).

Question 1: Is it correct to have your treatment and control groups matched at the beginning of their career? I read many different opinions on the matter online.

I run a regressions as follows, and explain it as follows:

enter image description here

Question 2: What is the difference between gamma and delta, I know both explain the effect of being an editor, but one is the interaction and one isn't?

This is the way the final table looks like:

enter image description here

The "interaction terms" are the values for delta.

Question 3: Would it be correct to state that:

  • The impact (number of citations) of a scientist increases every year they become an editor
  • The impact of a scientist increases more than three folds by their 10th year as an editor compared to a comparable scientist that did not become an editor?
  • Becoming an editor has a slight negative impact on productivity over the years?
  • 1
    $\begingroup$ I would recommend narrowing the question down and making it less opinion based. I think currently you are asking too many questions at once. Moreover, some questions are opinion based, like asking whether its stated clearly or how would you describe these results (e.g. a better question would be how to interpret the results if you have problem with that). Consider having look at our help center about what kind of questions are on and off topic here. $\endgroup$
    – 1muflon1
    Oct 29 '20 at 18:14
  • $\begingroup$ Thanks. I made the question more specific and less about asking for an opinion. $\endgroup$
    – BKS
    Oct 29 '20 at 18:51
  • $\begingroup$ 'One question per post' is usually the rule. $\endgroup$
    – Mox
    Nov 4 '20 at 1:14

The point of control group matching is to ensure similarity between the groups, to isolate the effect of the treatment variable. If are assuming that the development paths of all academics is identical, and at some point, all academics have a chance to become editors, and some do, then you a reasonable to match at career start.

That's not a good assumption. Academics differ widely in quality/competency/publication/citation over the course of their careers. Much better to find matches by identifying academics that were similar those the became editors, who (for whatever reason) didn't, so that your control group is identical to your treatment group.


Not the answer you're looking for? Browse other questions tagged or ask your own question.