# Event Study vs. Difference-in-Differences

I am trying to perform Event Study methodology to analyse impact of group of events on group of stocks from certain sector. I noticed that I am getting surprisingly lot of positive results (very low p-value like ~0.00). When I sampled set random dates from my time period and run same Event Study analysis I got multiple low p-values again. Approximately 50%-60% per 100 random dates for each company. I assume that high volatility of my companies (and index) might one reason however I need confirmation here. But I have started wondering about different thing. I use same events for all companies from one sector. Not events specific for each company (like m&s for example). So I started wondering whether I should use Difference-in-Differences here? I heard that:

“Difference-in-Differences (DID) is more appropriate for systematic events that affect the whole market while event study is designed to examine impact of events specific for single company”

How true is that statement? As I said before in my analysis I examine impact of same set of events for each company like change in regulations for entire sector. Are there other alternatives beside Event Study and DID?

## 1 Answer

The statement is reasonably accurate. When we talk about the 'standard event study' in economics and financial literature (e.g. MacKinlay, 1997), which based on your text you are, they are not appropriate for analyzing single effects that affect whole market. There are several reasons for this:

1. A key assumption of event study is cross-sectional independence. A cross-sectional independence will be violated when in your sample multiple events happen at the same time. A textbook example would be stock listings occurring at the same day (See Brooks Introductory Econometrics for Finance). Such event clustering makes your estimates of test statistics and consequently $$p$$-values biased and as a result any inference from such event study will be most likely incorrect. If you are analyzing an impact of a single event you will have the most extreme case of event clustering as you are modeling the regulation as a separate event for every firm that happens exactly at the same period of time.

There are some solutions to event clustering. For example, you could solve it by not aggregating across the firms and just examining event impact on firm level do a summary analysis of all individual firm level studies. An alternative would be to create a portfolio of firms that experience the event at the same time and analyze the portfolio as a single firm (see again Brooks). However, in the first case the solution is not very elegant and it has its own problems that are beyond scope of this answer, but you could in principle do it. The second solution is not appropriate because in your case you would end up with a one big portfolio of all firms so you would end up with one aggregate observation.

2. Usually when you are interested in evaluating an effect of policy you do not want to just know if the policy had an effect on one day but if there are also effects that persist. For example, if you evaluate a policy like deworming pills in developed countries on student performance, you dont want to just know if there was one-off effect of this policy on student performance which lasted for few days or months, you would be interesting in knowing what was the effect long term. In such cases using event study is problematic as that would require you to use very long event window. However, this introduces another issues as event study is very sensitive to even small misspecifications if the event window is long (see Brooks again). This is why longer event studies use buy-and-hold abnormal returns (BHAR) instead of just cumulative abnormal returns (CAR) but if your event window is too long you will still have problems.

There are also further issues but the two I mentioned above are in my opinion the major ones when compared to DiD. Also yes there are other alternatives but I think that mentioning them all would be too broad. For example, you could model single event as a structural break or even as regime switch and analyze it that way. It will always depends on the specifics of the problem, for that it is always best to do very careful literature review on the topic and see how other people approach the problem and which issues they consider most important.

• Thank you very much for your answer. However when it comes to your 2nd point: I tried event study so far because I believe some events have sudden but short impact on certain stocks while others have not. This is one thing I wanted to exam: which companies reacts more significant to certain policy changes. Also, how is reaction for changes that were expected and for those that were not. I assume some of them have very short effect while other might be more persistent and want to verify that if possible. Can regular DID be good choice for experiment like that? Sep 8 '20 at 10:16
• @Alexandros well I am bit confused now. Because in your Q is some systemic regulatory event. In that case no matter whether you use DiD or event study you will have to aggregate across the firms so you will loose that information about the heterogenious effect it will have on the firms. Also DiD can control for anticipatory effects so that would not be issue per se and it does not really matter whether effect is short or long DiD would be able to identify it.
– 1muflon1
Sep 8 '20 at 10:27
• @Alexandros if there are more than 1 systemic regulatory event then you would have to do DiD for them separately or maybe if they are close together group them into one aggregate event but it could still be done but if its too many of them it would really not be appropriate solution. In that case you might wanna go back to event study but you will still have the problem of event clustering. In such case maybe some different method altogether would be more appropriate. Maybe you could use bootstrapping then
– 1muflon1
Sep 8 '20 at 10:31
• sorry, my previous comment was messy. I explain better my main issue: I have a set of a companies from one sector. I want to examine, whether certain event (actually set of events, but I look at each event individually) have impact on companies from this sector. All events should affect entire sector. I was trying to perform it on individual companies to know which systematic event had impact on which company. Can it be done by DID? Maybe should I use sector index instead individual companies? Rest I wrote in previous comment is followup analysis, not that important. Sep 9 '20 at 16:52
• @Alexandros if you wanna look it at event by event then yes you can do DiD
– 1muflon1
Sep 9 '20 at 17:00