Dealing with Missing Values in Diff-in-Diff Estimation

Question

To preface this, I am asking this question on the Econ SE because I was made aware on Cross Validated that Difference in Difference estimation is quite an economics specific method.

The picture above depicts deviations from Covered Interest Parity, for the Yen, the Euro and Pound Sterling. The vertical gray bars mark quarter-end reporting dates. My plan was now to apply a Difference in Difference estimation in order to investigate if larger deviations from CIP coincide with quarter ends after January 2015. The reason behind suspected quarter-end dynamics is the change in reporting standards for global banks in 2015. From 2015 onwards European banks are coerced to report a snapshot of their balance sheet on the last day of each quarter. For U.S. banks a similar report is required, however in the form of the average of the month ends. The reasoning behind this hypothesis is that said reporting standards coupled with capital requirements cause balance sheet costs and therefore lead to a decrease of banks engaging in arbitrage.

This paper by DU et. al., which applies a difference in difference estimation, investigates the above-explained dynamics: https://onlinelibrary.wiley.com/doi/full/10.1111/jofi.1262

As can be seen in the graph the three time-series that I could acquire, have different missing values. Consequently, I am unsure of how to proceed. The procedure applied in the paper does not seem like it actually requires a panel, please correct me here if you disagree. What leads me to believe that a panel is not actually required here, is based on the fact that the treated and untreated group are not as normally the case two different parts of the panel, in this case they are treated and untreated groups, at quarter-end/not at quarter-end and prior to 2015/after 2015 in all three time-series.

Hence this results in two questions: Is it possible to build a panel considering that the three time-series have missing values at different points in time without losing every observation that is missing for one of the three currencies?

Is it possible to estimate this difference-in-difference for each currency by itself, considering the facts I have pointed out above?

Model:

$x_{1w,it}= \alpha_0 +\gamma_1Post15_t+ \beta_{1}QendW_{t}+\beta_{2}QendW_{t} \times Post15_t + \epsilon_{it}$

Where $QendW$ equals 1 if the settlement date of the contract is within a reporting date, $Post15$ equals 1 after the European Leverage Ratio Delegated Act was established, and zero otherwise.

I am very grateful for any tips or hints, thank you!

Answer 1

Just to make sure I understand:

1. You have a daily panel (with missing values), probably weekday only, running from 2013 to mid-2017 with $n=3$ cross-sectional units.
2. You believe that after 2015, there is a stronger incentive to do something different at quarter end, so the average basis should be different on month end days during that period.

The typical difference-in-difference regression compares two types of units, some that are treated and some that are not, before and after the start of treatment. You don't have that here, but I think that's okay, because you are instead looking at the difference between quarter end and the rest of the quarter, pre- and post-.

As a general matter, this setup is fine. As long as the missing-ness of the data is "missing completely at random" or "missing at random". The problem occurs when the pattern of missing data reflects selection on the LHS variable, omitted variables causing changes in the LHS, or the value of the variable that's missing is related to the reason it is missing.

Another concern would be empty cases. You want to make sure that you have quarter end and quarter not end observations for the pre- and post-period. If you use of unit FE, you want to make sure that you have data pre-/post-, quarter end / not, for each unit.

Make sure you are also satisfying the requirements of the difference-in-difference setup. Some are un-testable, but at a minimum you should make a graphical or statistical argument that your data satisfy the common trend assumption.

Here is some sample Python code that shows that you can recover the initial parameters in your setup. I did it with month end, not quarter end, because it makes it easier to have a large enough sample of missing data for both difference variables. It has an example of MAR and MCAR. It shows it works for both.

    import numpy as np
    import pandas as pd
    import statsmodels.formula.api as smf
    import itertools
    np.random.seed(9876789)

    startyear=2013
    endyear=2017
    daysinmonth = 21
    nummonths = 12
    numgroups = 3
    numyears = (endyear-startyear+1)
    dropfraction = 0.20
    # np.sort(np.tile(np.arange(2013,2018), 12*21))
    years   = np.repeat(np.arange(startyear,endyear+1), nummonths*daysinmonth*numgroups )
    days    = np.tile(np.arange(1,daysinmonth+1), nummonths*numyears*numgroups )
    months  = np.tile(np.repeat(np.arange(1,nummonths+1), daysinmonth),  numyears*numgroups)
    groups  = np.tile(np.repeat(np.arange(1,(numgroups+1)),nummonths*daysinmonth), numyears)
    df = pd.DataFrame({'year': years, 'month': months, 'day': days, 'group': groups})


    # Last weekday is monthend or the 21st 
    df['endofmonth'] = df.day == 21
    df['QendW'] = 1*df['endofmonth']
    df['Post15'] = 1* (df.year > 2015)
    df['random'] = np.random.rand(daysinmonth*nummonths*numyears*numgroups)
    df['missing_completely_at_random'] = df['random'] <= dropfraction
    df['missing_at_random'] = (df['random'] - 0.05 * df['Post15'])  <= dropfraction
    df['QendWxPost15'] = df['QendW'] * df['Post15']

    alpha0 =  0.2
    gamma1 =  0.05
    beta1  = -0.10
    beta2  =  0.15
    df['error'] = np.random.randn(daysinmonth*nummonths*numyears*numgroups) / 10

    df['x1'] = alpha0 + gamma1 * df['Post15'] + beta1 * df['QendW'] + beta2 * df['QendWxPost15'] +  df['error']

    df['x1_missing_completely_at_random'] = df['x1'].copy()
    df.loc[df['missing_completely_at_random'] == True, 'x1_missing_completely_at_random'] = np.nan

    df['x1_missing_at_random'] = df['x1'].copy()
    df.loc[df['missing_at_random'] == True, 'x1_missing_at_random'] = np.nan

    print("No missing data")
    model = smf.ols(formula='x1 ~ Post15 + QendW + QendWxPost15', data=df)
    results = model.fit()
    print(results.summary())


    print("Missing Completely At Random (MCAR)")
    model_missing_completely_at_random = smf.ols(formula='x1_missing_completely_at_random ~ Post15 + QendW + QendWxPost15', data=df)
    results_missing_completely_at_random = model_missing_completely_at_random.fit()
    print(results_missing_completely_at_random.summary())

    print("Number missing quarter end (MCAR)")
    print(np.sum(df.loc[df['missing_completely_at_random'] == True, 'QendW'] ))
    print("Number missing quarter end post 2015 (CAR)")
    print(np.sum(df.loc[df['missing_completely_at_random'] == True, 'QendWxPost15'] ))


    print("Missing At Random (MAR)")
    model_missing_at_random = smf.ols(formula='x1_missing_at_random ~ Post15 + QendW + QendWxPost15', data=df)
    results_missing_at_random = model_missing_at_random.fit()
    print(results_missing_at_random.summary())

    print("Number missing quarter end (MAR)")
    print(np.sum(df.loc[df['missing_at_random'] == True, 'QendW'] ))
    print("Number missing quarter end post 2015 (MAR)")
    print(np.sum(df.loc[df['missing_at_random'] == True, 'QendWxPost15'] ))