How to perform unbalanced panel data regression in R?

Question

I am attempting to perform an unbalanced panel data regression in R. My code is as follows:

pdata <- plm.data(b2, index = c("ticker", "year"))
try1 <- plm(formula = logDipLoanTotal ~ PrimeFiling  + 
logLiabBefore + logSalesBefore + EmplBefore + DebttoAssetRatio + 
squareDebttoAssetRatio + FSCashShortTermInvestments + Percent + 
NUM_OF_EMPLOYEES, data = pdata, model = "within", effect = "time")
summary(try1)

I run into the following error:

duplicate couples (id-time) in resulting pdata.frame
to find out which, use e.g. table(index(your_pdataframe), useNA = "ifany")
duplicate couples (id-time)

This is obvious - I am having issues because my panel data is unbalanced, where the ticker/year combination can be associated with multiple lines of data. This is because multiple "deals" were done in the year associated with the company ticker in the year, or sometimes no deals were run in the ticker/year combination. How do I run this kind of model in RStudio with unbalanced data? Let me know if this doesn't make sense and I can edit the question.

Answer 1

Here the solution would depend on what you want to accomplish. Note the problem is not just that the series is unbalanced, for an ordinary unbalanced panel data-set where firms have different number of $T$ observation the command would still work. Here no adjustments are necessary, you can easily try it yourself:

 install.packages("plm")
library("plm")
data("EmplUK", package="plm")
result<-plm(formula = wage ~ emp, data = EmplUK, model = "within")
summary(result)

Which gives you this output:

Unbalanced Panel: n = 140, T = 7-9, N = 1031

Residuals: Min. 1st Qu. Median 3rd Qu. Max. -12.13926 -1.21442 -0.20655 1.02437 17.11197

Coefficients: Estimate Std. Error t-value Pr(>|t|) emp -0.119813 0.031103 -3.8521 0.0001255 ***

So you can use unbalanced data without any additional adjustment to the plm code itself. I think authors there refer to adjustments that have to be made when programming the function to account for this (as this creates some problems with matrices - at least that is my understanding from the document you linked).

However, if I understand you correctly the problem you have is not just about panel being unbalanced but because there are duplicate observations like this:

Firm ID | Year | event | x | Y | 1 | 1999 | 1jan2000 | 2 | 4 | 1 | 1999 | 20jan2000 | 24| 54 |

This is more serious problem than just unbalanced panel. Here the problem is not the 'unbalanceness' but the fact that you will have 'duplicate' observations if you will run any panel model with Firm ID and Year as panel identifiers because you cant have one panel identifier assigned to two different different observations as that means your observations are not unique.

This could be solved in one of few ways:

1. Run the model on dates instead of years if the events do not occur at exactly the same time. So now your panel identifiers would be: Firm ID Event instead of Firm ID Year.

2. Create new firm ID for each event. So you could create new firm ID like this:

Firm ID | Year | event | x | Y | 1_1 | 1999 | 1jan2000 | 2 | 4 | 1_2 | 1999 | 20jan2000 | 24| 54 |

however, downside with within estimator would be that if these multiple events on same day occur only once these would drop, and also my intuition tells me this would led to some methodological issues if you would try to lets say cluster errors on firm level as one firm would be treated differently based on no of events.

3. You could also aggregate values of X and Y across events on a single day and just include extra dummy for firms with multiple events in a given year.

4. Consider different estimator. For example, instead of within estimator you could use pooled OLS where you treat each observation as individual event with firm fixed effects. I cant think of any reason why that could not be applied here.

Also here I would recommend for searching for panels with duplicates, I know this is not a genuine duplicate, but mostly when this issue is being discussed many people call these observations duplicates so it will probably give you better results than searching just for unbalanced panel data which is usually applied to panel data where you just have different $T$ for each panel ID.

Answer 2

It would be helpful to provide a reproductible example. In the paper Panel Data Econometrics in R: The plm Package, the authors explicitly mention that economic panel datasets often happen to be unbalanced, which case needs some adaptation to the methods. Hopefully, they provide a solution and the result of their work is bundled in the plm add-on package.

The following example is from Arellano and Bond (1991). Employment is explained by past values of employment (two lags), current and first lag of wages and output and current value of capital.

R> emp.gmm <- pgmm(dynformula(emp ~ wage + capital + output, lag = list(2, + 1, 0, 1),
                  log = TRUE), EmplUK, effect = "twoways", model = "twosteps", 
                 + gmm.inst = ~log(emp), lag.gmm = list(c(2, 99))) 
R> summary(emp.gmm)

Call: Unbalanced Panel: n=140, T=7-9, N=1031