Unbalanced panel data in prodest package in R

Question

I have a question regarding the usage of unbalanced panel data for TFP estimation by using the prodest package.

The dataset could be found here: https://drive.google.com/file/d/1W5pva05hRiruo1AMNc62ln0GNtcVr5p6/view?usp=sharing

As you can see from the dataset, there are a lot of firms that have not reported some values in specific years, ending up with unbalanced panel data.

The code is the following. BUT in case I do not filter the data with >0 values, I receive error messages coming from the estimation of TFP, saying that there are NaNs.

remove(list=ls())

library(plm)
library(dplyr)
library(ggplot2)
library(prodest)


pckg<-c("plm","readxl","dplyr","ggplot2","prodest")
#install.packages(c("plm","readxl","dplyr","ggplot2","prodest"))
lapply(pckg, require, character.only = TRUE)

# Set the working directory
setwd("C:/Users/vadya/Desktop/LT original currency")

# Downloading the survey data
Data <- read.csv("LToc.csv", header=TRUE, sep=",")
str(Data)

Data$ID<-as.numeric(as.factor(Data$ID))

summary(Data)

# Creating a panel data frame

DataA <- Data %>% 
  #filter(NACE == 'A') %>% 
  #filter(TURN > 0, TFA > 0, FA > 0, VA > 0, L > 0, M > 0) %>%
  mutate(ID = ID,
         Year = Year,
         turn = log(TURN),
         tfa = log(TFA),
         fa = log(FA),
         va = log(VA),
         cogs = log(COGS))
         #l = log(L),
         #m = log(M))
################################################################################################

mod2ACF <-  prodest::prodestACF(DataA$va, fX = DataA$cogs, sX = DataA$tfa, pX = DataA$cogs, idvar = DataA$ID, timevar = DataA$Year, 
                                R = 100, cX = NULL, opt = 'optim', theta0 = NULL, cluster = NULL)  

mod2ACF
omegaACF <- prodest::omega(mod2ACF)
summary(mod2ACF)
summary(omegaACF)


mod2W <-  prodest::prodestWRDG(DataA$turn, fX = DataA$cogs, sX = DataA$tfa, pX = DataA$cogs, idvar = DataA$ID, timevar = DataA$Year, 
                               cX = NULL)  
mod2W
omegaW <- prodest::omega(mod2W)
summary(mod2W)
summary(omegaW)

####################################################################################################################################

If I do not filter the dataframe, in the TFP estimation part (e.g., ACF method) I get the following error message.

Am I doing something wrong, and is there a way to make the prodest functions to work with unbalanced panel data?

Thanks in advance for the ideas! Would really much appreciate your concern!

Answer 1

A common technique we use in Python through Pandas is the introduction of means, medians and modes (depending on the structure of data) in the stead of these missing values.

Another great technique is introducing fitted values by a regression model, thus, predicting the missing values. In this case you'd want a command to drop your NaNs, estimate a model, and replace empty cells with their fitted value. Depending on your data's structure, you might encounter problems with this technique.

As a final comment, I'd suggest you use Python. Here's the pertinent website.