2
$\begingroup$

I want to use the pgmm function in R with a lag of 1, but I get the following error everytime:

Error in pdim.default(index[[1L]], index[[2L]]) : 
  duplicate couples (id-time)
In addition: Warning message:
In pdata.frame(data, index) :
  duplicate couples (id-time) in resulting pdata.frame
 to find out which, use, e.g., table(index(your_pdataframe), useNA = "ifany")

I understand from what I have read that this is because there are multiple year-id combinations, of which some of them are the same. However, I don't understand what the ID column would be in my case and, thus, I don't know how I should merge them (this is what I saw on the internet that would be the solution to the problem).

I run the following code:

growthregressionpgmm <- 
  pgmm(HDI ~ ND + GDPpcap + Fertility_rate + Life_expectancy + GDP_growth + 
         CO2_emissions | lag(ND, 1) + lag(GDPpcap,1) + lag(Fertility_rate, 1) + 
         lag(Life_expectancy, 1) + lag(GDP_growth, 1) + lag(CO2_emissions, 1), 
       data=fulldata, effect="twoways", model="twosteps")

Here is a part of my data:

> dput(fulldata)
structure(list(Year = c("1990", "1992", "1993", "1991", "1991", 
"1992", "1993", "1993", "1993", "1993", "1990", "1990", "1990", 
"1991", "1992", "1992", "1993", "1994", "1996", "1996", "1996", 
"1997", "1997", "1997", "1997", "1998", "1997", "1998", "1998", 
"1998", "1998", "1999", "1999", "1999", "2000", "2000", "2000", 
"1998", "1998", "1999", "2001", "2001", "2001", "2002", "2002", 
"2002", "2002", "2002", "2002"), `Disaster Type` = c("Storm", 
"Storm", "Storm", "Storm", "Landslide", "Flood", "Earthquake", 
"Storm", "Flood", "Storm", "Storm", "Earthquake", "Storm", "Storm", 
"Storm", "Storm", "Storm", "Volcanic activity", "Volcanic activity", 
"Volcanic activity", "Landslide", "Drought", "Storm", "Storm", 
"Wildfire", "Earthquake", "Storm", "Storm", "Storm", "Drought", 
"Drought", "Storm", "Drought", "Flood", "Earthquake", "Epidemic", 
"Epidemic", "Drought", "Storm", "Earthquake", "Earthquake", "Epidemic", 
"Storm", "Earthquake", "Storm", "Storm", "Storm", "Earthquake", 
"Volcanic activity"), Country = c("Fiji", "Fiji", "Fiji", "Marshall Islands (the)", 
"Papua New Guinea", "Papua New Guinea", "Papua New Guinea", "Papua New Guinea", 
"Papua New Guinea", "Solomon Islands", "Tonga", "Vanuatu", "Samoa", 
"Samoa", "Vanuatu", "Vanuatu", "Vanuatu", "Papua New Guinea", 
"Papua New Guinea", "Papua New Guinea", "Papua New Guinea", "Papua New Guinea", 
"Fiji", "Papua New Guinea", "Papua New Guinea", "Papua New Guinea", 
"Tonga", "Tonga", "Vanuatu", "Fiji", "Micronesia (Federated States of)", 
"Fiji", "Kiribati", "Papua New Guinea", "Papua New Guinea", "Micronesia (Federated States of)", 
"Marshall Islands (the)", "Solomon Islands", "Tonga", "Vanuatu", 
"Papua New Guinea", "Papua New Guinea", "Tonga", "Papua New Guinea", 
"Micronesia (Federated States of)", "Solomon Islands", "Micronesia (Federated States of)", 
"Papua New Guinea", "Papua New Guinea"), `Damage-to-GDP` = c(0.00468994375259065, 
0.000726874446693152, 0.0444821683115519, NA, NA, NA, 0.000527417655715239, 
0.000158225296714572, 0.000263708827857619, NA, 0.0102282455341558, 
NA, 0.51022968442747, 0.725915383444357, NA, NA, 0.014519768008217, 
0.0109523857215256, NA, NA, NA, NA, 0.0108773473588107, NA, NA, 
NA, NA, NA, NA, NA, NA, 0.00127934872374966, NA, 0.00438688566628981, 
NA, NA, NA, NA, NA, NA, NA, NA, 0.146837549271082, NA, NA, NA, 
0.00167377438200721, NA, NA), `Affected-people-to-total-population` = c(NA, 
NA, NA, 0.123956697793571, 0.00105807856741241, 0.0186095867906672, 
NA, NA, NA, 0.260674395588859, NA, 1.3645077879282e-05, NA, NA, 
6.44454469291745e-05, 0.00741122639685506, NA, NA, 5.64520135304186e-05, 
0.0002877170956267, 1.50538702747783e-06, 0.0917996981993122, 
NA, 0.00137699547298968, 0.001468795171189, 0.00176772137543665, 
0.0310497935188731, 0.00515293923654052, 0.01348413086349, 0.329254133876227, 
0.26615407363596, NA, 1.01238972183387, NA, 0.000855053692241551, 
0.0319454013891734, 0.00429531259235907, 0.000972630684450451, 
0.0316493527908319, 0.0777866659310954, 3.36422562806829e-05, 
0.0002334873010525, NA, 0.00073295258059158, 0.00163505559189012, 
0.0025491925260431, NA, 0.000163971494539503, 0.00213162942901354
), ND = c(0, 0, 1, 1, 0, 1, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 
0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 1, 0, 0, 1, 0, 0, 
1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0), HDI = c(0.662, 0.672, 0.675, 
NA, 0.389, 0.398, 0.411, 0.411, 0.411, NA, 0.654, NA, 0.633, 
0.634, NA, NA, NA, 0.417, 0.433, 0.433, 0.433, 0.435, 0.687, 
0.435, 0.435, 0.442, 0.675, 0.678, NA, 0.687, NA, 0.692, NA, 
0.445, 0.45, 0.546, NA, NA, 0.678, NA, 0.456, 0.456, 0.679, 0.462, 
0.559, 0.486, 0.559, 0.462, 0.462), CO2_emissions = c(1.11735406060889, 
1.0048343181516, 0.995724293773337, NA, 0.459388934233434, 0.453425890525591, 
0.443088179935911, 0.443088179935911, 0.443088179935911, 0.421240986851407, 
0.810011675730259, 0.450328505249944, 0.630676338888104, 0.648444788624182, 
0.401746471611781, 0.401746471611781, 0.390243139022436, 0.430716741605883, 
0.41194897189593, 0.41194897189593, 0.41194897189593, 0.47330106757541, 
0.999162394542445, 0.47330106757541, 0.47330106757541, 0.513086548656141, 
1.02473633550337, 0.906999752658917, 0.453257822200498, 0.962398629269006, 
1.0844299866923, 0.973254471333977, 0.353565058091886, 0.427890152149318, 
0.455899066725996, 1.16085361538891, NA, 0.366049558092927, 0.906999752658917, 
0.46529114831876, 0.537041714221921, 0.537041714221921, 0.893600170580889, 
0.571820580423017, 1.33619545921704, 0.362125429458561, 1.33619545921704, 
0.571820580423017, 0.571820580423017), GDPpcap = c(2926.57253822332, 
2956.74562527877, 2977.75102732399, NA, 1490.76034619757, 1658.37860177333, 
1915.54964077492, 1915.54964077492, 1915.54964077492, 1739.45172258361, 
2570.98733940999, 2644.90678139038, 2407.69721594666, 2335.19360351875, 
2643.63498560777, 2643.63498560777, 2586.82481239303, 1982.65811460788, 
1968.63071447196, 1968.63071447196, 1968.63071447196, 1845.78398296477, 
3131.2099301146, 1845.78398296477, 1845.78398296477, 1733.21642938823, 
3251.4184493856, 3317.1836235216, 2778.23593092383, 3142.49796380533, 
2528.7225952912, 3392.9763183624, 1648.96379800244, 1723.86945516225, 
1643.08412576665, 2707.56199905807, NA, 1834.61951190637, 3317.1836235216, 
2737.19559474216, 1606.1955026128, 1606.1955026128, 3547.32785335305, 
1571.03933225238, 2791.04965138618, 1256.29997638498, 2791.04965138618, 
1571.03933225238, 1571.03933225238), Fertility_rate = c(3.398, 
3.352, 3.33, NA, 4.756, 4.723, 4.7, 4.7, 4.7, 5.461, 4.644, 4.926, 
5.118, 5.034, 4.841, 4.841, 4.798, 4.683, 4.653, 4.653, 4.653, 
4.632, 3.209, 4.632, 4.632, 4.604, 4.34, 4.3, 4.573, 3.171, 4.471, 
3.132, 4.11, 4.569, 4.525, 4.3, NA, 4.872, 4.3, 4.531, 4.475, 
4.475, 4.236, 4.422, 4.105, 4.606, 4.105, 4.422, 4.422), Life_expectancy = c(65.379, 
65.278, 65.218, NA, 56.823, 57.152, 57.473, 57.473, 57.473, 64.961, 
68.935, 64.721, 66.281, 66.47, 65.349, 65.349, 65.633, 57.781, 
58.344, 58.344, 58.344, 58.594, 65.246, 58.594, 58.594, 58.828, 
69.471, 69.535, 66.899, 65.36, 64.298, 65.512, 62.829, 59.049, 
59.265, 64.55, NA, 66.665, 69.535, 67.134, 59.487, 59.487, 69.725, 
59.722, 64.888, 68.175, 64.888, 59.722, 59.722), CPI = c(49.9250037293591, 
59.4846188641146, 61.8987673569764, NA, NA, NA, NA, NA, NA, 45.2759095332228, 
32.005120180595, 55.8702936919443, 41.5819326210445, 22.8740281444522, 
38.2055963679636, 38.2055963679636, 38.5727156476084, NA, NA, 
NA, NA, NA, 67.6520051713937, NA, NA, NA, NA, NA, 42.9430895548233, 
69.5670319917053, NA, 71.8503332005384, 69.3852858719752, NA, 
NA, NA, NA, 56.7139321922187, NA, 44.8602213817088, 74.0722891566265, 
74.0722891566265, NA, 73.1084337349398, NA, 64.4171779141104, 
NA, 73.1084337349398, 73.1084337349398), GDP_growth = c(5.80000271926605, 
6.10000190987678, 2.13003242138274, NA, 9.54689770861241, 13.8490852689481, 
18.2022859527298, 18.2022859527298, 18.2022859527298, 3.99925595238095, 
-2.04409143450813, 11.6956997985937, -4.42145094868896, -2.30000940480883, 
2.5854137275348, 2.5854137275348, 0.735447995455772, 5.94210905967769, 
7.73369579796399, 7.73369579796399, 7.73369579796399, -3.90438965639359, 
-2.1999993686925, -3.90438965639359, -3.90438965639359, -3.76911321783457, 
1.22344961737785, 2.45875910300609, 1.17685436113621, 1.30000045153811, 
2.8505158599975, 8.79999871927239, -1.53846153846153, 1.85555399408817, 
-2.49484199260023, 4.83390378617111, NA, 1.29870129870129, 2.45875910300609, 
0.337293221894313, -0.121288605564772, -0.121288605564772, 3.74892569298142, 
-0.158900533082658, 0.546997672499344, -2.79654654654654, 0.546997672499344, 
-0.158900533082658, -0.158900533082658), Health_expenditure = c(NA, 
NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, 
NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, 
NA, 189516095.258765, 22580755.7012392, NA, NA, NA, NA, 215735067.486362, 
215735067.486362, 11653482.0216929, 253236754.502794, 23591006.3055499, 
58972354.1993895, 23591006.3055499, 253236754.502794, 253236754.502794
)), row.names = c(NA, -49L), class = c("tbl_df", "tbl", "data.frame"
))

Can someone provide me with a code that merges id with year such that the regression works? It would really help me because I have been struggling for a month now already with this!...

$\endgroup$
8
  • $\begingroup$ I am willing to take a look at it provided that you provide (1) What is the economic research question you are trying to answer (2) what is the statical model (write up a model equation) (3) what are the basic features of your data - what does independent and dependent variables measure. As long as this is not provided I personally consider this to be not a question about economics nor economic empirical methods and are as such in my view off topic (I am not a moderator). $\endgroup$ May 18 at 15:04
  • $\begingroup$ See this discussion for when R code questions are on topic economics.meta.stackexchange.com/questions/1912/… $\endgroup$ May 18 at 15:06
  • $\begingroup$ Hey @JesperHybel. Thank you. For question (1), the research question is: "What is the impact of severe natural disasters on poverty in 11 Pacific Island countries?" Question (2): HDI= βND +αX_(it-1 )+ δ_1 FE_i + δ_2 FE_t + ε_it Where HDI = human development index, ND is the natural disaster dummy, X contains the lag of the control variables (GDPpcap + Fertility_rate + Life_expectancy + GDP_growth + CO2_emissions), FE_i are country fixed effects and FE_t are time fixed effects. $\endgroup$ May 18 at 15:47
  • $\begingroup$ @JesperHybel I run the regression plm first and now I want to do a robustness check and for this I need to run this regression with pgmm. For (3), the Dependent variable is a proxy for poverty, namely the HDI. The independent variables are GDP per capita (constant US 2010 $), the fertility rate (births per woman), Life expectancy (at birth, in years), GDP growth (annual %) and CO2 emissions (in metric tons per capita). Hope this helps. Thank you in advance! $\endgroup$ May 18 at 15:53
  • 1
    $\begingroup$ I think you should add this to the question itself not in comments. Also I believe your data is then country panel. This means that your model index are given in variables ("Country","Year"). The error you get occurs due to duplicates in this index. So for some countries there are more than 1 observation in a given year. Also unless model index variables are the first two columns I believe you have to provide them as the argument "index" to the estimation call (this goes for all use of plm-package if I remember correctly) $\endgroup$ May 18 at 16:10
4
$\begingroup$

The plm package is for panel data. Therefore there has to be variables in dataset with (id,time) to indicate the panel structure. If the argument index is not used when calling an estimation routine such as pgmm() then it is automatically assumed that the first two columns indicates panel structure hence are (id,time).

The error occuring here is duplicates in the panel structure. These can occur if there are NA's in the variables indicating (id,time) or they can occur just because some observations are repeated. So do the following:

  1. Check for NA's in variables id and time and find some way to handle them by for example removing them (this is not necessarily the best from the point of view of estimation but it will get the code running).
  2. Check for duplicates after having removed NA's and remove these (again removal is not necessarily the best way to handle duplicates from an estimation perspective)

Once there are no NA's and no duplicates the error you are experiencing should dissappear.

To remove duplicates use for example the following code

library(data.table)
dt <- as.data.table(dt)
dt[,n:=1]
dt[,count:=cumsum(n),by=.(id,time)]
dt_no_dups <- dt[count==1,]

However, if you are not using the index argument to indicate which variables are id and time then the pgmm() routine assumes your first two variables are giving the panel structure even if they are not and offcourse there could be duplicates in these even if you have removed them from the variables you know to indicating panel structure. So if you variables are column ordered [id,dependent,time,independent] then pgmm() will look for duplicates in (id,dependent) which may or may not throw an error.

Here is R code to simulate a Arellano-Bond model and estimate. The model simulated is $$y_{it} = \rho y_{i,t-1} + x_{it}\beta + \mu_i + \epsilon_{it}.$$

If the variable with_duplicates is TRUE then duplicates are added to data before estimation in order to generate the error you are experiencing. The model simulated is

library(data.table)
library(plm)
# simulate simple model
# y_it = \rho y_{i,t-1} + x_{it}^\top\beta + \epsilon_{it}

with_duplicates <- TRUE # if TRUE duplicates are added to data throwing error
T <- 10
N <- 100 
dt <- data.table(id=rep(1:N,each=T),time=rep(1:T,N)) 

# Choose y0 for all N countries
y0 <- rnorm(N)
# Set model parameters
rho <- 0.5
b <- -0.5
mu <- 2*runif(N)    

# simulate epsilon and x
epsilon <- rnorm(N*T)
x <- rnorm(N*T)
z <- b*x + epsilon

index <- 1
y <- rep(0,N*T)
for (i in 1:N)
{
    for (t in 1:T)
    {

        if (t==1) 
            { 
                y[index] <- rho*y0[i] + z[index] + mu[i]
            } else {
                y[index] <- rho*y[index-1] + z[index] + mu[i]
            }
        index <- index + 1
    }
}

dt$y <- y
dt$x <- x
dt[,lag_y:=shift(y,1),by=id]

# Add a duplicate observation to generate error

if (with_duplicates) {dt <- rbind(dt,dt[4,])}

model <- pgmm(y ~ lag(y,1) + x| lag(y,2:4),effect="individual", model="twosteps",data=dt)
lm(y ~ lag_y + x,data=dt)
summary(model)
$\endgroup$
1
  • 1
    $\begingroup$ Thanks a lot! This really helped me. I will look into all of this and I understand what you are saying. Thanks once again. $\endgroup$ May 18 at 19:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.