0
$\begingroup$

I am trying to perform a regression on unbalanced panel data in R. In order to set the data as panel data, I use the following code:

pdata <- pdata.frame(MODEL_3, index = c("country","year"))

However, I run into the following error:

Warning message:
In pdata.frame(MODEL_3, index = c("country", "year")) :
  duplicate couples (id-time) in resulting pdata.frame
 to find out which, use e.g. table(index(your_pdataframe), useNA = "ifany")

When I run the suggested code in the error (table(index(your_pdataframe), useNA = "ifany")), I get the following (partial) output:

                         year
country                    1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990
  Afghanistan                 0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
  Albania                     0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    1    1    1    1    1    1    1    1    1    1    1
  Algeria                     0    0    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1
  Angola                      0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    1    1    1    1    1    1    1    1    1    1    1
  Argentina                   1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1
  Armenia                     0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
  Australia                   1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1
  Austria                     1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1
  Azerbaijan                  0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
  Bahrain                     0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    1    1    1    1    1    1    1    1    1    1    1
  Bangladesh                  0    0    0    0    0    0    0    0    0    0    0    0    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1
  Belarus                     0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
  Belgium                     1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1
  Benin                       1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1
  Bhutan                      0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    1    1    1    1    1    1    1    1    1    1    1
  Bolivia                     1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1
  Botswana                    0    0    0    0    0    0    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1
  Brazil                      1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1    1
  Bulgaria                    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    1    1    1    1    1    1    1    1    1    1    1

The 1's are all observations in my dataset, but all the 0's are observations which are not included (available) in my dataset since I could not find data on that specific country for that specific period. So, it looks like every observation of my dataset is regarded as being a duplicate couple. How can I solve this problem?

This are the first 45 rows of my dataset:

       country year iso2c                     region      GDP ccode scode polity polity2 AU CA OA D FD
1  Afghanistan 2002    AF                 South Asia 2.518913   700   AFG    -66      NA  0  0  0 0  0
2  Afghanistan 2003    AF                 South Asia 2.535397   700   AFG    -66      NA  0  0  0 0  0
3  Afghanistan 2004    AF                 South Asia 2.522727   700   AFG    -66      NA  0  0  0 0  0
4  Afghanistan 2005    AF                 South Asia 2.552954   700   AFG    -66      NA  0  0  0 0  0
5  Afghanistan 2006    AF                 South Asia 2.562631   700   AFG    -66      NA  0  0  0 0  0
6  Afghanistan 2007    AF                 South Asia 2.608043   700   AFG    -66      NA  0  0  0 0  0
7  Afghanistan 2008    AF                 South Asia 2.614912   700   AFG    -66      NA  0  0  0 0  0
8  Afghanistan 2009    AF                 South Asia 2.688687   700   AFG    -66      NA  0  0  0 0  0
9  Afghanistan 2010    AF                 South Asia 2.735042   700   AFG    -66      NA  0  0  0 0  0
10     Albania 1980    AL      Europe & Central Asia 3.299354   339   ALB     -9      -9  1  0  0 0  0
11     Albania 1981    AL      Europe & Central Asia 3.314918   339   ALB     -9      -9  1  0  0 0  0
12     Albania 1982    AL      Europe & Central Asia 3.318360   339   ALB     -9      -9  1  0  0 0  0
13     Albania 1983    AL      Europe & Central Asia 3.313922   339   ALB     -9      -9  1  0  0 0  0
14     Albania 1984    AL      Europe & Central Asia 3.299315   339   ALB     -9      -9  1  0  0 0  0
15     Albania 1985    AL      Europe & Central Asia 3.298051   339   ALB     -9      -9  1  0  0 0  0
16     Albania 1986    AL      Europe & Central Asia 3.313472   339   ALB     -9      -9  1  0  0 0  0
17     Albania 1987    AL      Europe & Central Asia 3.301364   339   ALB     -9      -9  1  0  0 0  0
18     Albania 1988    AL      Europe & Central Asia 3.286959   339   ALB     -9      -9  1  0  0 0  0
19     Albania 1989    AL      Europe & Central Asia 3.316032   339   ALB     -9      -9  1  0  0 0  0
20     Albania 1990    AL      Europe & Central Asia 3.264504   339   ALB      1       1  0  0  1 0  0
21     Albania 1991    AL      Europe & Central Asia 3.124442   339   ALB    -88       3  0  0  0 0  0
22     Albania 1992    AL      Europe & Central Asia 3.094684   339   ALB      5       5  0  0  1 0  0
23     Albania 1993    AL      Europe & Central Asia 3.136984   339   ALB      5       5  0  0  1 0  0
24     Albania 1994    AL      Europe & Central Asia 3.174290   339   ALB      5       5  0  0  1 0  0
25     Albania 1995    AL      Europe & Central Asia 3.231288   339   ALB      5       5  0  0  1 0  0
26     Albania 1996    AL      Europe & Central Asia 3.271812   339   ALB      0       0  0  1  0 0  0
27     Albania 1997    AL      Europe & Central Asia 3.224308   339   ALB      5       5  0  0  1 0  0
28     Albania 1998    AL      Europe & Central Asia 3.263790   339   ALB      5       5  0  0  1 0  0
29     Albania 1999    AL      Europe & Central Asia 3.319196   339   ALB      5       5  0  0  1 0  0
30     Albania 2000    AL      Europe & Central Asia 3.351145   339   ALB      5       5  0  0  1 0  0
31     Albania 2001    AL      Europe & Central Asia 3.389809   339   ALB      5       5  0  0  1 0  0
32     Albania 2002    AL      Europe & Central Asia 3.410394   339   ALB      7       7  0  0  0 1  0
33     Albania 2003    AL      Europe & Central Asia 3.435395   339   ALB      7       7  0  0  0 1  0
34     Albania 2004    AL      Europe & Central Asia 3.460504   339   ALB      7       7  0  0  0 1  0
35     Albania 2005    AL      Europe & Central Asia 3.486102   339   ALB      9       9  0  0  0 1  0
36     Albania 2006    AL      Europe & Central Asia 3.513738   339   ALB      9       9  0  0  0 1  0
37     Albania 2007    AL      Europe & Central Asia 3.542244   339   ALB      9       9  0  0  0 1  0
38     Albania 2008    AL      Europe & Central Asia 3.576985   339   ALB      9       9  0  0  0 1  0
39     Albania 2009    AL      Europe & Central Asia 3.594223   339   ALB      9       9  0  0  0 1  0
40     Albania 2010    AL      Europe & Central Asia 3.612186   339   ALB      9       9  0  0  0 1  0
41     Algeria 1962    DZ Middle East & North Africa 3.214507   615   ALG     -8      -8  1  0  0 0  0
42     Algeria 1963    DZ Middle East & North Africa 3.331813   615   ALG     -8      -8  1  0  0 0  0
43     Algeria 1964    DZ Middle East & North Africa 3.345343   615   ALG     -8      -8  1  0  0 0  0
44     Algeria 1965    DZ Middle East & North Africa 3.359953   615   ALG     -9      -9  1  0  0 0  0
45     Algeria 1966    DZ Middle East & North Africa 3.326563   615   ALG     -9      -9  1  0  0 0  0
>

Let me know if something doesn't make sense and I can edit the question.

Thank you in advance.

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ It would help if you could provide minimal working example, or at least link to the dataset or example how it looks like? $\endgroup$
    – 1muflon1 ♦
    May 21 '20 at 22:27
  • $\begingroup$ I tried to run panel FE regression using the following code below my comment between GDP and polity and it worked. So at least in example provided here there seem to be no duplicates. Also I see that many of your variables are time invariant - beware that time invariant variables cant be included in FE panel regression drop out due to the way how within estimator is calculated. Code that run without problem: library(plm) est1 <- plm(GDP ~ polity, data = example, index = c("country", "year"), model = "within") summary(est1) $\endgroup$
    – 1muflon1 ♦
    May 22 '20 at 11:58
  • $\begingroup$ Thank you for your feedback. I will not include the time-invariant variables in my model. However, when I use your code, I still run into the same error. Which type are the variables in your example dataset? $\endgroup$
    – Andres34v
    May 22 '20 at 13:53
  • $\begingroup$ as the code that I showed in my comment shows I just run a simple FE panel regression of GDP on polity- I don’t know what model you tried to estimate so I just used 2 variables in order they appear in the dataset to test if it works. Also it’s possible there actually are some duplicates somewhere but they are not in the example you posted and I can only work with that. Could be actually also due to model specification - that’s why I asked for minimal working example $\endgroup$
    – 1muflon1 ♦
    May 22 '20 at 13:55
  • $\begingroup$ Then I must have overlooked some duplicates. Do you know how I can find these duplicates and delete them from the dataset? $\endgroup$
    – Andres34v
    May 22 '20 at 14:06
1
$\begingroup$

You can use the dplyr::distinct function to keep only unique values, at a particular level of aggregation. So, for example, to only keep unique combinations of country and year in df, you can use

library(dplyr) new_df <- distinct(df, country, year)

df should be in long format, though.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.