I am curious: what exactly are the problems caused by using representative sampling rather than random sampling whenever creating a subsample of large dataset for nonexperimental analysis. Further, how effective is propensity score matching whenever one wants to create balanced comparison groups for non-experimental analyses?
$\begingroup$
$\endgroup$
7
-
2$\begingroup$ What do you mean by representative sampling? I think there are two questions here. I would separate them. Also, this is probably more appropriate ($\nsim$ offtopic) for stats.stackexchange.com $\endgroup$– luchonachoApr 5, 2017 at 8:58
-
$\begingroup$ I can separate the questions but they are synergistic and so I asked them together. The point being that propensity matching is a sometimes employed option for skirting sampling issues. $\endgroup$– 123Apr 5, 2017 at 14:14
-
$\begingroup$ By representative sampling, do you mean stratified sampling? $\endgroup$– luchonachoApr 5, 2017 at 14:15
-
$\begingroup$ Agreed with @luchonacho: "representative sampling" is not a very useful term. As discussed by Kruskal and Mosteller jstor.org/stable/1402564?seq=1#page_scan_tab_contents its interpretation depends on the writer and may mean nothing more than "general, unjustified acclaim for the data". $\endgroup$– GB supports the mod strikeApr 6, 2017 at 1:25
-
$\begingroup$ Okay. Well, for everyone not reaching for heterodox straws, let's pin this down. Representative sampling in this context means when one constructs a subsample from a large statistical population that adequately replicates that larger statistical population according to some characteristic of interest. So, I intended the word to mean what the overwhelming majority of economists want the word to mean. $\endgroup$– 123Apr 6, 2017 at 2:21
|
Show 2 more comments
2 Answers
$\begingroup$
$\endgroup$
To answer your first question: it depends on the subsample that you want to use.
A representative or stratified sample is constructed by dividing the population of interest into non-overlapping subsets, drawing a random sample from each subset, and then computing weights to adjust for the fact that not all elements from the sample had the same probability of being selected from the population.
The advantage of using a representative or stratified sample is that you can use information that you have about the population to construct your sample and so have more reliable estimates of statistics computed for the population of interest. The disadvantage is that the weights that you computed are right for the stratified sample that you constructed but if you want to explore characteristics that differ between the strata then these weights might very well be the wrong weights. Your estimates would be biased and you probably wouldn't be able to adjust for this bias.
As for propensity score matching. You can only match on what you can see. You'll still have to deal with the problem of omitted variable bias. It's possible that propensity score matching increases rather than decreases the bias. How effective it is depends on what assumptions you make and on whether these assumptions hold.
$\begingroup$
$\endgroup$
A sample being ``representative'' of the population has nothing to do with the distribution of some attribute in your sample, which is random. What counts is that the probability of a unit to be included in the sample is equal for the whole population. Let's say you want to estimate the share of females in a population. When you draw a random sample of people from the population, the share of females in your sample is a consistent estimate for the share of females in the population because your sample is random. It will not be the same because of sampling error. As you draw a larger and larger sample your estimate of the female share will converge to the population value.
Now let's say you already know the population share of females and you want to estimate something else. Let's say your population of interest consists of 6 people, 2 females and 4 males. You draw a sample of 3 people without replacement. In the random sampling case, the sampling probability of each person in the population is 1/2. If you draw a stratified sample consisting of 1 woman and 2 men, the sampling probability is still 1/2 for each person in the population, so both ways of sampling are representative of the population.
Do you have anything to gain by stratifying your sample? If the thing you'd like to estimate is independent of gender, then you will gain nothing. However, if you'd like to estimate something that is not independent of gender, then a stratified sample will give you a more precise estimate by reducing sampling error. The drawback is that if you for some reason use incorrect sampling probabilities and do not adjust for it, then you will get a biased estimate.
RE: I just realized this answer is mostly a duplicate of the one above. Sorry about that.
