Skip to main content
added 15 characters in body
Source Link
FooBar
  • 10.8k
  • 1
  • 30
  • 60

Exogenous Variation

As Andre says, you are after causality, not just correlation. And the best way to find that is exogenous variation - he mentions experiments. Unless you can randomly force some people to use SO - and forbid it to a control group, there is little you can do with true experiments.

There may be salvation, however. You are after the "treatment of the treated" effect - those who are informed about SO, and decide to use it. That is hard to get, because the latter step will correlate with ability. Some people are just better/more efficient/smarter and hence can benefit from SO more/less.

In order to remove this self selection, you could instead look at something like the "average treatment effect". We inform a subsample about SO. Some will use it, some will not. What is the average increase of productivity in this sample?

IV Estimation

Now, you will either randomly inform individuals about SO (something that youStackExchange as a company might actually do), or you/they might already have data on "being informed about SO", lets call that inform. As long as inform does not correlate with relevant characteristics (such as skill), you can use that as a proxy. Unfortunately, one might argue that it actually does: Perhaps smarter/better programmers are also better at creating networks / using google, and hence are more likely to find SO and become inform.

Of course, if you also have data for ability, you can control for that, so this particular bias would be removed.

Then, all you have to do is regress some measure of productivity/success/career advancement onto inform, and perhaps ability.

Exogenous Variation

As Andre says, you are after causality, not just correlation. And the best way to find that is exogenous variation - he mentions experiments. Unless you can randomly force some people to use SO - and forbid it to a control group, there is little you can do with true experiments.

There may be salvation, however. You are after the "treatment of the treated" effect - those who are informed about SO, and decide to use it. That is hard to get, because the latter step will correlate with ability. Some people are just better/more efficient/smarter and hence can benefit from SO more/less.

In order to remove this self selection, you could instead look at something like the "average treatment effect". We inform a subsample about SO. Some will use it, some will not. What is the average increase of productivity in this sample?

IV Estimation

Now, you will either randomly inform individuals about SO (something that you as a company might actually do), or you might already have data on "being informed about SO", lets call that inform. As long as inform does not correlate with relevant characteristics (such as skill), you can use that as a proxy. Unfortunately, one might argue that it actually does: Perhaps smarter/better programmers are also better at creating networks / using google, and hence are more likely to find SO and become inform.

Of course, if you also have data for ability, you can control for that, so this particular bias would be removed.

Then, all you have to do is regress some measure of productivity/success/career advancement onto inform, and perhaps ability.

Exogenous Variation

As Andre says, you are after causality, not just correlation. And the best way to find that is exogenous variation - he mentions experiments. Unless you can randomly force some people to use SO - and forbid it to a control group, there is little you can do with true experiments.

There may be salvation, however. You are after the "treatment of the treated" effect - those who are informed about SO, and decide to use it. That is hard to get, because the latter step will correlate with ability. Some people are just better/more efficient/smarter and hence can benefit from SO more/less.

In order to remove this self selection, you could instead look at something like the "average treatment effect". We inform a subsample about SO. Some will use it, some will not. What is the average increase of productivity in this sample?

IV Estimation

Now, you will either randomly inform individuals about SO (something that StackExchange as a company might actually do), or you/they might already have data on "being informed about SO", lets call that inform. As long as inform does not correlate with relevant characteristics (such as skill), you can use that as a proxy. Unfortunately, one might argue that it actually does: Perhaps smarter/better programmers are also better at creating networks / using google, and hence are more likely to find SO and become inform.

Of course, if you also have data for ability, you can control for that, so this particular bias would be removed.

Then, all you have to do is regress some measure of productivity/success/career advancement onto inform, and perhaps ability.

Source Link
FooBar
  • 10.8k
  • 1
  • 30
  • 60

Exogenous Variation

As Andre says, you are after causality, not just correlation. And the best way to find that is exogenous variation - he mentions experiments. Unless you can randomly force some people to use SO - and forbid it to a control group, there is little you can do with true experiments.

There may be salvation, however. You are after the "treatment of the treated" effect - those who are informed about SO, and decide to use it. That is hard to get, because the latter step will correlate with ability. Some people are just better/more efficient/smarter and hence can benefit from SO more/less.

In order to remove this self selection, you could instead look at something like the "average treatment effect". We inform a subsample about SO. Some will use it, some will not. What is the average increase of productivity in this sample?

IV Estimation

Now, you will either randomly inform individuals about SO (something that you as a company might actually do), or you might already have data on "being informed about SO", lets call that inform. As long as inform does not correlate with relevant characteristics (such as skill), you can use that as a proxy. Unfortunately, one might argue that it actually does: Perhaps smarter/better programmers are also better at creating networks / using google, and hence are more likely to find SO and become inform.

Of course, if you also have data for ability, you can control for that, so this particular bias would be removed.

Then, all you have to do is regress some measure of productivity/success/career advancement onto inform, and perhaps ability.