# Has Stack Overflow been economically beneficial to programmer productivity?

Has Stack Overflow an overall positive impact on programmers productivity?

On one side, it takes time and effort to ask and answer questions -- thus decreasing productivity.

On the other side, more and more programmer questions are answered and easily found through Google. This allows many programmers to find solutions to their problems quicker than ever before, thus increasing productivity.

Has Stack Overflow been economically beneficial to programmer productivity? If so, can this be quantified in terms of money?

Note: I've asked this related question on Skeptics.SE, but it turns out there is no peer-reviewed research on the matter.

-
Why do you think answering questions decreases productivity? Apart from outright obsessive checking of SO every five minutes, of course :) –  Luaan May 12 at 11:00
From the linked question: It isn't obvious how economists might measure value to an economy. It isn't obvious how computer scientists might measure productivity. Both value to an economy and productivities are variables that we have a pretty good grip on, I would disagree on both. –  FooBar May 12 at 12:35
If it is as straightforward as you claim, how would you define the "productivity of a programmer"? –  André Peseur May 12 at 12:45
One straightforward definition for the productivity of any worker is the value of his output (his value added), as appreciated/evaluated by the firm. This includes all within-firm externalities such as peer effects. In proper notation, with $\mathcal L$ being the set of workers at that firm, the productivity of worker $i$ would be $F(\mathcal L) - F(\mathcal L_{-i})$ –  FooBar May 12 at 13:21
Productivity at the level of a sector or an enterprise is fine and often used. But the productivity of an individual worker ??? That does not make a lot of sense to me. –  André Peseur May 12 at 15:56

### Exogenous Variation

As Andre says, you are after causality, not just correlation. And the best way to find that is exogenous variation - he mentions experiments. Unless you can randomly force some people to use SO - and forbid it to a control group, there is little you can do with true experiments.

There may be salvation, however. You are after the "treatment of the treated" effect - those who are informed about SO, and decide to use it. That is hard to get, because the latter step will correlate with ability. Some people are just better/more efficient/smarter and hence can benefit from SO more/less.

In order to remove this self selection, you could instead look at something like the "average treatment effect". We inform a subsample about SO. Some will use it, some will not. What is the average increase of productivity in this sample?

### IV Estimation

Now, you will either randomly inform individuals about SO (something that StackExchange as a company might actually do), or you/they might already have data on "being informed about SO", lets call that inform. As long as inform does not correlate with relevant characteristics (such as skill), you can use that as a proxy. Unfortunately, one might argue that it actually does: Perhaps smarter/better programmers are also better at creating networks / using google, and hence are more likely to find SO and become inform.

Of course, if you also have data for ability, you can control for that, so this particular bias would be removed.

Then, all you have to do is regress some measure of productivity/success/career advancement onto inform, and perhaps ability.

-
How should that work in practice? These are just some general considerations ... –  André Peseur May 12 at 12:35
And where would you get the "control group" from? –  André Peseur May 12 at 12:47
I thought my answer was clear on that: the control group would be those with inform = 0. –  FooBar May 12 at 13:23
Not entirely. That's why I asked for clarification. –  André Peseur May 12 at 13:26
And where will you find programmers who don't know about StackOverflow? Probably in North Korea. But then one had to question the validity of the control group. –  André Peseur May 12 at 13:29

As many have mentioned it can be quantified in monetary terms. As Foobar mentions what we need is exogenous variation* in access to Stack Exchange. Here is one study design:

Inform and give a short introduction to random cohorts in their second year at university about StackOverflow, then come back in few years and compare grades and maybe later work outcomes (such as salary). If it is done in second year, grades from first year can be introduced as controls and to see if randomization was truly successful.

*Variation that is not (cor)related to other characteristics related to programming productivity.

-

Unfortunately this question is impossible to answer. Indeed, we don't know what the productivity of programmers would have been in the absence of StackOverflow. Economics is a social science and it is impossible to run experiments like you would do in hard sciences.

It is also worth noting that StackOverflow doesn't come out of nothing. If it did not exist, the actual users would be active elsewhere, on other media, such as classical forums or mailing lists. And these places have existed loooong before StackOverflow has been created.

StackOverflow is much fancier than a (no html please) mailing list or a classical forum. But substantially, nothing has changed. People come up with questions and devoted volunteers spend time answering them.

-
Economics is a social science and it is impossible to run experiments like you would do in hard sciences.. Also, you imply that experiments are the only method of finding causation. I strongly disagree with both. –  FooBar May 12 at 12:30
Experiments are probably not the only way to find out about causation. But by far the least bumpy way. And i stress again that economics, as a social science, does not have the same tools as the hard sciences. That's a fact. –  André Peseur May 12 at 12:56
It's not clear to me why we can't run an experiment taking two representative samples of programmers and give to one access to stack overflow, and not to the other, then measure productivity (e.g. how long does it take to perform a programming task), Assuming we can remove any observer effect, it would be quite an effective way to answer this question. –  Sklivvz May 12 at 13:18
In your answer, you only looked at experiments, see no possibility, and conclude impossible to answer. Now you're saying it is "the least bumpy one". Perhaps you should change your answer to reflect this change of mind. Second: i stress again that economics, as a social science, does not have the same tools as the hard sciences. That's a fact.. No, it's not a fact, it's your claim. We have exactly the same tool set. There is no reason why any estimator should be "hard science specific". –  FooBar May 12 at 13:28
@AndréPeseur there's plenty of journals publishing peer-reviewed experiments on programming productivity. The setup I describe is quite typical. For examples of this see this google scholar search: all the results come from experiments run in the US or Europe. –  Sklivvz May 12 at 13:33