6
$\begingroup$

Has Stack Overflow an overall positive impact on programmers productivity?

On one side, it takes time and effort to ask and answer questions -- thus decreasing productivity.

On the other side, more and more programmer questions are answered and easily found through Google. This allows many programmers to find solutions to their problems quicker than ever before, thus increasing productivity.

Has Stack Overflow been economically beneficial to programmer productivity? If so, can this be quantified in terms of money?

Note: I've asked this related question on Skeptics.SE, but it turns out there is no peer-reviewed research on the matter.

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ Why do you think answering questions decreases productivity? Apart from outright obsessive checking of SO every five minutes, of course :) $\endgroup$
    – Luaan
    Commented May 12, 2015 at 11:00
  • $\begingroup$ From the linked question: It isn't obvious how economists might measure value to an economy. It isn't obvious how computer scientists might measure productivity. Both value to an economy and productivities are variables that we have a pretty good grip on, I would disagree on both. $\endgroup$
    – FooBar
    Commented May 12, 2015 at 12:35
  • $\begingroup$ If it is as straightforward as you claim, how would you define the "productivity of a programmer"? $\endgroup$
    – user4239
    Commented May 12, 2015 at 12:45
  • $\begingroup$ One straightforward definition for the productivity of any worker is the value of his output (his value added), as appreciated/evaluated by the firm. This includes all within-firm externalities such as peer effects. In proper notation, with $\mathcal L$ being the set of workers at that firm, the productivity of worker $i$ would be $F(\mathcal L) - F(\mathcal L_{-i})$ $\endgroup$
    – FooBar
    Commented May 12, 2015 at 13:21
  • $\begingroup$ Productivity at the level of a sector or an enterprise is fine and often used. But the productivity of an individual worker ??? That does not make a lot of sense to me. $\endgroup$
    – user4239
    Commented May 12, 2015 at 15:56

2 Answers 2

Reset to default
2
$\begingroup$

Exogenous Variation

As Andre says, you are after causality, not just correlation. And the best way to find that is exogenous variation - he mentions experiments. Unless you can randomly force some people to use SO - and forbid it to a control group, there is little you can do with true experiments.

There may be salvation, however. You are after the "treatment of the treated" effect - those who are informed about SO, and decide to use it. That is hard to get, because the latter step will correlate with ability. Some people are just better/more efficient/smarter and hence can benefit from SO more/less.

In order to remove this self selection, you could instead look at something like the "average treatment effect". We inform a subsample about SO. Some will use it, some will not. What is the average increase of productivity in this sample?

IV Estimation

Now, you will either randomly inform individuals about SO (something that StackExchange as a company might actually do), or you/they might already have data on "being informed about SO", lets call that inform. As long as inform does not correlate with relevant characteristics (such as skill), you can use that as a proxy. Unfortunately, one might argue that it actually does: Perhaps smarter/better programmers are also better at creating networks / using google, and hence are more likely to find SO and become inform.

Of course, if you also have data for ability, you can control for that, so this particular bias would be removed.

Then, all you have to do is regress some measure of productivity/success/career advancement onto inform, and perhaps ability.

Improve this answer
$\endgroup$
9
  • $\begingroup$ How should that work in practice? These are just some general considerations ... $\endgroup$
    – user4239
    Commented May 12, 2015 at 12:35
  • $\begingroup$ And where would you get the "control group" from? $\endgroup$
    – user4239
    Commented May 12, 2015 at 12:47
  • $\begingroup$ I thought my answer was clear on that: the control group would be those with inform = 0. $\endgroup$
    – FooBar
    Commented May 12, 2015 at 13:23
  • $\begingroup$ Not entirely. That's why I asked for clarification. $\endgroup$
    – user4239
    Commented May 12, 2015 at 13:26
  • 1
    $\begingroup$ Is your claim that the only people not aware of StackOverflow are living in North Korea? Because then half of my department (who do a lot of programming) should be deported back. Anyhow, given your comments here and in the remainder of this question, I don't think we will reach a conclusion. Perhaps we can agree to disagree and move on with our lives. $\endgroup$
    – FooBar
    Commented May 12, 2015 at 13:31
0
$\begingroup$

As many have mentioned it can be quantified in monetary terms. As Foobar mentions what we need is exogenous variation* in access to Stack Exchange. Here is one study design:

Inform and give a short introduction to random cohorts in their second year at university about StackOverflow, then come back in few years and compare grades and maybe later work outcomes (such as salary). If it is done in second year, grades from first year can be introduced as controls and to see if randomization was truly successful.

*Variation that is not (cor)related to other characteristics related to programming productivity.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.