Like other social sciences, software engineering is an under-specifiable domain for double-blind studies. TopCoder might be a good example of how far you can ta

Like other social sciences, software engineering is an under-specifiable domain for double-blind studies. TopCoder might be a good example of how far you can take it. It provides a wealth of interesting comparisons that extend to the group of engineers that want to go do TopCoder, across the platforms that TopCoder supports. But it's a little artificial environment and doesn't tell you much about collaboration. The other problem is that there's more variety and less consensus in judging the work of creative human thought than there is in something like focus group response to commercial messages, or even response to sociolinguistic cues. So all the results of your studies on software have high entropy.