Convincing evidence for claims regarding software engineering should be based on: 1. A large number of programmers. Statistical significance, also real develop

Convincing evidence for claims regarding software engineering should be based on: 1. A large number of programmers. Statistical significance, also real development is done in teams. 2. A long time - conclusions valid for short snippets do not necessarily translate to large projects where people in a team have to cooperate. 3. Controlled environment - passively getting statistics from existing companies won't cut it. So you need to pay a programmer's salary to a large number of people for a longish time which is expensive. Perhaps this could be worked around by having deals with existing companies to let you control some factors in their existing processes. Maybe not perfect but still better than anecdata.