3 ms·

It's very difficult to perform a fair test. If I test random programmers by getting them to do the same project either with TDD or without, TDD proponents migh

by rlpb 14y ago

It's very difficult to perform a fair test. If I test random programmers by getting them to do the same project either with TDD or without, TDD proponents might (justifiably) complain that the programmers didn't do TDD properly, since they didn't know how. If I use TDD proponents to do the TDD side of the test, and TDD skeptics the non-TDD side, then one might claim that there is a bias in the quality of programmer (either people who like TDD make better programmers, or TDD skeptics make better programmers, depending on the results). For every test you come up with, there's a potential bias that you cannot eliminate, since there will be some hypothetical correlation somewhere that doesn't correspond to TDD itself.

JohnLBevan 14y ago
Do four tests - TDDers doing TDD, TDDers doing non-TDD, non-TDDers doing TDD and non-TDDers doing non-TDD. From this you can get a better idea of whether it's the method or the coder's skill affecting the results. Clearly if TDD is better the TDDers will do better than the non-TDDers when both doing TDD because they've more experience. Comparing the non-TDDers in TDD vs non-TDD will show you if the method alone is enough, or if other factors have affected it (e.g. learning curve of the new method, ability of programmers who chose that method). The TDDers doing non-TDD helps contribute to the question of are they just better programmers (i.e. do they do comparatively well in non-TDD / are they worse that those TDDers doing TDD). Doing other tests with entirely randomly chosen programmers ensuring they're trained up in these techniques would be another way - though as you point out the training would have to ensure they got the techniques they were being tested against / you'd need to avoid bias of people having spent the last few weeks exclusively in an exclusively TDD environment due to the training.

by rlpb 14y ago

JohnLBevan 14y ago

Do four tests - TDDers doing TDD, TDDers doing non-TDD, non-TDDers doing TDD and non-TDDers doing non-TDD. From this you can get a better idea of whether it's the method or the coder's skill affecting the results. Clearly if TDD is better the TDDers will do better than the non-TDDers when both doing TDD because they've more experience. Comparing the non-TDDers in TDD vs non-TDD will show you if the method alone is enough, or if other factors have affected it (e.g. learning curve of the new method, ability of programmers who chose that method). The TDDers doing non-TDD helps contribute to the question of are they just better programmers (i.e. do they do comparatively well in non-TDD / are they worse that those TDDers doing TDD). Doing other tests with entirely randomly chosen programmers ensuring they're trained up in these techniques would be another way - though as you point out the training would have to ensure they got the techniques they were being tested against / you'd need to avoid bias of people having spent the last few weeks exclusively in an exclusively TDD environment due to the training.