3 ms·
This has been the case in software development for a very long time, it is almost impossible to get reliable scientific data about these things because it's alm
by AIPedant 1y ago
This has been the case in software development for a very long time, it is almost impossible to get reliable scientific data about these things because it's almost impossible to run a controlled experiment, and even if you did it's very difficult to decide what to measure and how to measure it. (It's easy to say "compare complexity and quality metrics" but how the fk are you going to define that? Does it really make sense to use the same complexity metric for Go and C++?)
Software developers have had many flame wars over the decades because the lack of data forces the conversation to be anecdotal and ideological:
- should we use dynamic/gradual typing and prioritize developer productivity, or static typing to help enforce correctness?
- agile vs waterfall
- OO vs procedural vs functional
- is Rust's fussiness around memory management more trouble than it's worth for large projects?
- when should you use a 3rd-party library vs doing it yourself?
So this really is nothing new. You are badly underestimating the scientific challenges, instead just hoping big data will plow through. It won't.
- belter 1y agoYour point cuts both ways: If measuring GenAI impact is as difficult as you say, then the bold claims of dramatic productivity gains are just as fragile. How, exactly, do nine out of ten CEOs know it works?
- AIPedant 1y agoYes, that is correct. (I don't use LLMs at all for ethical reasons, so I don't have a dog in this specific fight.)