3 ms·
As far as I know we don't have decades of research about the improvements in productivity and correctness brought by static type systems. We have one study on
by agentultra 1y ago
As far as I know we don't have decades of research about the improvements in productivity and correctness brought by static type systems.
We have one study on test driven development. Another study that attempted to reproduce the results but found flaws in the original. Nothing conclusive.
The field of empirical research in software development practices is... woefully underfunded and incomplete. I think all we can say is, "more data needed."
hwayne did a talk on this [0].
If you ever try to read the literature it's spartan. We certainly haven't improved enough in recent years to make conclusions about the productivity of LLM-based coding tools. We have a study on CoPilot by Microsoft employees who studied Microsoft employees using it (Microsoft owns and develops CoPilot). There's another study that suggests CoPilot increases error rates in code bases by 41%.
What the author is getting at is that you can't rely on personal anecdotes and blog posts and social media influencers to understand the effects of AI on productivity.
If we want to know how it affects productivity we need to fund more and better studies.
[0] https://www.hillelwayne.com/talks/ese/ https://www.hillelwayne.com/talks/ese/
- Yoric 1y ago> As far as I know we don't have decades of research about the improvements in productivity and correctness brought by static type systems. I seem to recall at least some research dating back to the 90s on the topic, which showed how much better (by some metric I can't remember) Ada was wrt most other languages of the time.