3 ms·

If anything the last few months have made me more confident that at the end of the day, we’re going nowhere fast. With all of the money and effort put into GPT

If anything the last few months have made me more confident that at the end of the day, we’re going nowhere fast. With all of the money and effort put into GPT-4, and now all of the attention from 3rd party devs, we have very little to show for it? At best we have something that improves productivity by a few percent. It’s..ok. The more you use it the more you realize it’s very flawed. I will never completely discount that future models will be fundamentally different, but I’d put the chances of that happening in the next 5 years very low, despite what we’re hearing from certain industry folks.

falcor84 3y ago
I'll take you up on that - what are the odds you'd put on AI-integrated tooling making the typical developer in 2027 twice as productive as in 2022? As for me, I'd put that at more likely than not.
- skepticATX 3y ago
  If we are using “twice as productive” to mean that a typical developer can complete twice as much work (not just writing code, but overall work) in a fixed amount of time, I’d put that at ~5%. If we are talking about specific workflows, I would agree with you that this is quite likely to occur.
ben_w 3y ago
I'm not exactly sure how one might properly define developer productivity: LOC being famously terrible precisely because it's easy to game; plus we keep saying "I know I should write documentation and tests…" and often still don't, but LLMs can at least help with both; and how much of the problem is a game-of-telephone communication failure somewhere between the user and the coder, which is (IMO) one of the strengths of these language models? Given all the people worried about AI safety issues[1], I agree it's likely we might not see any substantial quality[2] improvements for five years. Even without that, there's no way to tell how close we are to a "solution" to the question of intelligence; it may well be that LLMs are to AGI what all the self-driving car startups you've never heard of are to SAE's Level 5. [1] alignment: this entire area is described as "pre-paradigmatic" by the actual researchers 6 months ago privacy preservation: anything that might come under GDPR audience suitability: e.g. no adult content for minors [2] speed/efficiency yes, but scoring similarly on the tests.

falcor84 3y ago

I'll take you up on that - what are the odds you'd put on AI-integrated tooling making the typical developer in 2027 twice as productive as in 2022? As for me, I'd put that at more likely than not.

skepticATX 3y ago
If we are using “twice as productive” to mean that a typical developer can complete twice as much work (not just writing code, but overall work) in a fixed amount of time, I’d put that at ~5%. If we are talking about specific workflows, I would agree with you that this is quite likely to occur.

ben_w 3y ago

I'm not exactly sure how one might properly define developer productivity: LOC being famously terrible precisely because it's easy to game; plus we keep saying "I know I should write documentation and tests…" and often still don't, but LLMs can at least help with both; and how much of the problem is a game-of-telephone communication failure somewhere between the user and the coder, which is (IMO) one of the strengths of these language models? Given all the people worried about AI safety issues[1], I agree it's likely we might not see any substantial quality[2] improvements for five years. Even without that, there's no way to tell how close we are to a "solution" to the question of intelligence; it may well be that LLMs are to AGI what all the self-driving car startups you've never heard of are to SAE's Level 5. [1] alignment: this entire area is described as "pre-paradigmatic" by the actual researchers 6 months ago privacy preservation: anything that might come under GDPR audience suitability: e.g. no adult content for minors [2] speed/efficiency yes, but scoring similarly on the tests.