3 ms·

The situation certainly won't be similar 3-5 years down the line, but it also won't necessarily be different in the way that people are predicting. In the sect

The situation certainly won't be similar 3-5 years down the line, but it also won't necessarily be different in the way that people are predicting. In the section about the 70% problem, the author writes: > The good news? This gap will likely narrow as tools improve. This is not a fact, it's a prediction, an article of faith. It's a common prediction, that these tools will only get better over time, but that's not guaranteed! I think it's likely as well, provided you define "improve" incredibly pedantically, but the unspoken part of this prediction is that the tools will improve _significantly,_ and _that_ part is one I have doubts about. Honestly, I think it's pretty likely that we've just about hit the local maximum of our current techniques - training newer, bigger models is fantastically expensive and doesn't seem to have the same jumps in capability as earlier generations did, and stringing together a bunch of models in an agent produces frankly only modest improvements for the cost increase. I've written about "the 70% problem" before (not by name, though), and my biggest worry there is that you require experience and good judgement to be able to use these tools effectively, and these tools by their nature erode that good judgement and deny you the experience. Juniors won't have to work through "the tough parts" of programming and build up the skills required to understand when an LLM is leading them astray, and even experienced programmers can lose familiarity with their own codebases as they rely on LLMs to provide the "understanding." Think of all the times where you haven't had to interact with a service for months, how long it takes to warm back up to it -- what about when that's _most_ of your code, because LLMs have been preventing you from needing to gain any deep understanding about it? What happens if, in 3-5 years, LLMs don't get significantly better, but we've stopped producing properly capable Intermediate and Senior developers, and even started atrophying the skills of existing Seniors? (Couple this with LLM-using developers who can move mountains in a day to kickstart new projects and get promoted, but leave behind a trashfire of a codebase for more "classically" experienced developers to clean up and maintain at a much slower pace. That's a problem that's _always_ been in the industry, just magnified due to the power of LLMs.)

MrMcCall 2y ago
[flagged]
- danielbln 2y ago
  Predicting the next word correctly in the right context requires tremendous, vast amounts of knowledge; about language, about the world, about code. You're trying to be clever while being reductive, but you're also wrong. Every programming project starts with a brand new problem to solve? That's obviously false. The vast majority of programming projects are not only composed of smaller and simpler problems (of which most are not novel in any way), they aren't often even novel to begin with. These models are far from infallible (and frequently quite fallible), but if you can't even see the smallest sliver of utility for using LLMs for coding, I question your ability to judge technology.
  - DanHulton 2y ago
    Not to get into a pissing war, but I'll judge ya right back. =) There's a phrase from aviation that I keep thinking about when I think about LLMs - "behind the airplane." When you're "behind the airplane," you're in a situation where the airplane is doing something you don't intimately understand and you're reacting to it. It is an _incredibly_ dangerous place to be, and is indicative of you being in an emergency situation and over your head. You want to be "ahead of the airplane," where you understand intimately what is happening, and you are making forward-thinking plans about what you expect to do in the coming seconds, minutes, and hours. When I'm coding, I generally feel "ahead of the code." I'm thinking ahead about the architecture and design of what it is I want to accomplish, refactors are done with a high level of understanding and for a well-understood reason, etc. When coding with an LLM, I feel like I'm "behind the code" - I'm being given code that I now have to evaluate for purpose. I don't intimately know what it's doing, why it's written that way, the broader effects it will have or how to integrate it into the larger system. When you are "behind the airplane," that's when mistakes happen and planes crash. I think that metaphor extends very well to coding here, as well. =)
    - danielbln 2y ago
      This is not how I use LLMs for coding. I request small snippets or edits, and most of my time is spent conversing and evaluating solution paths. Once I do request and accept code, it is heavily typed and evaluated by the IDE, it is checked against tests and the overall architecture and data flow is still coming from me, I merely off-shore the minute implementation details. And here lies the crux: there are so many different ways of using these tools, some will make the plane crash, some won't and it's all kind of in flux. That said, even the staunchest critic should see that there is _something_ useful there.
noobermin 2y ago
I work in a certain niche. The older I get and the longer I get, I realise I have to use less crutches and abstractions because intimate knowledge of my code is more important than supposed speed. LLMs lead everyone in the opposite path and that to me seems to ruin everything.

MrMcCall 2y ago

[flagged]

danielbln 2y ago
Predicting the next word correctly in the right context requires tremendous, vast amounts of knowledge; about language, about the world, about code. You're trying to be clever while being reductive, but you're also wrong. Every programming project starts with a brand new problem to solve? That's obviously false. The vast majority of programming projects are not only composed of smaller and simpler problems (of which most are not novel in any way), they aren't often even novel to begin with. These models are far from infallible (and frequently quite fallible), but if you can't even see the smallest sliver of utility for using LLMs for coding, I question your ability to judge technology.
- DanHulton 2y ago
  Not to get into a pissing war, but I'll judge ya right back. =) There's a phrase from aviation that I keep thinking about when I think about LLMs - "behind the airplane." When you're "behind the airplane," you're in a situation where the airplane is doing something you don't intimately understand and you're reacting to it. It is an _incredibly_ dangerous place to be, and is indicative of you being in an emergency situation and over your head. You want to be "ahead of the airplane," where you understand intimately what is happening, and you are making forward-thinking plans about what you expect to do in the coming seconds, minutes, and hours. When I'm coding, I generally feel "ahead of the code." I'm thinking ahead about the architecture and design of what it is I want to accomplish, refactors are done with a high level of understanding and for a well-understood reason, etc. When coding with an LLM, I feel like I'm "behind the code" - I'm being given code that I now have to evaluate for purpose. I don't intimately know what it's doing, why it's written that way, the broader effects it will have or how to integrate it into the larger system. When you are "behind the airplane," that's when mistakes happen and planes crash. I think that metaphor extends very well to coding here, as well. =)
  - danielbln 2y ago
    This is not how I use LLMs for coding. I request small snippets or edits, and most of my time is spent conversing and evaluating solution paths. Once I do request and accept code, it is heavily typed and evaluated by the IDE, it is checked against tests and the overall architecture and data flow is still coming from me, I merely off-shore the minute implementation details. And here lies the crux: there are so many different ways of using these tools, some will make the plane crash, some won't and it's all kind of in flux. That said, even the staunchest critic should see that there is _something_ useful there.

noobermin 2y ago

I work in a certain niche. The older I get and the longer I get, I realise I have to use less crutches and abstractions because intimate knowledge of my code is more important than supposed speed. LLMs lead everyone in the opposite path and that to me seems to ruin everything.