5 ms·

Well said. That has been my experience as well, but from the perspective of using these tools on my own. Sure, I can now generate thousands of lines of code rel

by imiric 1y ago

Well said. That has been my experience as well, but from the perspective of using these tools on my own. Sure, I can now generate thousands of lines of code relatively quickly, but the hard part is actually reviewing the code to ensure that it does what I asked, fix bugs, hunt for security issues, refactor, simplify and remove code, and so on. I've found that it's often much more productive to write the code myself, and rely on the LLM for simple autocomplete tasks on the way. I imagine that this workflow would be much harder when you have to communicate with a less experienced human who will in turn need to translate it to an LLM, because of the additional layers of indirection. I suspect that the majority of the people who claim that these tools are making them more productive are simply skipping these tasks altogether, or they never cared to do them in the first place. Then the burden for maintaining code quality is on the few who actually care, which has now grown much larger because of the amount of code that's thrown at them. Unfortunately, these people are often seen as pedants and sticklers who block PRs for no good reason. That sometimes does happen, but most of the time, these are the folks who actually care about the product shipped to users. I don't have a suggestion for improving this, but rather a grim outlook that it's only going to get worse. The industry will continue to be flooded by software developers trained on LLM use exclusively, and the companies who build these tools will keep promoting the same marketing BS because it builds hype, and by extension, their valuation.

fhd2 1y ago
> I suspect that the majority of the people who claim that these tools are making them more productive are simply skipping these tasks altogether I think that's probably true, but I think there are multiple layers here. There's what's commonly called vibe coding, where you don't even look at the code. Then there's what I'd call augmented coding, where you generate a good chunk of the code, but still refactor and generally try to understand it. And then there's understanding every line of it. For this, in particular, I don't believe LLMs speed things up. You can get the LLM to _explain_ every line to you, but what I mean is to look at documentation and specs to build your understanding and test out fine grained changes to confirm it. This is something you naturally do while writing code, and unless you type comically slow, I'm not convinced it's not faster this way around. There's a very tight feedback loop when you are writing and testing code atomically. In my experience, this prevents an unreasonable amount of emergencies and makes debugging orders of magnitude faster. I'd say the bulk of my work is either in the second or the third bucket, depending on whether it's production code, the risks involved etc. These categories have existed before LLMs. Maybe the first two are cheaper now, but I've seen a lot of code bases that fall into them - copy pasting from examples and SO. That is, ultimately, what LLMs speed up. And I think it's OK for some software to fall into these categories. Maybe we'll see too much fall into them for a while. I think eventually, the incredibly long feedback cycles of business decisions will bite and correct this. If our industry really flies off the handle, we tend to have a nice software crisis and sort it out. I'm optimistic that, whatever we land on eventually, generative AI will have reasonable applications in software development. I personally already see some.
- AbstractH24 1y ago
  If you are writing code to solve a one off task the first category is ok. What boggles my mind is people are writing code that’s the foundation of products like that. Maybe it’s imposter syndrome though to think it wasn’t already being done before the rise of LLMs
  - bluefirebrand 1y ago
    > Maybe it’s imposter syndrome though to think it wasn’t already being done before the rise of LLMs It may well have been happening before the rise of LLMs, but the volume was a lot more manageable Now it's an unrestricted firehose of crap that there just not enough good devs to wrangle
    - AbstractH24 1y ago
      Would be interesting to look at the real world impact of the rise of outsourcing coding to the cheapest lowest skilled overseas body shop en mass, around the 2000s. Or the impact of trash version of commodified products flooding Amazon. The volume here is orders of magnitude greater, but that’s the closest example I can think of.
      - techexec22 1y ago
        > Would be interesting to look at the real world impact of the rise of outsourcing coding to the cheapest lowest skilled overseas body shop en mass, around the 2000s. Tech exec here. It is all about gamed metrics. If the board-observed metric is mean salary per tech employee, you'll get masses of people hired in india. In our case, we hire thousands in India. Only about 20% are productive, but % productive isnt the metric, so no one cares. You throw bodies at the problem and hope someone solves it. Its great for generations of overseas workers, many of whom may not have had a job otherwise. You probably have dozens of Soham Parekhs . Western execs also like this because it inflates headcount, which is usually what exec comp is based on "i run a team of 150.." Their lieutenants also like it because they can say "i run a team of 30", as do their sub-lieutenants "i run a team of 6"
SkyBelow 1y ago
>I suspect that the majority of the people who claim that these tools are making them more productive are simply skipping these tasks altogether, or they never cared to do them in the first place. I think this follows a larger pattern of AI. It helps someone with enough maturity to not rely on it too blindly and enough foresight to know they still need to grow their own skills, but does well enough that those looking for an easy or quick answer is now given that tool that lets them skip doing more of the hard work. It empowers seniors (developer or senior level in unrelated fields) but traps juniors. Same as using AI to solve a math problem. Is the student verifying their own solution against the AI's, or copying and pasting while thinking they are learning by doing so (or even recognizing their aren't but not worrying about it since the AI can handle it and not realizing how this will trap them on ever harder problems in the future). >...but rather a grim outlook that it's only going to get worse. The industry will continue to be flooded by software developers... I somewhat agree, but even more grim, I think we are looking at this across many more fields than just software development. The way companies make use of this and the market forces at the corporate level might be different, but it is also impacting education and that alone should be enough to negatively impact other areas.

by imiric 1y ago

fhd2 1y ago

> I suspect that the majority of the people who claim that these tools are making them more productive are simply skipping these tasks altogether I think that's probably true, but I think there are multiple layers here. There's what's commonly called vibe coding, where you don't even look at the code. Then there's what I'd call augmented coding, where you generate a good chunk of the code, but still refactor and generally try to understand it. And then there's understanding every line of it. For this, in particular, I don't believe LLMs speed things up. You can get the LLM to _explain_ every line to you, but what I mean is to look at documentation and specs to build your understanding and test out fine grained changes to confirm it. This is something you naturally do while writing code, and unless you type comically slow, I'm not convinced it's not faster this way around. There's a very tight feedback loop when you are writing and testing code atomically. In my experience, this prevents an unreasonable amount of emergencies and makes debugging orders of magnitude faster. I'd say the bulk of my work is either in the second or the third bucket, depending on whether it's production code, the risks involved etc. These categories have existed before LLMs. Maybe the first two are cheaper now, but I've seen a lot of code bases that fall into them - copy pasting from examples and SO. That is, ultimately, what LLMs speed up. And I think it's OK for some software to fall into these categories. Maybe we'll see too much fall into them for a while. I think eventually, the incredibly long feedback cycles of business decisions will bite and correct this. If our industry really flies off the handle, we tend to have a nice software crisis and sort it out. I'm optimistic that, whatever we land on eventually, generative AI will have reasonable applications in software development. I personally already see some.

AbstractH24 1y ago
If you are writing code to solve a one off task the first category is ok. What boggles my mind is people are writing code that’s the foundation of products like that. Maybe it’s imposter syndrome though to think it wasn’t already being done before the rise of LLMs
- bluefirebrand 1y ago
  > Maybe it’s imposter syndrome though to think it wasn’t already being done before the rise of LLMs It may well have been happening before the rise of LLMs, but the volume was a lot more manageable Now it's an unrestricted firehose of crap that there just not enough good devs to wrangle
  - AbstractH24 1y ago
    Would be interesting to look at the real world impact of the rise of outsourcing coding to the cheapest lowest skilled overseas body shop en mass, around the 2000s. Or the impact of trash version of commodified products flooding Amazon. The volume here is orders of magnitude greater, but that’s the closest example I can think of.
    - techexec22 1y ago
      > Would be interesting to look at the real world impact of the rise of outsourcing coding to the cheapest lowest skilled overseas body shop en mass, around the 2000s. Tech exec here. It is all about gamed metrics. If the board-observed metric is mean salary per tech employee, you'll get masses of people hired in india. In our case, we hire thousands in India. Only about 20% are productive, but % productive isnt the metric, so no one cares. You throw bodies at the problem and hope someone solves it. Its great for generations of overseas workers, many of whom may not have had a job otherwise. You probably have dozens of Soham Parekhs . Western execs also like this because it inflates headcount, which is usually what exec comp is based on "i run a team of 150.." Their lieutenants also like it because they can say "i run a team of 30", as do their sub-lieutenants "i run a team of 6"

SkyBelow 1y ago

>I suspect that the majority of the people who claim that these tools are making them more productive are simply skipping these tasks altogether, or they never cared to do them in the first place. I think this follows a larger pattern of AI. It helps someone with enough maturity to not rely on it too blindly and enough foresight to know they still need to grow their own skills, but does well enough that those looking for an easy or quick answer is now given that tool that lets them skip doing more of the hard work. It empowers seniors (developer or senior level in unrelated fields) but traps juniors. Same as using AI to solve a math problem. Is the student verifying their own solution against the AI's, or copying and pasting while thinking they are learning by doing so (or even recognizing their aren't but not worrying about it since the AI can handle it and not realizing how this will trap them on ever harder problems in the future). >...but rather a grim outlook that it's only going to get worse. The industry will continue to be flooded by software developers... I somewhat agree, but even more grim, I think we are looking at this across many more fields than just software development. The way companies make use of this and the market forces at the corporate level might be different, but it is also impacting education and that alone should be enough to negatively impact other areas.