3 ms·

I disagree with the author here. His point can be summarized as "LLMs produce bad code, LLMs lead to the teams's average skills decreasing and they can't mainta

I disagree with the author here. His point can be summarized as "LLMs produce bad code, LLMs lead to the teams's average skills decreasing and they can't maintain large codebases over time". Yet nowhere he addresses the #1 flaws to his position: rate of improvement of the technology, and its promise to deliver on saved money and gained speed. In all the companies I've seen engineering leadership hardly really gives a shit about things OP says are important. They just care that customers are happy, the system is stable, and its malleable enough to allow for pivots when need be. Good discussions & documentation about architecture before starting the work that gets peer-reviewed + A non-stupid engineer putting on good guardrails around the LLM's output + the extensive unit test suites in CD/CI + peer reviews on the PRs = all downsides near eliminated while all upside gained. This is how we work at my company today (health startup). Google and Meta also boast publicly +30% of new lines of code are AI-generated in their companies today. That the state of *today*; assume in 5 years these AIs are 10x better... I simply cannot foresee a world where LLM-assisted coding is not the de-facto way be a software engineer.

jasminebelmont 1y ago
[flagged]
zaphar 1y ago
The LLM trend lines may indicate that further research, experimentation, and investment in LLMs is warranted. But it is a mistake to assume that trend lines predict the future. Yet this is the primary evidence that is raised in support of LLMs in response to critics. It's not at all clear that the upside gained outweighs the cost in your hypothetical scenario nor is should it be taken as a given that the trend line will continue as is for long enough to create enough upside to outweigh that cost.
yahoozoo 1y ago
Are they going to be 10x better though?
walterbell 1y ago
> Assume in [N] years these AIs are 10x better Why has OpenAI been acquiring "application" layer companies for large financial sums, instead of improving their own tools to build application layer codebases? > 30% of new lines of code is AI-generated "Watching AI drive Microsoft employees insane", 500 comments, https://news.ycombinator.com/item?id=44050152 https://news.ycombinator.com/item?id=44050152
lucideer 1y ago
Your comment is talking about profitability, the op is talking about product quality. These two don't necessarily correlate. Your examples of people "not giving a shit" about things the OP says are important & Google/Meta boasting about AI use are all revenue driven; people in leadership roles commonly place company revenue over product quality, in which case they shouldn't give a shit about the OP's topic. As an engineer / IC I care about product quality because that's what gives me personal fulfillment. As a founder I care about product quality because I entered into this enterprise to solve a problem, not to sell a solution. Many people do the latter (very successfully) & this article isn't for those people. But it's relevant to me.
moooo99 1y ago
> Google and Meta also boast publicly +30% of new lines of code are AI-generated in their companies today. Both are companies with heavy investment into their AI products, hence an extremely biased view. I’d take that with a huge grain of salt. > assume in 5 years these AIs are 10x better... I simply cannot foresee a world Over the last 3ish years the improvements to the performance were significant, but incremental. What makes you think that the models will continue to improve at a substantially faster rate than today? Especially considering past releases have already demonstrated the diminishing returns with larger models and more computational power. Then there is also the pressure on the training data: downwards quality as well as ongoing litigation. From my POV there is more reason to believe the future development of LLMs will slow down somewhat rather than accelerate significantly.
bandoti 1y ago
> Google and Meta also boast publicly +30% of new lines of code are AI-generated in their companies today. You realize that Google and Meta sell AI products, right? So what you cited is effectively an ad campaign. Also the 30% NEW code is likely whittled down to 5-10% added to production after heavy edits. The devil is in the omitted details. :) > In all the companies I've seen engineering leadership hardly really gives a shit about things OP says are important. People put too much stock in “engineers” and “managers”. What we’re really talking about here is in the realm of sociology and psychology. I think there’s a lot of evidence (just ask anyone in academia) that AI is already diminishing people’s ability to think for themselves. There’s a lot of power in AI it’s true—but let’s not get blinded by the gold and leave everyone bloodied in the muck.

jasminebelmont 1y ago

[flagged]

zaphar 1y ago

The LLM trend lines may indicate that further research, experimentation, and investment in LLMs is warranted. But it is a mistake to assume that trend lines predict the future. Yet this is the primary evidence that is raised in support of LLMs in response to critics. It's not at all clear that the upside gained outweighs the cost in your hypothetical scenario nor is should it be taken as a given that the trend line will continue as is for long enough to create enough upside to outweigh that cost.

yahoozoo 1y ago

Are they going to be 10x better though?

walterbell 1y ago

> Assume in [N] years these AIs are 10x better Why has OpenAI been acquiring "application" layer companies for large financial sums, instead of improving their own tools to build application layer codebases? > 30% of new lines of code is AI-generated "Watching AI drive Microsoft employees insane", 500 comments, https://news.ycombinator.com/item?id=44050152 https://news.ycombinator.com/item?id=44050152

lucideer 1y ago

Your comment is talking about profitability, the op is talking about product quality. These two don't necessarily correlate. Your examples of people "not giving a shit" about things the OP says are important & Google/Meta boasting about AI use are all revenue driven; people in leadership roles commonly place company revenue over product quality, in which case they shouldn't give a shit about the OP's topic. As an engineer / IC I care about product quality because that's what gives me personal fulfillment. As a founder I care about product quality because I entered into this enterprise to solve a problem, not to sell a solution. Many people do the latter (very successfully) & this article isn't for those people. But it's relevant to me.

moooo99 1y ago

> Google and Meta also boast publicly +30% of new lines of code are AI-generated in their companies today. Both are companies with heavy investment into their AI products, hence an extremely biased view. I’d take that with a huge grain of salt. > assume in 5 years these AIs are 10x better... I simply cannot foresee a world Over the last 3ish years the improvements to the performance were significant, but incremental. What makes you think that the models will continue to improve at a substantially faster rate than today? Especially considering past releases have already demonstrated the diminishing returns with larger models and more computational power. Then there is also the pressure on the training data: downwards quality as well as ongoing litigation. From my POV there is more reason to believe the future development of LLMs will slow down somewhat rather than accelerate significantly.

bandoti 1y ago

> Google and Meta also boast publicly +30% of new lines of code are AI-generated in their companies today. You realize that Google and Meta sell AI products, right? So what you cited is effectively an ad campaign. Also the 30% NEW code is likely whittled down to 5-10% added to production after heavy edits. The devil is in the omitted details. :) > In all the companies I've seen engineering leadership hardly really gives a shit about things OP says are important. People put too much stock in “engineers” and “managers”. What we’re really talking about here is in the realm of sociology and psychology. I think there’s a lot of evidence (just ask anyone in academia) that AI is already diminishing people’s ability to think for themselves. There’s a lot of power in AI it’s true—but let’s not get blinded by the gold and leave everyone bloodied in the muck.