4 ms·

The key is to look at the long term structural changes the industry is going through, and whether or not AI helps, or hinders that goal In general, the industr

by 20k 1y ago

The key is to look at the long term structural changes the industry is going through, and whether or not AI helps, or hinders that goal In general, the industry has been making huge efforts to push errors from runtime, to compile time. If you imagine points where we can catch errors being laid out from left to right, we have the following: Caught by: Compiler -> code review -> tests -> runtime checks -> 'caught' in prod The industry is trying to push errors leftwards. Rust, heavier review, safety in general - its all about cutting down costs by eliminating expensive errors earlier in the production chain. Every industry does this, its much less costly to catch a defective oxygen mask in the factory, than when it sets a plane on fire. Its also better to catch a defective component in the design phase, than when you're doing tests on it AI is all about trying to push these errors rightwards. The only way that it can save in engineer time is if it goes through inadequate testing, validation, and review. 90% of the complexity of programming is building a mental model of what you're doing, and ensuring that it meets the spec of what you want to do. A lot of that work is currently pure mental work with no physical component - we try and offload it increasingly to compilers in safe languages, and add tests and review to minimise the slippage. But even in a safe language, it still requires a very high amount of mental work to be done to make sure that everything is correct. Tests and review are a stop gap to try and cover the fallibility of the human brain So if you chop down on that critical mental work by using something probabilistically correct, you're introducing errors that will be more costly down the line. It'll be fine in the short term, but in the long term it'll cost you more money. That's the primary reason why I don't think AI will catch on - its short termist thinking from people who don't understand what makes software complex to build, or how to actually produce software that's cheap in the long term. Its also exactly the same reason that Boeing is getting its ass absolutely handed to it in the aviation world. Use AI if you want to go bankrupt in 5 years but be rich now

WaxProlix 1y ago
That's true up to a certain threshold on 'probabilistically correct', right? At a certain number of 9s, it's fine. And increasingly I use AI to help ask me questions, refine my understanding of problem spaces, do deep research on existing patterns or trends in a space and then use the results as context to have a planning session, which provides context for architecture, etc. So, I don't know that the tools are inherently rightward-pushing
- 20k 1y ago
  The problem is, given the inherent limitations of natural language as a format to feed to an AI, it can never have enough information to be able to solve your problem adequately. Often the constraints of what you're trying to solve only crop up during the process of trying to solve the problem itself, as it was unclear that they even existed beforehand An AI tool that could have a precise enough specification fed into it to produce the result that you wanted with no errors, would be a programming language I don't disagree at all that AI can be helpful, but there's a huge difference between using it as a research tool (which is very valid), and the folks who are trying to use it to replace programmers en masse. The latter is what's driving the bubble, not the former
- B56b 1y ago
  AI code reliability is nowhere near any number of 9s
bugglebeetle 1y ago
I’m not sure this follows as SOTA LLMs are pretty good at writing Rust, so wouldn’t this also make it easier for codebases to move leftward in your analogy? For example, I was resistant to use Rust for a lot of things because a). The community is somewhat annoying and pedantic, even by software engineering standards b). The overhead in getting colleagues up to speed on Rust code was too much of a time suck. LLMs solve both those problems and we’re now migrating lots of stuff to Rust, my colleagues can ask lots of questions (of Google Gemini Pro 2.5), without burdening anyone or being met with disdain, and seem generally more curious and positive about these moves/Rust overall.
jjk166 1y ago
> It'll be fine in the short term, but in the long term it'll cost you more money. That's the primary reason why I don't think AI will catch on - its short termist thinking from people who don't understand what makes software complex to build, or how to actually produce software that's cheap in the long term. Its also exactly the same reason that Boeing is getting its ass absolutely handed to it in the aviation world. Use AI if you want to go bankrupt in 5 years but be rich now I think your analysis is sound from a technical perspective, but your closing statement is why AI is going to be mass adopted. The people who want to be rich now and don't care about what will happen in 5 years have been calling the shots for a very long time now, and as much as we technical folks insist this can't possibly keep going on forever, it's probably not going to stop sometime soon.
- prairieroadent 1y ago
  I'm coming to the realization that unsustainable behavior can continue for a long time... in our context, for generations
  - pas 1y ago
    The market can remain irrational longer than you can remain solvent.
    - nathan_douglas 1y ago
      Things can stay stupid longer than you can stay sane.
lloeki 1y ago
> its short termist thinking from people who don't understand what makes software complex to build Ironically you don't need AI to see this pattern. Maybe AI makes it a little bit more obvious who's thinking long term and who's not (both at the top and in the trenches) > Use AI if you want to go bankrupt in 5 years but be rich now Or, as some would put it "Use AI if you want to be rich now, exit, and have someone else go bankrupt in 5 years"
contextfree 1y ago
In the broader context, you could look further left: Conception -> design -> compiler -> code review ... If AI tools allow for better rapid prototyping, they could help catch "errors" in the conception and design phases. I don't know how useful this actually is, though.
- 20k 1y ago
  One of the problems with using AI for prototyping (or just in general), is that the act of creating the prototype is what's valuable, not the prototype itself. You learn lessons in trying to build it that you use to build the real product. Using the AI to skip the learning step and produce the prototype directly would be missing the point of prototyping at all
  - amilios 1y ago
    Sometimes you need to just pump out an MVP and start iterating. AI can exponentially speed this up in my experience.
  - contextfree 1y ago
    That's definitely an issue and I've gotten burnt by similar problems when using AI to help navigate and find things in codebases, where I've used it to read and understand code for me, with seemingly miraculous results at first, but ended up wishing I'd read more code "manually" myself, as my shallow understanding led to wasting more time on net than I saved. I still feel like it should be possible to find some kind of a balance, but it's tricky.
marcosdumay 1y ago
> That's the primary reason why I don't think AI will catch on It's almost enraging that with the hype around LLMs, the development of real automatic programming AI seems to have halted.
Nicook 1y ago
>The industry is trying to push errors leftwards. Is this true? Most software devs would like to. But I think business is more interested in spee which pushes errors to the right. Which seems to be more profitable in most software. Even stuff thats been around for a decade(s).
- pas 1y ago
  nah, in general there's a serious industry-wide push for this. software testing is changing (involve QAs early so they can help with the spec so when they get the software they know what to test against), agile is about delivering small valuable parts of the product as soon as possible, VC investing (lean startups!) is about testing business ideas as soon as possible, etc. it's all part of the shift left ideology. (same with security, you cannot really add it later, same with GDPR and other data protection stuff, you cannot track consent for various data processing purposes after you already have a lot of users onboarded - unless you want to do the sneaky very not nice "ToS updated, pay or die, kthxbai" thing [which is what Meta did], etc.) ... of course this usually means that many times people want to go from the "barely idea as a Figma proto" to "mature product maintained by distributed high-velocity teams" without realizing that there are trade-offs. shift left is makes good business and engineering sense and all, as it allows you to focus on the things that work, but it requires more iterations to go from that to something mature.
827a 1y ago
I disagree entirely, and I can convince you I'm right with one sentence: More lines of JS/TS are written by AI than lines of Rust are written at all. We don't have the data to assert this as true, but I think the vast majority of people would agree with that statement. This statement being true disproves the statement "the industry has been making huge efforts to push errors from runtime, to compile time." The industry is not a monolith. Different actors have different, individualized goals.
- pas 1y ago
  it's absolutely not a problem that people are writing more JS/TS than Rust. if that Rust is for decades and the JS/TS gets thrown out in 1 year. there's a lot of shitty C being written still around the world yet the Rust that goes into the kernel has real value long term. there's an adoption cycle. Rust is probably already well over the hype peak, and now it's slowly climbing upward to its "plateau of productivity". (and I would argue that yes there are pretty good things happening nowadays in the industry. for many domains efficient and safe libraries, frameworks, and platforms are getting to be the norm. for example see how Blender became a success story. how deterministic testing is getting adopted for distributed databases.)
- raincole 1y ago
  And I have a very hard time understanding why AI is "pushing the errors to the right". > Compiler -> code review -> tests -> runtime checks -> 'caught' in prod With AI we still compile the code. We still do code review. We still run tests (before code review, obviously; don't know why it's listed like this). We still do QA at runtime. I feel like anti-AI people are those one who actually treat AI as magic, not the AI users. AI doesn't magically prevent you from doing the things that helped you in pre-AI era.
  - Balinares 1y ago
    No, but if a shop added overnight 5 fledgling juniors for each current employee on the project, not only would the delivery not be sped up on account of Brooks's law, but the stack would soon tank under the weight of its own issues. So outside of such situations as hilarious consultancy disasters, no one did that. Now with AI they do.
dubbel 1y ago
You are looking at LLMs for code generation exclusively, but that is not the only application within software engineering. In my company some people are using LLMs to generate some of their code, but more are using them to get a first code review, before requesting a review by their colleagues. This helps getting the easy/nitpicky stuff out of the way and thereby often saves us one feedback+fix cycle. Examples would be "you changed this unit test, but didn't update the unit test name", "you changed this function but not the doc string", or "if you reorder these if statements you can avoid deep nesting". Nothing groundbreaking, but nice things. We still review like we did before, but can often focus a little more on the "what" instead of the "how". In this application, the LLM is kind of like a linter with fuzzy rules. We didn't stop reviewing code just because many languages come with standard formatters nowadays, either. While the whole code generation aspect of AI is all the rage right now (and to quote the article): > Focus on tangible changes in areas that you care about that really do seem connected to AI
- 20k 1y ago
  So while I don't disagree with you at all, in terms of AI being a bubble, none of that is why the tech is being so hyped up. The current speculative hype push is being driven by two factors: 1. The promise that AI will replace most if not all developers 2. Alternatively, that AI will turn every developer into a 10-100x developer My personal opinion is that it'll end up being one of many tools that's situationally useful, eg you're 100% right in that having it as an additional code review step is a great idea. But the amount of money being pumped into the industry isn't enough to sustain mild use cases like that and that isn't why the tech is being pushed. The trillions of dollars being dumped into improving clang tidy isn't sustainable if that's the end use case
  - giantrobot 1y ago
    > 1. The promise that AI will replace most if not all developers 2. Alternatively, that AI will turn every developer into a 10-100x developer The AI hype train is promising to deliver the 20 year old developer with 30 years of experience that a company can pay $10 an hour.
darksaints 1y ago
This has been a huge frustration for me, but the wild thing is that we've built up so many tools over time that help humans only for AI coding tools to wild west it and not use them. The best AI coding tools will read docs websites, terminal error messages, write/run tests, etc. But we have so many better tools that none of them seem to use: * profilers * debuggers * linters * static analyzers * language server protocol * wire protocol analyzers * decompilers * call graph analyzers * database structure crawlers In the absence of models that can do perfect oneshot software engineering, we're gonna have to fall back on well-integrated tool usage, and nobody seems to do that well yet.
- deleted 1y ago
  [deleted]
- 20k 1y ago
  I think a lot of these use cases for AI are incidental byproducts of the actual goal, which is to replace software developers. They're trying to salvage some kind of utility. Because I agree that the AI tools in use are marginal improvements, or downgrades in a lot of cases I've heard people say they use AI agents to set up a new project with git. Just use tortoisegit or something, its free and takes one click - its just using AI for the sake of it
aerhardt 1y ago
Exabytes of code are being written in Python and JS though… I don’t think that fits with your narrative that everything is being pushed to compile-time. C#, Java and Go remain popular sure but have they grown that much relative to other languages? Rust is being adopted primarily in projects that used to be in C or C++ if I’m not mistaken.
- 20k 1y ago
  Those languages have been going through exactly the same evolution though, like the JS -> typescript migration is one of the most direct practical examples of this imo
abalashov 1y ago
This is the best and most enlightening take I've heard in a good while. I have articulated this to friends and colleagues who are on the LLM hype train somewhat differently, in terms of the unwieldiness of accumulated errors and entropy, disproportionate human bottlenecks when you do have to engage with the code but didn't write any of it and don't understand it, etc. However, your formulation really ties a lot of this together better. Thanks!
bwfan123 1y ago
thanks for articulating so nicely what needs to be said in this debate. Pushing errors leftward vs rightwards is such a nice metaphor, not to mention the metaphor on mental models. Also, your comment on why natural language is unable to describe the problem adequately (later in this thread), since sometimes constraints are discovered during the solution process, and otherwise, and if problems could be described adequately, thats what we call a programming language is very nice - ie, for natural language to adequately describe a problem, that becomes a formal language. Only experienced engineers who have been through failed projects will understand what you are saying, and the rest of those in the grip of the ai-mania will come to terms with it soon.
dimal 1y ago
Vibe coding pushes errors rightward, but using AI to speed up typing or summarizing documentation doesn’t. Vibe coding will fail, but that doesn’t mean using AI to code will fail. You’re looking at one (admittedly stupid) use case and generalizing too hastily. If I have an LLM fix a bug where it gets the feedback from the type checker, linter and tests in realtime, no errors were pushed rightward. It’s not a free lunch though. I still have to refactor afterwards or else I’ll be adding tech debt. To do that, I need to have an accurate mental model of the problem. I think this is where most people will go wrong. Most people have a mindset of “if it compiles and works, it ships.” This will lead to a tangled mess. Basically, if people treat AI as a silver bullet for dealing with complexity, they’re going to have a bad time. There still is no silver bullet.

by 20k 1y ago

WaxProlix 1y ago

That's true up to a certain threshold on 'probabilistically correct', right? At a certain number of 9s, it's fine. And increasingly I use AI to help ask me questions, refine my understanding of problem spaces, do deep research on existing patterns or trends in a space and then use the results as context to have a planning session, which provides context for architecture, etc. So, I don't know that the tools are inherently rightward-pushing

20k 1y ago
The problem is, given the inherent limitations of natural language as a format to feed to an AI, it can never have enough information to be able to solve your problem adequately. Often the constraints of what you're trying to solve only crop up during the process of trying to solve the problem itself, as it was unclear that they even existed beforehand An AI tool that could have a precise enough specification fed into it to produce the result that you wanted with no errors, would be a programming language I don't disagree at all that AI can be helpful, but there's a huge difference between using it as a research tool (which is very valid), and the folks who are trying to use it to replace programmers en masse. The latter is what's driving the bubble, not the former
B56b 1y ago
AI code reliability is nowhere near any number of 9s

bugglebeetle 1y ago

I’m not sure this follows as SOTA LLMs are pretty good at writing Rust, so wouldn’t this also make it easier for codebases to move leftward in your analogy? For example, I was resistant to use Rust for a lot of things because a). The community is somewhat annoying and pedantic, even by software engineering standards b). The overhead in getting colleagues up to speed on Rust code was too much of a time suck. LLMs solve both those problems and we’re now migrating lots of stuff to Rust, my colleagues can ask lots of questions (of Google Gemini Pro 2.5), without burdening anyone or being met with disdain, and seem generally more curious and positive about these moves/Rust overall.

jjk166 1y ago

> It'll be fine in the short term, but in the long term it'll cost you more money. That's the primary reason why I don't think AI will catch on - its short termist thinking from people who don't understand what makes software complex to build, or how to actually produce software that's cheap in the long term. Its also exactly the same reason that Boeing is getting its ass absolutely handed to it in the aviation world. Use AI if you want to go bankrupt in 5 years but be rich now I think your analysis is sound from a technical perspective, but your closing statement is why AI is going to be mass adopted. The people who want to be rich now and don't care about what will happen in 5 years have been calling the shots for a very long time now, and as much as we technical folks insist this can't possibly keep going on forever, it's probably not going to stop sometime soon.

prairieroadent 1y ago
I'm coming to the realization that unsustainable behavior can continue for a long time... in our context, for generations
- pas 1y ago
  The market can remain irrational longer than you can remain solvent.
  - nathan_douglas 1y ago
    Things can stay stupid longer than you can stay sane.

lloeki 1y ago

> its short termist thinking from people who don't understand what makes software complex to build Ironically you don't need AI to see this pattern. Maybe AI makes it a little bit more obvious who's thinking long term and who's not (both at the top and in the trenches) > Use AI if you want to go bankrupt in 5 years but be rich now Or, as some would put it "Use AI if you want to be rich now, exit, and have someone else go bankrupt in 5 years"

contextfree 1y ago

In the broader context, you could look further left: Conception -> design -> compiler -> code review ... If AI tools allow for better rapid prototyping, they could help catch "errors" in the conception and design phases. I don't know how useful this actually is, though.

20k 1y ago
One of the problems with using AI for prototyping (or just in general), is that the act of creating the prototype is what's valuable, not the prototype itself. You learn lessons in trying to build it that you use to build the real product. Using the AI to skip the learning step and produce the prototype directly would be missing the point of prototyping at all
- amilios 1y ago
  Sometimes you need to just pump out an MVP and start iterating. AI can exponentially speed this up in my experience.
- contextfree 1y ago
  That's definitely an issue and I've gotten burnt by similar problems when using AI to help navigate and find things in codebases, where I've used it to read and understand code for me, with seemingly miraculous results at first, but ended up wishing I'd read more code "manually" myself, as my shallow understanding led to wasting more time on net than I saved. I still feel like it should be possible to find some kind of a balance, but it's tricky.

marcosdumay 1y ago

> That's the primary reason why I don't think AI will catch on It's almost enraging that with the hype around LLMs, the development of real automatic programming AI seems to have halted.

Nicook 1y ago

>The industry is trying to push errors leftwards. Is this true? Most software devs would like to. But I think business is more interested in spee which pushes errors to the right. Which seems to be more profitable in most software. Even stuff thats been around for a decade(s).

pas 1y ago
nah, in general there's a serious industry-wide push for this. software testing is changing (involve QAs early so they can help with the spec so when they get the software they know what to test against), agile is about delivering small valuable parts of the product as soon as possible, VC investing (lean startups!) is about testing business ideas as soon as possible, etc. it's all part of the shift left ideology. (same with security, you cannot really add it later, same with GDPR and other data protection stuff, you cannot track consent for various data processing purposes after you already have a lot of users onboarded - unless you want to do the sneaky very not nice "ToS updated, pay or die, kthxbai" thing [which is what Meta did], etc.) ... of course this usually means that many times people want to go from the "barely idea as a Figma proto" to "mature product maintained by distributed high-velocity teams" without realizing that there are trade-offs. shift left is makes good business and engineering sense and all, as it allows you to focus on the things that work, but it requires more iterations to go from that to something mature.

827a 1y ago

I disagree entirely, and I can convince you I'm right with one sentence: More lines of JS/TS are written by AI than lines of Rust are written at all. We don't have the data to assert this as true, but I think the vast majority of people would agree with that statement. This statement being true disproves the statement "the industry has been making huge efforts to push errors from runtime, to compile time." The industry is not a monolith. Different actors have different, individualized goals.

pas 1y ago
it's absolutely not a problem that people are writing more JS/TS than Rust. if that Rust is for decades and the JS/TS gets thrown out in 1 year. there's a lot of shitty C being written still around the world yet the Rust that goes into the kernel has real value long term. there's an adoption cycle. Rust is probably already well over the hype peak, and now it's slowly climbing upward to its "plateau of productivity". (and I would argue that yes there are pretty good things happening nowadays in the industry. for many domains efficient and safe libraries, frameworks, and platforms are getting to be the norm. for example see how Blender became a success story. how deterministic testing is getting adopted for distributed databases.)
raincole 1y ago
And I have a very hard time understanding why AI is "pushing the errors to the right". > Compiler -> code review -> tests -> runtime checks -> 'caught' in prod With AI we still compile the code. We still do code review. We still run tests (before code review, obviously; don't know why it's listed like this). We still do QA at runtime. I feel like anti-AI people are those one who actually treat AI as magic, not the AI users. AI doesn't magically prevent you from doing the things that helped you in pre-AI era.
- Balinares 1y ago
  No, but if a shop added overnight 5 fledgling juniors for each current employee on the project, not only would the delivery not be sped up on account of Brooks's law, but the stack would soon tank under the weight of its own issues. So outside of such situations as hilarious consultancy disasters, no one did that. Now with AI they do.

dubbel 1y ago

You are looking at LLMs for code generation exclusively, but that is not the only application within software engineering. In my company some people are using LLMs to generate some of their code, but more are using them to get a first code review, before requesting a review by their colleagues. This helps getting the easy/nitpicky stuff out of the way and thereby often saves us one feedback+fix cycle. Examples would be "you changed this unit test, but didn't update the unit test name", "you changed this function but not the doc string", or "if you reorder these if statements you can avoid deep nesting". Nothing groundbreaking, but nice things. We still review like we did before, but can often focus a little more on the "what" instead of the "how". In this application, the LLM is kind of like a linter with fuzzy rules. We didn't stop reviewing code just because many languages come with standard formatters nowadays, either. While the whole code generation aspect of AI is all the rage right now (and to quote the article): > Focus on tangible changes in areas that you care about that really do seem connected to AI

20k 1y ago
So while I don't disagree with you at all, in terms of AI being a bubble, none of that is why the tech is being so hyped up. The current speculative hype push is being driven by two factors: 1. The promise that AI will replace most if not all developers 2. Alternatively, that AI will turn every developer into a 10-100x developer My personal opinion is that it'll end up being one of many tools that's situationally useful, eg you're 100% right in that having it as an additional code review step is a great idea. But the amount of money being pumped into the industry isn't enough to sustain mild use cases like that and that isn't why the tech is being pushed. The trillions of dollars being dumped into improving clang tidy isn't sustainable if that's the end use case
- giantrobot 1y ago
  > 1. The promise that AI will replace most if not all developers 2. Alternatively, that AI will turn every developer into a 10-100x developer The AI hype train is promising to deliver the 20 year old developer with 30 years of experience that a company can pay $10 an hour.

darksaints 1y ago

This has been a huge frustration for me, but the wild thing is that we've built up so many tools over time that help humans only for AI coding tools to wild west it and not use them. The best AI coding tools will read docs websites, terminal error messages, write/run tests, etc. But we have so many better tools that none of them seem to use: * profilers * debuggers * linters * static analyzers * language server protocol * wire protocol analyzers * decompilers * call graph analyzers * database structure crawlers In the absence of models that can do perfect oneshot software engineering, we're gonna have to fall back on well-integrated tool usage, and nobody seems to do that well yet.

deleted 1y ago
[deleted]
20k 1y ago
I think a lot of these use cases for AI are incidental byproducts of the actual goal, which is to replace software developers. They're trying to salvage some kind of utility. Because I agree that the AI tools in use are marginal improvements, or downgrades in a lot of cases I've heard people say they use AI agents to set up a new project with git. Just use tortoisegit or something, its free and takes one click - its just using AI for the sake of it

aerhardt 1y ago

Exabytes of code are being written in Python and JS though… I don’t think that fits with your narrative that everything is being pushed to compile-time. C#, Java and Go remain popular sure but have they grown that much relative to other languages? Rust is being adopted primarily in projects that used to be in C or C++ if I’m not mistaken.

20k 1y ago
Those languages have been going through exactly the same evolution though, like the JS -> typescript migration is one of the most direct practical examples of this imo

abalashov 1y ago

This is the best and most enlightening take I've heard in a good while. I have articulated this to friends and colleagues who are on the LLM hype train somewhat differently, in terms of the unwieldiness of accumulated errors and entropy, disproportionate human bottlenecks when you do have to engage with the code but didn't write any of it and don't understand it, etc. However, your formulation really ties a lot of this together better. Thanks!

bwfan123 1y ago

thanks for articulating so nicely what needs to be said in this debate. Pushing errors leftward vs rightwards is such a nice metaphor, not to mention the metaphor on mental models. Also, your comment on why natural language is unable to describe the problem adequately (later in this thread), since sometimes constraints are discovered during the solution process, and otherwise, and if problems could be described adequately, thats what we call a programming language is very nice - ie, for natural language to adequately describe a problem, that becomes a formal language. Only experienced engineers who have been through failed projects will understand what you are saying, and the rest of those in the grip of the ai-mania will come to terms with it soon.

dimal 1y ago

Vibe coding pushes errors rightward, but using AI to speed up typing or summarizing documentation doesn’t. Vibe coding will fail, but that doesn’t mean using AI to code will fail. You’re looking at one (admittedly stupid) use case and generalizing too hastily. If I have an LLM fix a bug where it gets the feedback from the type checker, linter and tests in realtime, no errors were pushed rightward. It’s not a free lunch though. I still have to refactor afterwards or else I’ll be adding tech debt. To do that, I need to have an accurate mental model of the problem. I think this is where most people will go wrong. Most people have a mindset of “if it compiles and works, it ships.” This will lead to a tangled mess. Basically, if people treat AI as a silver bullet for dealing with complexity, they’re going to have a bad time. There still is no silver bullet.