5 ms·

> Vibe coding would be catastrophic here. Not because the AI can't write the code - it usually can - but because the failure mode is invisible. A hallucinated e

> Vibe coding would be catastrophic here. Not because the AI can't write the code - it usually can - but because the failure mode is invisible. A hallucinated edge case in a tax calculation doesn't throw an error. It just produces a slightly wrong number that gets posted to a real accounting platform and nobody notices until the accountant does their review. How is that different from handwritten code ? Sounds like stuff you deal with architecturally (auditable/with review/rollback) and with tests.

kamaal 8mo ago
>>How is that different from handwritten code ? I think the point he is trying to make is that you can't outsource your thinking to a automated process and also trust it to make the right decisions at the same time. In places where a number, fraction, or a non binary outcome is involved there is an aspect of growing the code base with time and human knowledge/failure. You could argue that speed of writing code isn't everything, many times being correct and stable likely is more important. For eg- A banking app, doesn't have be written and shipped fast. But it has to be done right. ECG machines, money, meat space safety automation all come under this.
- rafaelmn 8mo ago
  Replace LLM with employee in your argument - what changes ? Unless everyone at your workplace owns the system they are working on - this is a very high bar and maybe 50% of devs I've worked with are capable of owning a piece of non trivial code, especially if they didn't write it. Realiy is you don't solve these problems by to relying on everyone to be perfect - everyone slips up - to achieve results consistently you need process/systems to assure quality. Safety critical system should be even better equipped to adopt this because they already have the systems to promote correct outputs. The problem is those systems weren't built for LLMs specifically so the unexpected failure cases and the volume might not be a perfect fit - but then you work on adapting the quality control system.
  - kamaal 8mo ago
    >>replace LLM with employee in your argument - what changes ? I mentioned this part in my comment. You cannot trust an automated process to a thing, and expect the same process to verify if it did it right. This is with regards to any automated process, not just code. This is not the same as manufacturing, as in manufacturing you make the same part thousands of times. In code the automated process makes a specific customised thing only once, and it has to be right. >>The problem is those systems weren't built for LLMs specifically so the unexpected failure cases ... We are not talking of failures. There is a space between success and failure where the LLM can go into easily.
skydhash 8mo ago
With handwritten code, the humans know what they don’t know. If you want some constants or some formula, you don’t invent or guess it, you ask the domain expert.
- michaelcampbell 8mo ago
  > With handwritten code, the humans know what they don’t know. I find this often not to be the case at all.
  - Fellshard 8mo ago
    Let's put it this way: the human author is capable of doing so. The LLM is not. You can cultivate the human to learn to think in this way. You can for a brief period coerce an LLM to do so.
    - michaelcampbell 8mo ago
      True, but IMO irrelevant. "What could have been" (capabilities) is just another "if only..."
ryan_n 8mo ago
It’s shocking to me that people even ask this type of question. How do you not see the difference between a machine that will hallucinate something random if it doesn’t know the answer vs a human that will logic through things and find the correct answer.
- d-j-k 8mo ago
  [dead]
- slekker 8mo ago
  I know right, had this discussion last week and it's difficult to argue when people are blinded by the "magic" and hype of the slot machine.
  - roxolotl 8mo ago
    Which interestingly is the meat of this article. The key points aren’t that “vibe coding is bad” but that the design and experience of these tools is actively blinding and seductive in a way that impairs ability to judge effectiveness.
    - mystraline 8mo ago
      Basically, instead of developers developing, they've been half-elevated to the management class where they manage really dumb but really fast interns (LLM's). But they dont get the management pay, and they are 100% responsible for the LLMs under them. Whereas real managers get paid more and can lay blame and fire people under them.
- aspenmartin 8mo ago
  It's shocking to me that people make this claim as if humans, especially in some legacy accounting system, would somehow be much better at (1) recognizing their mistakes, and (2) even when they don't, not fudge-fingering their implementation. Like the criticisms of agents are valid, but the incredulity that they will ever be used in production or high risk systems to me is just as incredible. Of course they will -- where is Opus 4.6 compared to Sonnet 4? We've hit an inflection point where replacing hand coding with an agent and interacting only via prompt is not only doable, highly skilled people are already routinely doing it. Companies are already _requiring_ that people do it. We will then hit an inflection point at some time soon where the incredulity at using agents even in the highest stakes application will age really really poorly. Let's see!
jayd16 8mo ago
One major difference is the code has an owner who might consider what needs a test or ask questions if they don't understand. To argue that all work is fungible because perfection cannot be achieved is actually a pretty out there take. Replace your thought experiment with "Is one shot consultant code different from expert code?" Yes. They are different. Code review is good and needed for human code, right? But if its "vibe coded", suddenly its not important? The differences are clear.
- linsomniac 8mo ago
  >perfection cannot be achieved That's not what I get out of the comment you are replying to. In the case being discussed here, one of code matching the tax code, perfection is likely possible; perfection is defined by the tax code. The SME on this should be writing the tests that demonstrate adhering with the tax code. Once they do that, then it doesn't matter if they, or the AI, or a one shot consultant write it, as far as correctness goes. If the resulting AI code has subtle bugs in it that pass the test, the SME likely didn't understand the corner cases of this part of the tax code as well as they thought, and quite possibly could have run into the same bugs. That's what I get out of what you are replying to.
tqian 8mo ago
Humans make such mistakes slowly. It's much harder to catch the "drift" introduced by LLM because it happens so quickly and silently. By the time you notice something is wrong, it has already become the foundation for more code. You are then looking at a full rewrite.
- hinkley 8mo ago
  The rate of the mistakes versus the rate of consumers and testers finding them was a ratio we could deal with and we don’t have the facilities to deal with the new ratio. It is likely over time that AI code will necessitate the use of more elaborate canary systems that increase the cost per feature quite considerably. Particularly for small and mid sized orgs where those costs are difficult to amortize. Or maybe this is a SaaS opportunity for someone.

kamaal 8mo ago

>>How is that different from handwritten code ? I think the point he is trying to make is that you can't outsource your thinking to a automated process and also trust it to make the right decisions at the same time. In places where a number, fraction, or a non binary outcome is involved there is an aspect of growing the code base with time and human knowledge/failure. You could argue that speed of writing code isn't everything, many times being correct and stable likely is more important. For eg- A banking app, doesn't have be written and shipped fast. But it has to be done right. ECG machines, money, meat space safety automation all come under this.

rafaelmn 8mo ago
Replace LLM with employee in your argument - what changes ? Unless everyone at your workplace owns the system they are working on - this is a very high bar and maybe 50% of devs I've worked with are capable of owning a piece of non trivial code, especially if they didn't write it. Realiy is you don't solve these problems by to relying on everyone to be perfect - everyone slips up - to achieve results consistently you need process/systems to assure quality. Safety critical system should be even better equipped to adopt this because they already have the systems to promote correct outputs. The problem is those systems weren't built for LLMs specifically so the unexpected failure cases and the volume might not be a perfect fit - but then you work on adapting the quality control system.
- kamaal 8mo ago
  >>replace LLM with employee in your argument - what changes ? I mentioned this part in my comment. You cannot trust an automated process to a thing, and expect the same process to verify if it did it right. This is with regards to any automated process, not just code. This is not the same as manufacturing, as in manufacturing you make the same part thousands of times. In code the automated process makes a specific customised thing only once, and it has to be right. >>The problem is those systems weren't built for LLMs specifically so the unexpected failure cases ... We are not talking of failures. There is a space between success and failure where the LLM can go into easily.

skydhash 8mo ago

With handwritten code, the humans know what they don’t know. If you want some constants or some formula, you don’t invent or guess it, you ask the domain expert.

michaelcampbell 8mo ago
> With handwritten code, the humans know what they don’t know. I find this often not to be the case at all.
- Fellshard 8mo ago
  Let's put it this way: the human author is capable of doing so. The LLM is not. You can cultivate the human to learn to think in this way. You can for a brief period coerce an LLM to do so.
  - michaelcampbell 8mo ago
    True, but IMO irrelevant. "What could have been" (capabilities) is just another "if only..."

ryan_n 8mo ago

It’s shocking to me that people even ask this type of question. How do you not see the difference between a machine that will hallucinate something random if it doesn’t know the answer vs a human that will logic through things and find the correct answer.

d-j-k 8mo ago
[dead]
slekker 8mo ago
I know right, had this discussion last week and it's difficult to argue when people are blinded by the "magic" and hype of the slot machine.
- roxolotl 8mo ago
  Which interestingly is the meat of this article. The key points aren’t that “vibe coding is bad” but that the design and experience of these tools is actively blinding and seductive in a way that impairs ability to judge effectiveness.
  - mystraline 8mo ago
    Basically, instead of developers developing, they've been half-elevated to the management class where they manage really dumb but really fast interns (LLM's). But they dont get the management pay, and they are 100% responsible for the LLMs under them. Whereas real managers get paid more and can lay blame and fire people under them.
aspenmartin 8mo ago
It's shocking to me that people make this claim as if humans, especially in some legacy accounting system, would somehow be much better at (1) recognizing their mistakes, and (2) even when they don't, not fudge-fingering their implementation. Like the criticisms of agents are valid, but the incredulity that they will ever be used in production or high risk systems to me is just as incredible. Of course they will -- where is Opus 4.6 compared to Sonnet 4? We've hit an inflection point where replacing hand coding with an agent and interacting only via prompt is not only doable, highly skilled people are already routinely doing it. Companies are already _requiring_ that people do it. We will then hit an inflection point at some time soon where the incredulity at using agents even in the highest stakes application will age really really poorly. Let's see!

jayd16 8mo ago

One major difference is the code has an owner who might consider what needs a test or ask questions if they don't understand. To argue that all work is fungible because perfection cannot be achieved is actually a pretty out there take. Replace your thought experiment with "Is one shot consultant code different from expert code?" Yes. They are different. Code review is good and needed for human code, right? But if its "vibe coded", suddenly its not important? The differences are clear.

linsomniac 8mo ago
>perfection cannot be achieved That's not what I get out of the comment you are replying to. In the case being discussed here, one of code matching the tax code, perfection is likely possible; perfection is defined by the tax code. The SME on this should be writing the tests that demonstrate adhering with the tax code. Once they do that, then it doesn't matter if they, or the AI, or a one shot consultant write it, as far as correctness goes. If the resulting AI code has subtle bugs in it that pass the test, the SME likely didn't understand the corner cases of this part of the tax code as well as they thought, and quite possibly could have run into the same bugs. That's what I get out of what you are replying to.

tqian 8mo ago

Humans make such mistakes slowly. It's much harder to catch the "drift" introduced by LLM because it happens so quickly and silently. By the time you notice something is wrong, it has already become the foundation for more code. You are then looking at a full rewrite.

hinkley 8mo ago
The rate of the mistakes versus the rate of consumers and testers finding them was a ratio we could deal with and we don’t have the facilities to deal with the new ratio. It is likely over time that AI code will necessitate the use of more elaborate canary systems that increase the cost per feature quite considerably. Particularly for small and mid sized orgs where those costs are difficult to amortize. Or maybe this is a SaaS opportunity for someone.