9 ms·

A compiler uses rigorous modeling and testing to ensure that generated code is semantically equivalent. It can do this because it is translating from one formal

A compiler uses rigorous modeling and testing to ensure that generated code is semantically equivalent. It can do this because it is translating from one formal language to another. Translating a natural prompt on the other hand requires the LLM to make thousands of small decisions that will be different each time you regenerate the artifact. Even ignoring non-determinism, prompt instability means that any small change to the spec will result in a vastly different program. A natural language spec and test suite cannot be complete enough to encode all of these differences without being at least as complex as the code. Therefore each time you regenerate large sections of code without review, you will see scores of observable behavior differences that will surface to the user as churn, jank, and broken workflows. Your tests will not encode every user workflow, not even close. Ask yourself if you have ever worked on a non trivial piece of software where you could randomly regenerate 10% of the implementation while keeping to the spec without seeing a flurry of bug reports. This may change if LLMs improve such that they are able to reason about code changes to the degree a human can. As of today they cannot do this and require tests and human code review to prevent them from spinning out. But I suspect at that point they’ll be doing our job, as well as the CEOs and we’ll have bigger problems.

raw_anon_1111 7mo ago
As if when you delegate tasks to humans they are deterministic. I would hope that your test cases cover the requirements. If not, your implementation is just as brittle when other developers come online or even when you come back to a project after six months.
- sarchertech 7mo ago
  1. Agents aren’t humans. A human can write a working 100k LOC application with zero tests (not saying they should but they could and have). An agent cannot do this. Agents require tests to keep them from spinning out and your tests do not cover all of the behaviors you care about. 2. If you doubt that your tests don’t cover all your requirements, 99.9% of every production bug you’ve ever had completely passed your test suite.
  - raw_anon_1111 7mo ago
    I have never known a human that could or did write 100K lines of bug free working code without running parts of it first and testing. So humans also don’t write bug free code or tests that cover all use cases - how is that an argument that humans are better?
    - sarchertech 7mo ago
      Not that humans can't write 100k line programs bug free or without running parts of it. An AI cannot write a 100k line program on its own without external guard rails otherwise it spins out. This has nothing to do with whether the agent is allowed to run the code itself. This is well documented. Look at what was required to allow Claude to write a "C compiler". This has nothing to do with whether it's bug free. It literally can't produce a working 100k LOC program without external guardrails.
      - raw_anon_1111 7mo ago
        Absolutely no one is arguing that you shouldn’t have a combination of manual and automated tests around either AI or human generated code or that you shouldn’t have a thoughtful design
        sarchertech 7mo ago
        In a non-trivial app you can't test your way through all of the e2e workflows and thoughtful design isn't what I'm talking about. How many bugs have you seen that passed your automated and manual testing? Probably 99.9% of them. Now imagine that you take those same test suites and you unleash an agent on the code that has far worse reasoning capabilities than a human and you tell them they can change anything in the code as long as the tests pass.
        raw_anon_1111 7mo ago
        So if bugs pass through testing which they have forever, wouldn’t that imply that humans are just as fallible as AI - and slower? I never suggested letting agents code for a day on end. I use AI to code well defined tasks and treat it like a mid level ticket taker
petcat 7mo ago
> A compiler uses rigorous modeling and testing to ensure that generated code is semantically equivalent. Here are the reported miscompilation bugs in GCC so far in 2026. The ones labeled "wrong-code". https://gcc.gnu.org/bugzilla/buglist.cgi?chfield=%5BBug%20creation%5D&chfieldfrom=2026-01-01&chfieldto=2026-12-31&keywords=wrong-code&keywords_type=allwords https://gcc.gnu.org/bugzilla/buglist.cgi?chfield=%5BBug%20cr... I count 121 of them.
- sarchertech 7mo ago
  If you can’t understand the difference between a bug that will rarely cause a compiler encountering an edge case to generate a wrong instruction and an LLM that will generate 2 completely different programs with zero overlap because you added a single word to your prompt, then I don’t know what to tell you.
  - petcat 7mo ago
    The point is that expert humans (the GCC developers) writing code (C++) that generates code (ASM) does not appear to be as deterministic as you seem to think it is.
    - sarchertech 7mo ago
      I’m very aware of that, but I’m also aware that it’s rare enough that the compiler doesn’t emit semantically equivalent code that most people can ignore it. That’s not the case with LLMs. I’m also not particularly concerned with non-determinism but with chaos. Determinism in LLMs is likely solvable, prompt instability is not.
    - jplusequalt 7mo ago
      Classic HN-ism. To focus on the semantics of a statement while ignoring the greater point in order to argue why someone is wrong.
      - petcat 7mo ago
LogicFailsMe 7mo ago
I don't see a world where a motivated soul can build a business from a laptop and a token service as a problem. I see it as opportunity. I feel similarly about Hollywood and the creation of media. We're not there in either case yet, but we will be. That's pretty clear. and when I look at the feudal society that is the entertainment industry here, I don't understand why so many of the serfs are trying to perpetuate it in its current state. And I really don't get why engineers think this technology is going to turn them into serfs unless they let that happen to them themselves. If you can build things, AI coding agents will let you build faster and more for the same amount of effort. I am assuming given the rate of advance of AI coding systems in the past year that there is plenty of improvement to come before this plateaus. I'm sure that will include AI generated systems to do security reviews that will be at human or better level. I've already seen Claude find 20 plus-year-old bugs in my own code. They weren't particularly mission critical but they were there the whole time. I've also seen it do amazingly sophisticated reverse engineering of assembly code only to fall over flat on its face for the simplest tasks.
- sarchertech 7mo ago
  That depends on how fast that change happens. If 45% of jobs evaporate in a a 5 year period, a complete societal collapse is the likely outcome.
  - LogicFailsMe 7mo ago
    Sounds like influencer nonsense to me. Touch grass. If the people are fed and housed, there's no collapse. And if the billionaire class lets them starve, they will finally go through some things just like the aristocracy in France once did. And I think even Peter Thiel is smarter than that. You can feed yourself for <$1000 a year on beans and rice. Not saying you'd enjoy it, but you won't starve. So for ~$40B annually, the billionaires buy themselves revolution insurance. Fantastic value. OTOH if what you're really talking about is the long-term collapse in our ludicrous carbon footprint when we finally run out of fossil fuels and we didn't invest in renewables or nuclear to replace them, well, I'm with you there.

raw_anon_1111 7mo ago

As if when you delegate tasks to humans they are deterministic. I would hope that your test cases cover the requirements. If not, your implementation is just as brittle when other developers come online or even when you come back to a project after six months.

sarchertech 7mo ago
1. Agents aren’t humans. A human can write a working 100k LOC application with zero tests (not saying they should but they could and have). An agent cannot do this. Agents require tests to keep them from spinning out and your tests do not cover all of the behaviors you care about. 2. If you doubt that your tests don’t cover all your requirements, 99.9% of every production bug you’ve ever had completely passed your test suite.
- raw_anon_1111 7mo ago
  I have never known a human that could or did write 100K lines of bug free working code without running parts of it first and testing. So humans also don’t write bug free code or tests that cover all use cases - how is that an argument that humans are better?
  - sarchertech 7mo ago
    Not that humans can't write 100k line programs bug free or without running parts of it. An AI cannot write a 100k line program on its own without external guard rails otherwise it spins out. This has nothing to do with whether the agent is allowed to run the code itself. This is well documented. Look at what was required to allow Claude to write a "C compiler". This has nothing to do with whether it's bug free. It literally can't produce a working 100k LOC program without external guardrails.
    - raw_anon_1111 7mo ago
      Absolutely no one is arguing that you shouldn’t have a combination of manual and automated tests around either AI or human generated code or that you shouldn’t have a thoughtful design
      - sarchertech 7mo ago
        In a non-trivial app you can't test your way through all of the e2e workflows and thoughtful design isn't what I'm talking about. How many bugs have you seen that passed your automated and manual testing? Probably 99.9% of them. Now imagine that you take those same test suites and you unleash an agent on the code that has far worse reasoning capabilities than a human and you tell them they can change anything in the code as long as the tests pass.
        raw_anon_1111 7mo ago
        So if bugs pass through testing which they have forever, wouldn’t that imply that humans are just as fallible as AI - and slower? I never suggested letting agents code for a day on end. I use AI to code well defined tasks and treat it like a mid level ticket taker

petcat 7mo ago

> A compiler uses rigorous modeling and testing to ensure that generated code is semantically equivalent. Here are the reported miscompilation bugs in GCC so far in 2026. The ones labeled "wrong-code". https://gcc.gnu.org/bugzilla/buglist.cgi?chfield=%5BBug%20creation%5D&chfieldfrom=2026-01-01&chfieldto=2026-12-31&keywords=wrong-code&keywords_type=allwords https://gcc.gnu.org/bugzilla/buglist.cgi?chfield=%5BBug%20cr... I count 121 of them.

sarchertech 7mo ago
If you can’t understand the difference between a bug that will rarely cause a compiler encountering an edge case to generate a wrong instruction and an LLM that will generate 2 completely different programs with zero overlap because you added a single word to your prompt, then I don’t know what to tell you.
- petcat 7mo ago
  The point is that expert humans (the GCC developers) writing code (C++) that generates code (ASM) does not appear to be as deterministic as you seem to think it is.
  - sarchertech 7mo ago
    I’m very aware of that, but I’m also aware that it’s rare enough that the compiler doesn’t emit semantically equivalent code that most people can ignore it. That’s not the case with LLMs. I’m also not particularly concerned with non-determinism but with chaos. Determinism in LLMs is likely solvable, prompt instability is not.
  - jplusequalt 7mo ago
    Classic HN-ism. To focus on the semantics of a statement while ignoring the greater point in order to argue why someone is wrong.
    - petcat 7mo ago

LogicFailsMe 7mo ago

I don't see a world where a motivated soul can build a business from a laptop and a token service as a problem. I see it as opportunity. I feel similarly about Hollywood and the creation of media. We're not there in either case yet, but we will be. That's pretty clear. and when I look at the feudal society that is the entertainment industry here, I don't understand why so many of the serfs are trying to perpetuate it in its current state. And I really don't get why engineers think this technology is going to turn them into serfs unless they let that happen to them themselves. If you can build things, AI coding agents will let you build faster and more for the same amount of effort. I am assuming given the rate of advance of AI coding systems in the past year that there is plenty of improvement to come before this plateaus. I'm sure that will include AI generated systems to do security reviews that will be at human or better level. I've already seen Claude find 20 plus-year-old bugs in my own code. They weren't particularly mission critical but they were there the whole time. I've also seen it do amazingly sophisticated reverse engineering of assembly code only to fall over flat on its face for the simplest tasks.

sarchertech 7mo ago
That depends on how fast that change happens. If 45% of jobs evaporate in a a 5 year period, a complete societal collapse is the likely outcome.
- LogicFailsMe 7mo ago
  Sounds like influencer nonsense to me. Touch grass. If the people are fed and housed, there's no collapse. And if the billionaire class lets them starve, they will finally go through some things just like the aristocracy in France once did. And I think even Peter Thiel is smarter than that. You can feed yourself for <$1000 a year on beans and rice. Not saying you'd enjoy it, but you won't starve. So for ~$40B annually, the billionaires buy themselves revolution insurance. Fantastic value. OTOH if what you're really talking about is the long-term collapse in our ludicrous carbon footprint when we finally run out of fossil fuels and we didn't invest in renewables or nuclear to replace them, well, I'm with you there.