3 ms·

If you can’t understand the difference between a bug that will rarely cause a compiler encountering an edge case to generate a wrong instruction and an LLM that

If you can’t understand the difference between a bug that will rarely cause a compiler encountering an edge case to generate a wrong instruction and an LLM that will generate 2 completely different programs with zero overlap because you added a single word to your prompt, then I don’t know what to tell you.

petcat 7mo ago
The point is that expert humans (the GCC developers) writing code (C++) that generates code (ASM) does not appear to be as deterministic as you seem to think it is.
- sarchertech 7mo ago
  I’m very aware of that, but I’m also aware that it’s rare enough that the compiler doesn’t emit semantically equivalent code that most people can ignore it. That’s not the case with LLMs. I’m also not particularly concerned with non-determinism but with chaos. Determinism in LLMs is likely solvable, prompt instability is not.
- jplusequalt 7mo ago
  Classic HN-ism. To focus on the semantics of a statement while ignoring the greater point in order to argue why someone is wrong.
  - petcat 7mo ago
    I argued the greater point? Software code-generation is not deterministic, whether it's done by expert humans or by LLMs.
    - sarchertech 7mo ago
      It has nothing to do with determinism. It's the difference between nearly perfectly but not quite perfectly translating between rigorously specified formal languages and translating an ambiguous natural language specification into a formal one. The first is a purely mechanical process, the second is not and requires thousands of decisions that can go either way.
      - raw_anon_1111 7mo ago
        And that’s no different than human developers
        sarchertech 7mo ago
        The difference is that a human is that a human can reason about their code changes to a much higher degree than an AI can. If you don't think this is true and you think we're working with AGI, why would you bother architecting anything all or building in any guard rails. Why not just feed the AI the text of the contract your working from and let it rip.
        raw_anon_1111 7mo ago
        You give way too much credit to the average mid level ticket taker. And again, why do I care how the code does it as long as it meets the functional and none functional requirements?
        sarchertech 7mo ago
        Because in a real application with real users all of the functional and non-functional requirements aren't documented anywhere but in the code.
        raw_anon_1111 7mo ago
        If only a coding agent had access to your code…
        sarchertech 7mo ago
        You realize that coding agents aren’t AGI right? They aren’t capable of reasoning about a code changes impact on anything other than their immediate goal to anywhere near the level even a terrible human programmer is. That why we have the agentic workflow in the first place. They absolutely require guardrails. Claude will absolutely change anything that’s not bolted to the floor. If you’ve used it on legacy software with users or if you reviewed the output you’d see this.
  - anthonyrstevens 7mo ago
    I think it's a perfectly fine point. The OP said (my interpretation) that LLMs are messy, non-deterministic, and can produce bad code. The same is true of many humans, even those whose "job" is to produce clean, predictable, good code. The OP would like the argument to be narrowly about LLMs, but the bigger point even is "who generates the final code, and why and how much do we trust them?"
    - sarchertech 7mo ago
      As of right now agents have almost no ability to reason about the impact of code changes on existing functionality. A human can produce a 100k LOC program with absolute no external guardrails at all. An agent Can't do that. To produce a 100k LOC program they require external feedback forcing them from spiraling off into building something completely different. This may change. Agents may get better.

petcat 7mo ago

The point is that expert humans (the GCC developers) writing code (C++) that generates code (ASM) does not appear to be as deterministic as you seem to think it is.

sarchertech 7mo ago
I’m very aware of that, but I’m also aware that it’s rare enough that the compiler doesn’t emit semantically equivalent code that most people can ignore it. That’s not the case with LLMs. I’m also not particularly concerned with non-determinism but with chaos. Determinism in LLMs is likely solvable, prompt instability is not.
jplusequalt 7mo ago
Classic HN-ism. To focus on the semantics of a statement while ignoring the greater point in order to argue why someone is wrong.
- petcat 7mo ago
  I argued the greater point? Software code-generation is not deterministic, whether it's done by expert humans or by LLMs.
  - sarchertech 7mo ago
    It has nothing to do with determinism. It's the difference between nearly perfectly but not quite perfectly translating between rigorously specified formal languages and translating an ambiguous natural language specification into a formal one. The first is a purely mechanical process, the second is not and requires thousands of decisions that can go either way.
    - raw_anon_1111 7mo ago
      And that’s no different than human developers
      - sarchertech 7mo ago
        The difference is that a human is that a human can reason about their code changes to a much higher degree than an AI can. If you don't think this is true and you think we're working with AGI, why would you bother architecting anything all or building in any guard rails. Why not just feed the AI the text of the contract your working from and let it rip.
        raw_anon_1111 7mo ago
        You give way too much credit to the average mid level ticket taker. And again, why do I care how the code does it as long as it meets the functional and none functional requirements?
        sarchertech 7mo ago
        Because in a real application with real users all of the functional and non-functional requirements aren't documented anywhere but in the code.
        raw_anon_1111 7mo ago
        If only a coding agent had access to your code…
        sarchertech 7mo ago
        You realize that coding agents aren’t AGI right? They aren’t capable of reasoning about a code changes impact on anything other than their immediate goal to anywhere near the level even a terrible human programmer is. That why we have the agentic workflow in the first place. They absolutely require guardrails. Claude will absolutely change anything that’s not bolted to the floor. If you’ve used it on legacy software with users or if you reviewed the output you’d see this.
- anthonyrstevens 7mo ago
  I think it's a perfectly fine point. The OP said (my interpretation) that LLMs are messy, non-deterministic, and can produce bad code. The same is true of many humans, even those whose "job" is to produce clean, predictable, good code. The OP would like the argument to be narrowly about LLMs, but the bigger point even is "who generates the final code, and why and how much do we trust them?"
  - sarchertech 7mo ago
    As of right now agents have almost no ability to reason about the impact of code changes on existing functionality. A human can produce a 100k LOC program with absolute no external guardrails at all. An agent Can't do that. To produce a 100k LOC program they require external feedback forcing them from spiraling off into building something completely different. This may change. Agents may get better.