3 ms·

It’s not just about non-determinism, but about how chaotic LLMs are. A one word difference in a spec can and frequently does produce unrecognizably different ou

It’s not just about non-determinism, but about how chaotic LLMs are. A one word difference in a spec can and frequently does produce unrecognizably different output. If you are using an LLM as a high level language, that means that every time you make a slight change to anything and “recompile” all of the thousands upon thousands of unspecified implementation details are free to change. You could try to ameliorate this by training LLMs to favor making fewer changes, but that would likely end up encoding every bad architecture decisions made along the way and essentially forcing a convergence on bad design. Fixing this I think requires judgment on a level far beyond what LLMs have currently demonstrated.

orbital-decay 9mo ago
>It’s not just about non-determinism I'm very specifically addressing prompt reproducibility mentioned above, because it's a notorious red herring in these discussions. What you want is correctness, not determinism/reproducibility which is relatively trivial. (although thinking of it more, maybe not that trivial... if you want usable repro in the long run, you'll have to store the model snapshot, the inference code, and make it deterministic too) >A one word difference in a spec can and frequently does produce unrecognizably different output. This is well out of scope for the reproducibility and doesn't affect it in the slightest. And for practical software development this is also a red herring, the real issue is correctness and spec gaming. As long as the output is correct and doesn't circumvent the intention of the spec, prompt instability is unimportant, it's just the ambiguous nature of the domain LLMs and humans operate in.
- sarchertech 9mo ago
  Well if you want to use it as a high level language where you check in the spec and regenerate the code then prompt instability/chaotic output makes that infeasible. You can’t just tell users “sorry there are a million tiny differences all over the app every time we change the slightest thing, that’s just the ambiguous nature of reality”.
  - orbital-decay 9mo ago
    >where you check in the spec and regenerate the code then prompt instability/chaotic output makes that infeasible What, why would you want to write the code anew? Identify the changes in the spec and bring the existing code in line with them.
    - sarchertech 9mo ago
      That’s the whole thesis of the article. Using an LLM as a high level language. >The codebase should be reconstructable from the documentation
      - orbital-decay 9mo ago
        And this kind of reproducibility is insensitive to prompt instability. If your inference is deterministic (trivial if within the lifetime of a snapshot), you give it the same docs and get the same result.
        sarchertech 9mo ago
        Obviously this article is advocating for spec as source code and not checking in code to keep it around between “compilations”. Why would you care about reproducibility at all if you were checking in the code as well. If you check in the code, you’d never need to rerun the same prompt.

orbital-decay 9mo ago

>It’s not just about non-determinism I'm very specifically addressing prompt reproducibility mentioned above, because it's a notorious red herring in these discussions. What you want is correctness, not determinism/reproducibility which is relatively trivial. (although thinking of it more, maybe not that trivial... if you want usable repro in the long run, you'll have to store the model snapshot, the inference code, and make it deterministic too) >A one word difference in a spec can and frequently does produce unrecognizably different output. This is well out of scope for the reproducibility and doesn't affect it in the slightest. And for practical software development this is also a red herring, the real issue is correctness and spec gaming. As long as the output is correct and doesn't circumvent the intention of the spec, prompt instability is unimportant, it's just the ambiguous nature of the domain LLMs and humans operate in.

sarchertech 9mo ago
Well if you want to use it as a high level language where you check in the spec and regenerate the code then prompt instability/chaotic output makes that infeasible. You can’t just tell users “sorry there are a million tiny differences all over the app every time we change the slightest thing, that’s just the ambiguous nature of reality”.
- orbital-decay 9mo ago
  >where you check in the spec and regenerate the code then prompt instability/chaotic output makes that infeasible What, why would you want to write the code anew? Identify the changes in the spec and bring the existing code in line with them.
  - sarchertech 9mo ago
    That’s the whole thesis of the article. Using an LLM as a high level language. >The codebase should be reconstructable from the documentation
    - orbital-decay 9mo ago
      And this kind of reproducibility is insensitive to prompt instability. If your inference is deterministic (trivial if within the lifetime of a snapshot), you give it the same docs and get the same result.
      - sarchertech 9mo ago
        Obviously this article is advocating for spec as source code and not checking in code to keep it around between “compilations”. Why would you care about reproducibility at all if you were checking in the code as well. If you check in the code, you’d never need to rerun the same prompt.