4 ms·

If we’re talking about “can we ignore the code the way we mostly ignore assembly and treat prompts as the new high level language”, determinism isn’t the hard p

If we’re talking about “can we ignore the code the way we mostly ignore assembly and treat prompts as the new high level language”, determinism isn’t the hard problem. The real issue is prompt instability (chaos). A one word change to a prompt/spec will produce a drastically different program. Until that is solved there’s no world where we just check in the prompt and almost no one ever has to worry about the code.

gurjeet 8mo ago
It’s much worse than that. LLMs today don’t produce the same output for the same prompt.
- seanmcdirmid 8mo ago
  If you think that’s bad you should see how non-deterministic the alternative is (human programmers). Thankfully LLMs can iterate on the code they write, anyone who is using them to generate the same code from scratch each time a change is made needs some extra education. They are not code generators, they are junior programmers.
  - sarchertech 8mo ago
    And junior programmers need supervision. You’re agreeing with my point though which is that LLMs aren’t higher level compilers. LLMs aren’t abstraction, they are delegation.
    - seanmcdirmid 8mo ago
      Yes, lots of supervision. So humans are still useful in the loop. LLMS are definitely not algorithmic transformers (ie compilers). Well, they are more like “coders” when the term meant the secretaries who translated the mathematician and scientists (the “programmers”) notation into machine code that could run on computer, by hand.
      - sarchertech 8mo ago
        The analogy I’ve found most helpful is that it’s like having access to a team of 100 offshore junior developers that rotate out completely every few hours. There are certainly useful things you can do with that. But give that team to a PM who can’t read the output and anything beyond a todo app would collapse under the chaos.
        seanmcdirmid 8mo ago
        A good LLM is a junior developer who is somehow really proficient at doing unreliable research. They won’t say no, I don’t know how, like lots of junior developers, but maybe sometimes they should? They also follow instructions much better than junior developers does, and they don’t mind being micro managed.
        sarchertech 8mo ago
        All those things are true, but there are a few things to add to it. They are much much better at writing plausible looking code or providing plausible sounding answers that look correct but aren’t than any junior developer. They are much better at writing subtle bugs that won’t show up for days and will take hours to debug than any junior developer is. You can’t mentor them into being a senior developer.
        seanmcdirmid 8mo ago
        Ya, but anyone who is using an LLM to write just code and isn’t also using them to write tests is nuts: they are just going to get garbage out. An LLM can totally write correct code, just not without some kind of feedback loop going on, even if that feedback loop is some form of oracle testing (where the LLM is also writing the tests, and the tests can be wrong themselves). They are also great enumerating edge cases, which feeds nicely into test coverage. 80% of your prompting work is in getting the LLM to write tests and get good coverage, maybe 20% is getting it to write code. Junior developers (human) can kind of get by without testing, at least in the short term. The LLM can never get by without testing unless it’s some simple one off logic.
virgil_disgr4ce 8mo ago
> A one word change to a prompt/spec will produce a drastically different program So in other words, determinism (or lack thereof) is the hard problem!
- ddtaylor 8mo ago
  Obviously they are cryptographic hashing functions as they exhibit an avalanche effect!
- ChadNauseam 8mo ago
  A deterministic program must have the same output for the same input, but determinism does not restrict the outputs for different inputs
  - 1718627440 8mo ago
    I think most people perceive program semantic not only to be defined for the whole program as a whole, but also towards individual subexpressions. If the "LLM compiler" changes the output completely when a single word is added, then it is not deterministic for the subexpressions, that don't contain that addition. Granted that is not the rigorous definition of determinism. Also some existing compilers are non-deterministic with such a definition, namely when "undefined behaviour" would be triggered. That's precisely the problem with UB.

gurjeet 8mo ago

It’s much worse than that. LLMs today don’t produce the same output for the same prompt.

seanmcdirmid 8mo ago
If you think that’s bad you should see how non-deterministic the alternative is (human programmers). Thankfully LLMs can iterate on the code they write, anyone who is using them to generate the same code from scratch each time a change is made needs some extra education. They are not code generators, they are junior programmers.
- sarchertech 8mo ago
  And junior programmers need supervision. You’re agreeing with my point though which is that LLMs aren’t higher level compilers. LLMs aren’t abstraction, they are delegation.
  - seanmcdirmid 8mo ago
    Yes, lots of supervision. So humans are still useful in the loop. LLMS are definitely not algorithmic transformers (ie compilers). Well, they are more like “coders” when the term meant the secretaries who translated the mathematician and scientists (the “programmers”) notation into machine code that could run on computer, by hand.
    - sarchertech 8mo ago
      The analogy I’ve found most helpful is that it’s like having access to a team of 100 offshore junior developers that rotate out completely every few hours. There are certainly useful things you can do with that. But give that team to a PM who can’t read the output and anything beyond a todo app would collapse under the chaos.
      - seanmcdirmid 8mo ago
        A good LLM is a junior developer who is somehow really proficient at doing unreliable research. They won’t say no, I don’t know how, like lots of junior developers, but maybe sometimes they should? They also follow instructions much better than junior developers does, and they don’t mind being micro managed.
        sarchertech 8mo ago
        All those things are true, but there are a few things to add to it. They are much much better at writing plausible looking code or providing plausible sounding answers that look correct but aren’t than any junior developer. They are much better at writing subtle bugs that won’t show up for days and will take hours to debug than any junior developer is. You can’t mentor them into being a senior developer.
        seanmcdirmid 8mo ago
        Ya, but anyone who is using an LLM to write just code and isn’t also using them to write tests is nuts: they are just going to get garbage out. An LLM can totally write correct code, just not without some kind of feedback loop going on, even if that feedback loop is some form of oracle testing (where the LLM is also writing the tests, and the tests can be wrong themselves). They are also great enumerating edge cases, which feeds nicely into test coverage. 80% of your prompting work is in getting the LLM to write tests and get good coverage, maybe 20% is getting it to write code. Junior developers (human) can kind of get by without testing, at least in the short term. The LLM can never get by without testing unless it’s some simple one off logic.

virgil_disgr4ce 8mo ago

> A one word change to a prompt/spec will produce a drastically different program So in other words, determinism (or lack thereof) is the hard problem!

ddtaylor 8mo ago
Obviously they are cryptographic hashing functions as they exhibit an avalanche effect!
ChadNauseam 8mo ago
A deterministic program must have the same output for the same input, but determinism does not restrict the outputs for different inputs
- 1718627440 8mo ago
  I think most people perceive program semantic not only to be defined for the whole program as a whole, but also towards individual subexpressions. If the "LLM compiler" changes the output completely when a single word is added, then it is not deterministic for the subexpressions, that don't contain that addition. Granted that is not the rigorous definition of determinism. Also some existing compilers are non-deterministic with such a definition, namely when "undefined behaviour" would be triggered. That's precisely the problem with UB.