3 ms·

It’s much worse than that. LLMs today don’t produce the same output for the same prompt.

• seanmcdirmid 8mo ago
  If you think that’s bad you should see how non-deterministic the alternative is (human programmers). Thankfully LLMs can iterate on the code they write, anyone who is using them to generate the same code from scratch each time a change is made needs some extra education. They are not code generators, they are junior programmers.
  • sarchertech 8mo ago
    And junior programmers need supervision. You’re agreeing with my point though which is that LLMs aren’t higher level compilers. LLMs aren’t abstraction, they are delegation.
    • seanmcdirmid 8mo ago
      Yes, lots of supervision. So humans are still useful in the loop. LLMS are definitely not algorithmic transformers (ie compilers). Well, they are more like “coders” when the term meant the secretaries who translated the mathematician and scientists (the “programmers”) notation into machine code that could run on computer, by hand.
      • sarchertech 8mo ago
        The analogy I’ve found most helpful is that it’s like having access to a team of 100 offshore junior developers that rotate out completely every few hours. There are certainly useful things you can do with that. But give that team to a PM who can’t read the output and anything beyond a todo app would collapse under the chaos.
        • seanmcdirmid 8mo ago
          A good LLM is a junior developer who is somehow really proficient at doing unreliable research. They won’t say no, I don’t know how, like lots of junior developers, but maybe sometimes they should? They also follow instructions much better than junior developers does, and they don’t mind being micro managed.
          • sarchertech 8mo ago
            All those things are true, but there are a few things to add to it. They are much much better at writing plausible looking code or providing plausible sounding answers that look correct but aren’t than any junior developer. They are much better at writing subtle bugs that won’t show up for days and will take hours to debug than any junior developer is. You can’t mentor them into being a senior developer.
            • seanmcdirmid 8mo ago
              Ya, but anyone who is using an LLM to write just code and isn’t also using them to write tests is nuts: they are just going to get garbage out. An LLM can totally write correct code, just not without some kind of feedback loop going on, even if that feedback loop is some form of oracle testing (where the LLM is also writing the tests, and the tests can be wrong themselves). They are also great enumerating edge cases, which feeds nicely into test coverage. 80% of your prompting work is in getting the LLM to write tests and get good coverage, maybe 20% is getting it to write code. Junior developers (human) can kind of get by without testing, at least in the short term. The LLM can never get by without testing unless it’s some simple one off logic.
              • sarchertech 8mo ago
                It’s definitely true that LLMs require tests, but tests aren’t an antidote to what I’m taking about because LLMs are also good at writing plausible looking tests that are actually terrible. They’re also very bad at understanding what edge cases are useful to test and what edge cases can be ignored. I had a situation just the other day where the LLM produced tests for scores of edge cases, but a managed to leave out a specific sequence that caused a very difficult to diagnose bug. I’m assuming because it decided that sequence wasn’t possible. But it did include dozens of redundant cases that no sane human ever would have.