4 ms·

It may not be as stupid as it sounds. Randomising LLM outputs (temperature) results is outputs that will always have some degree of hallucination. That’s just

It may not be as stupid as it sounds. Randomising LLM outputs (temperature) results is outputs that will always have some degree of hallucination. That’s just math. You can’t mix a random factor in and magically expect it to not exist. There will always be p(generates random crap) > 0. However, in any probabilistic system, you can run a function k times and you’ll get an output distribution that is meaningful if k is high enough. 3 is not high enough. At 3, this is stupid; all you’re observing is random variance. …but, in general, running the same prompt multiple times and taking some kind of general solution from the distribution isn’t totally meaningless, I guess. The thing with LLMs is they scale in a way that actually allows this to be possible, in a way that scaling with humans can’t. … like the monkeys and Shakespeare, there probably a limit to the value it can offer; but it’s not totally meaningless to try it.

horsawlarway 2y ago
I think this is an interesting idea, but I also somewhat suspect you've replaced a tedious problem with a harder, more tedious problem. Take your idea further. Now I've got 100 agents, and 100 PRs, and some small percentage of them are decent. The task went from "implement a feature" to "review 100 PRs and select the best one". Even assuming you can ditch 50 percent right off the bat as trash... Reviewing 50 potentially buggy implementations of a feature and selecting the best genuinely sounds worse than just writing the solution. Worse... If you haven't solved the problem before anyways, you're woefully unqualified as a reviewer.
- skeledrew 2y ago
  There should be test cases ran, coverage ensured. This is trivially automated. LLMs should also review the PRs, at least initially, using the test results as part of the input.
  - horsawlarway 2y ago
    So now either the agent is writing the tests, in which case you're right back to the same issue (which tests are actually worth running?) or your job is now just writing tests (bleh...). And for the llm review of the pr... Why do you assume it'll be worth any more then the original implementation? Or are we just recursing down a level again (if 100 llms review each of the 100 PRs... To infinity and beyond!) This by definition is not trivially automated.
    - skeledrew 2y ago
      The LLMs can help with the writing of the tests, but you should verify that they're testing critical aspects and known edge cases are covered. A single review-promoted LLM can then utilize those across the PRs and provide a summary for acceptance the the best. Or discard all and do manually; that initial process should only have taken a few minutes, so minimal wastage in the grand scheme of things, given over time there are a decent amount of acceptances, compared to the alternative 100% manual effort and associated time sunk.
  - lolinder 2y ago
    Who tests the tests? How do you know that the LLM-generated tests are actually asserting anything meaningful and cover the relevant edge cases? The tests are part of the code that needs to be reviewed in the PR by a human. They don't solve the problem, they just add more lines to the reviewer's job.
- CognitiveLens 2y ago
  The linked article from Steve Yegge (https://sourcegraph.com/blog/revenge-of-the-junior-developer https://sourcegraph.com/blog/revenge-of-the-junior-developer) provides a 'solution', which he thinks is also imminent - supervisor AI agents, where you might have 100+ coding agents creating PRs, but then a layer of supervisors that are specialized on evaluating quality, and the only PRs that a human being would see would be the 'best', as determined by the supervisor agent layer. From my experience with AI agents, this feels intuitively possible - current agents seem to be ok (thought not yet 'great') at critiquing solutions, and such supervisor agents could help keep the broader system in alignment.
  - kgeist 2y ago
    >but then a layer of supervisors that are specialized on evaluating quality Why would supervisor agents be any better than the original LLMs? Aren't they still prone to hallucinations and subject to the same limitations imposed by training data and model architecture? It feels like it just adds another layer of complexity and says, "TODO: make this new supervisor layer magically solve the issue." But how, exactly? If we already know the secret sauce, why not bake it into the first layer from the start?
    - immibis 2y ago
      Just add a CxO layer that monitors the supervisors! And the board of directors watches the CEO and the shareholders monitor the board of directors. It's agents all the way up!
      - deleted 2y ago
        [deleted]
    - pama 2y ago
      Similar to how human brains behave, it is easier to train a model to select a better solution between many choices than to check an individual solution for correctness [1], which is in turn an easier task to learn than writing a correct single solution in the first place. [1] the diffs in logic can suggest good ideas that may have been missed in subsets of solutions.
- namaria 2y ago
  Theory of constraints is clear: speeding up something that isn't a bottleneck worsens system performance. The idea that too little code is the problem is the problem. Code is liability. Making more of it faster (and probabilistic) is a fantastically bad idea.
- cyanydeez 2y ago
  What if we replace PR with QA test rig? Then hire a bunch of QA monkies to find bugs in them and select the "bug free" one.
regularfry 2y ago
Three is high enough, in my eyes. Two might be. Remember that we don't care about any but the best solution. With one sample you've only got one 50/50 shot to get above the median. With three, the odds of the best of the three being above the median is 87.5%. Of course picking the median as the random crap boundary is entirely arbitrary, but it'll do until there's a justification for a better number.
fn-mote 2y ago
> taking some kind of general solution from the distribution My instinct is that this should be the temperature 0K response (no randomness).
AdieuToLogic 2y ago
> It may not be as stupid as it sounds. It is. > However, in any probabilistic system, you can run a function k times and you’ll get an output distribution that is meaningful if k is high enough. This is the underlying flaw in this approach. Attempting to use probabilistic algorithms to produce a singular verifiably correct result requires an external agent to select what is correct in the output of "k times" invocations. This is a person capable of making said determination. > The thing with LLMs is they scale in a way that actually allows this to be possible, in a way that scaling with humans can’t. For the "k times" generation of text part, sure. Not for the determination of which one within k, if any, are acceptable for the problem at hand. EDIT: clarified "produce a verifiably correct result" to be "produce a singular verifiably correct result"
AdieuToLogic 2y ago
> like the monkeys and Shakespeare, there probably a limit to the value it can offer; but it’s not totally meaningless to try it. Whenever someone uses this analogy, a question never addressed is: Assuming sufficient monkeys, typewriters, and time, how would anyone know if a Shakespearean work was produced unless one reviewed *all* output?

horsawlarway 2y ago

I think this is an interesting idea, but I also somewhat suspect you've replaced a tedious problem with a harder, more tedious problem. Take your idea further. Now I've got 100 agents, and 100 PRs, and some small percentage of them are decent. The task went from "implement a feature" to "review 100 PRs and select the best one". Even assuming you can ditch 50 percent right off the bat as trash... Reviewing 50 potentially buggy implementations of a feature and selecting the best genuinely sounds worse than just writing the solution. Worse... If you haven't solved the problem before anyways, you're woefully unqualified as a reviewer.

skeledrew 2y ago
There should be test cases ran, coverage ensured. This is trivially automated. LLMs should also review the PRs, at least initially, using the test results as part of the input.
- horsawlarway 2y ago
  So now either the agent is writing the tests, in which case you're right back to the same issue (which tests are actually worth running?) or your job is now just writing tests (bleh...). And for the llm review of the pr... Why do you assume it'll be worth any more then the original implementation? Or are we just recursing down a level again (if 100 llms review each of the 100 PRs... To infinity and beyond!) This by definition is not trivially automated.
  - skeledrew 2y ago
    The LLMs can help with the writing of the tests, but you should verify that they're testing critical aspects and known edge cases are covered. A single review-promoted LLM can then utilize those across the PRs and provide a summary for acceptance the the best. Or discard all and do manually; that initial process should only have taken a few minutes, so minimal wastage in the grand scheme of things, given over time there are a decent amount of acceptances, compared to the alternative 100% manual effort and associated time sunk.
- lolinder 2y ago
  Who tests the tests? How do you know that the LLM-generated tests are actually asserting anything meaningful and cover the relevant edge cases? The tests are part of the code that needs to be reviewed in the PR by a human. They don't solve the problem, they just add more lines to the reviewer's job.
CognitiveLens 2y ago
The linked article from Steve Yegge (https://sourcegraph.com/blog/revenge-of-the-junior-developer https://sourcegraph.com/blog/revenge-of-the-junior-developer) provides a 'solution', which he thinks is also imminent - supervisor AI agents, where you might have 100+ coding agents creating PRs, but then a layer of supervisors that are specialized on evaluating quality, and the only PRs that a human being would see would be the 'best', as determined by the supervisor agent layer. From my experience with AI agents, this feels intuitively possible - current agents seem to be ok (thought not yet 'great') at critiquing solutions, and such supervisor agents could help keep the broader system in alignment.
- kgeist 2y ago
  >but then a layer of supervisors that are specialized on evaluating quality Why would supervisor agents be any better than the original LLMs? Aren't they still prone to hallucinations and subject to the same limitations imposed by training data and model architecture? It feels like it just adds another layer of complexity and says, "TODO: make this new supervisor layer magically solve the issue." But how, exactly? If we already know the secret sauce, why not bake it into the first layer from the start?
  - immibis 2y ago
    Just add a CxO layer that monitors the supervisors! And the board of directors watches the CEO and the shareholders monitor the board of directors. It's agents all the way up!
    - deleted 2y ago
      [deleted]
  - pama 2y ago
    Similar to how human brains behave, it is easier to train a model to select a better solution between many choices than to check an individual solution for correctness [1], which is in turn an easier task to learn than writing a correct single solution in the first place. [1] the diffs in logic can suggest good ideas that may have been missed in subsets of solutions.
namaria 2y ago
Theory of constraints is clear: speeding up something that isn't a bottleneck worsens system performance. The idea that too little code is the problem is the problem. Code is liability. Making more of it faster (and probabilistic) is a fantastically bad idea.
cyanydeez 2y ago
What if we replace PR with QA test rig? Then hire a bunch of QA monkies to find bugs in them and select the "bug free" one.

regularfry 2y ago

Three is high enough, in my eyes. Two might be. Remember that we don't care about any but the best solution. With one sample you've only got one 50/50 shot to get above the median. With three, the odds of the best of the three being above the median is 87.5%. Of course picking the median as the random crap boundary is entirely arbitrary, but it'll do until there's a justification for a better number.

fn-mote 2y ago

> taking some kind of general solution from the distribution My instinct is that this should be the temperature 0K response (no randomness).

AdieuToLogic 2y ago

> It may not be as stupid as it sounds. It is. > However, in any probabilistic system, you can run a function k times and you’ll get an output distribution that is meaningful if k is high enough. This is the underlying flaw in this approach. Attempting to use probabilistic algorithms to produce a singular verifiably correct result requires an external agent to select what is correct in the output of "k times" invocations. This is a person capable of making said determination. > The thing with LLMs is they scale in a way that actually allows this to be possible, in a way that scaling with humans can’t. For the "k times" generation of text part, sure. Not for the determination of which one within k, if any, are acceptable for the problem at hand. EDIT: clarified "produce a verifiably correct result" to be "produce a singular verifiably correct result"

AdieuToLogic 2y ago

> like the monkeys and Shakespeare, there probably a limit to the value it can offer; but it’s not totally meaningless to try it. Whenever someone uses this analogy, a question never addressed is: Assuming sufficient monkeys, typewriters, and time, how would anyone know if a Shakespearean work was produced unless one reviewed *all* output?