13 ms·

Chain of Recursive Thoughts: Make AI think harder by making it argue with itself

539 points by miles 1y ago

lenerdenator 1y ago
I, too, like to give Terminator lite anxiety.
hnuser123456 1y ago
I'm having a lot of fun experimenting with stuff like this. I'm trying to put together an unrealengine blueprints style graph editor to allow people to design workflows like this where you start with the user prompt input, which goes to one agent, which makes an initial attempt, and then that conversation history gets passed to another "agent" with a different system prompt telling it to be a harsh critic, but to also give a pass/fail signal, and loop back until the critic judges pass, then send that back to the user as output. Ideally as a little website that can call your own LLM endpoints and save/load/share workflow graphs. Mistral small 3.1 and gemma 3 feel like the first semi-competent models that can be run locally, but that competence is just a seed, and they still need to be guided with a framework that keeps them on track. Try giving it python execution in a loop and tell it to explore the world. It'll start trying to download and read news and stuff.
- deleted 1y ago
  [deleted]
- globalise83 1y ago
  Have you tried n8n? It allows you to build flows like that - you can run the community version in a Docker container within a few minutes and share the configurations for the flows you have built very easily.
  - mecsred 1y ago
    _#_ has to be one of the worst word shortening schemes I've ever seen get widespread. It only works with a very small number of long-lived technologies, in which case they basically just get a nickname, "k8s" "i18n". It does not at all work for larger contexts. You're basically making someone solve a crossword (2 across, 10 letters with two filled in) just to parse your sentence.
    - jjj123 1y ago
      I just googled it and it looks like “n8n” is the name of the service. The op wasn’t abbreviating anything so I don’t think it’s the same phenomenon as what you’re describing.
      - lgas 1y ago
        Well, the service is doing the same thing though. The part I don't understand is that I assume n8n is short for "Nation" but literally every single person I've seen talk about it on YouTube (which is quite a lot) say "En Eight En" every time.
        nemomarx 1y ago
        nation is too short for 8 - maybe navigation?
        pkaye 1y ago
        Looks like n8n is short for nodemation
        firesteelrain 1y ago
        Why do we do this to ourselves?
        Y_Y 1y ago
        Techno-flagellation is the only way to atone
        lgas 1y ago
        So the 8 stands for "odematio"? That sounds about right.
      - oppodeldoc 1y ago
        https://github.com/n8n-io/n8n?tab=readme-ov-file#what-does-n8n-mean https://github.com/n8n-io/n8n?tab=readme-ov-file#what-does-n...
  - hnuser123456 1y ago
    I had not, but that looks awesome. Microsoft put out something called "agent flows" that also fits this category.[1] I'm working on more of an "at home" version - no "talk to sales" button. https://www.microsoft.com/en-us/microsoft-copilot/blog/copilot-studio/introducing-agent-flows-transforming-automation-with-ai-first-workflows/ https://www.microsoft.com/en-us/microsoft-copilot/blog/copil...
- andai 1y ago
  I am thinking the same thing! Multiple "personalities", in parallel, or in series. For example, I have approximated, in GPT, some of Gemini's ability to call out nonsense, sloppy thinking, by telling GPT to be mean! (The politeness seems to filter out much that is of great value!) However, the result is not pleasant to read. Gemini solved this in their training, by doing it in two phases... and making the first phase private! ("Thinking.") So I thought, what I need is a two-phase approach, where that "mean" output gets humanized a little bit. (It gets harsh to work in that way for more than short intervals.) As a side note, I think there would be great value in a UI that allows a "group chat" of different LLM personalities. I don't know if such a thing exists, but I haven't seen it yet, although the message object format seems to have been designed with it in mind (e.g. every message has a name, to allow for multiple users and multiple AIs). Even better if it supports multiple providers, since they have different strengths. (It's like getting a second opinion.)
  - NitpickLawyer 1y ago
    > As a side note, I think there would be great value in a UI that allows a "group chat" of different LLM personalities. This is the basic idea behind autogen. They also have a web UI now in autogen studio, it's gotten a bit better. You can create "teams" of agents (with different prompts, themes, tools, etc.) and have them discuss / cooperate. I think they even added memory recently. Have a look at it, might be what you need.
  - jbm 1y ago
    I disagree. If anything, telling GPT to be blunt seems to downgrade its IQ; it hallucinates more and makes statements without considering priors or context. I jokingly call it Reddit mode.
    - dingnuts 1y ago
      why would that be a joke? there's a ton of Reddit comments in the training data, and the output is of similar quality. LLMs are literally outputting average Reddit comments.
- deleted 1y ago
  [deleted]
- irthomasthomas 1y ago
  I think you can do most of this already with llm-consortium (maybe needs the llm-openrouter plugin with my pr merging) A consortium sends the same prompt to multiple models in parallel and the responses are all sent to one arbiter model which judges the model responses. The arbiter decides if more iterations are required. It can also be forced to iterate more until confidence-threshold or min-iterations. Now, using the pr i made to llm-openrouter, you can save an alias to a model that includes lots of model options. For examples, you can do llm openrouter save -m qwen3 -o online -o temperature 0, system "research prompt" --name qwen-researcher And now, you can build a consortium where one member is an online research specialist. You could make another uses JSON mode for entity extraction, and a third which writes a blind draft. The arbiter would then make use of all that and synthesize a good answer.
  - kridsdale1 1y ago
    Any links or names of example implementations of this?
    - irthomasthomas 1y ago
      https://github.com/irthomasthomas/llm-consortium https://github.com/irthomasthomas/llm-consortium also, you aren't limited to cli. When you save a consortium it creates a model. You can then interact with a consortium as if it where a normal model (albeit slower and higher quality). You can then serve your custom models on an openai endpoint and use them with any chat client that supports custom openai endpoints. The default behaviour is to output just the final synthesis, and this should conform to your user prompt. I recently added the ability to continue conversations with a consortium. In this case it only includes your user prompt and final synthesis in the conversation, so it mimics a normal chat, unlike running multiple iterations in the consortium, where full iteration history and arbiter responses are included. UV tool install llm llm install llm-consortium llm install llm-model-gateway llm consortium save qwen-gem-sonnet -m qwen3-32b -n 2 -m sonnet-3.7 -m gemini-2.5-pro --arbiter gemini-2.5-flash --confidence-threshold 95 --max-iterations 3 llm serve qwen-gem-sonnet In this example I used -n 2 on the qwen model since it's so cheap we can include multiple instances of it in a consortium Gemini flash works well as the arbiter for most prompts. However if your prompt has complex formatting requirements, then embedding that within an already complex consortium prompt often confuses it. In that case use gemini-2.5-pro for the arbiter. .
Xcelerate 1y ago
I think this is how we get ML models to come up with novel ideas. Diagonalize against all the ideas they’ve already tried and dismissed via self-argument but keep certain consistency constraints. (Obviously much easier said than done.)
- andai 1y ago
  What you just said is what I tried and failed to say ten minutes ago! https://news.ycombinator.com/item?id=43835798 https://news.ycombinator.com/item?id=43835798
  - Nevermark 1y ago
    It’s working! Oh, wait … These models have limitations obviously, but many critiques apply equally or more to people. If people were tasked with one shot, 10 second answers, to be written out in near errorless grammar, the LLM’s viewing our responses to prompts would be spending a lot of time discussing our limitations and how to game us into better responses. Humor, not at all humor.
- jwally 1y ago
  Scaled up and spread out - this probably gets you pretty close to consciousness(?) Conway's game of life, but instead of colored squares with rules, they're LLM's with some kind of weighting - all chattering back and forth with one another - bubbling up somehow to cause speach/action
  - lubujackson 1y ago
    Decades ago I read The Society of Mind by Marvin Minsky. He pushed this sort of idea, that consciousness is composed of individual, competing processes. Worth a revisit!
cube2222 1y ago
This is really cool! One strategy I often use (which is much simpler and more limited than this), is to finish my message with: “Please do a round of thinking in <thinking></thinking> tags, then a round of self-critique in <critique></critique> tags, and then a final round of <thinking>, before responding.” It works very well. Similarly just asking it to “find the 5 biggest issues with its proposal” works pretty good (the 5 forcing it to find something, even if it’s mostly irrelevant).
- danielbln 1y ago
  I always do "now again but put on your critical hat"
  - CSSer 1y ago
    Makes me wonder how it would do if you tell it "put on your robe and wizard hat"
    - tomrod 1y ago
      ChatGPT calls you a superstar and it drops into bruhspeak. Emojis proliferate.
    - sumtechguy 1y ago
      it proceeds to spit out the entirety of bash.org
- bentt 1y ago
  Oh I really like that. It makes me want to have it score its ideas with metrics and then keep iterating until it meets some score.
- zoogeny 1y ago
  This is one of the reasons I like the massive context window in Gemini. You can do this as part of the message chain. I don't try to one shot it, just use the same idea across 3 messages. 1. Figure out a plan (it responds with the plan) 2. Point out flaws in the plan (it responds with the flaws) 3. Update the plan to address the flaws (it responds with an up to date plan) The other things I tend to ask are "what might we be missing?", "what are the [performance|security|legal|cost] considerations?". I can often iterate on the "anything else?" kind of nudging prompts, especially guiding it on topics to consider, for a few messages. After each: update the plan to take those into consideration.
casenmgreen 1y ago
[flagged]
- rapfaria 1y ago
  Or "thinking" just got a new meaning and it's to convey information in the field - perhaps the Oxford dictionary will add it soon?
- consumer451 1y ago
  I am not sure that you can make that absolute statement. Reasoning is subdivided into types, and one of those types is inductive reasoning. > Inductive reasoning refers to a variety of methods of reasoning in which the conclusion of an argument is supported not with deductive certainty, but with some degree of probability. Unlike deductive reasoning (such as mathematical induction), where the conclusion is certain, given the premises are correct, inductive reasoning produces conclusions that are at best probable, given the evidence provided. Doesn't predicting the next token qualify as doing just that? https://en.wikipedia.org/wiki/Inductive_reasoning https://en.wikipedia.org/wiki/Inductive_reasoning
  - dttze 1y ago
    Markov chains have done that for ages. They aren't AI. This is just that scaled up. Just because it can infer a token doesn't mean it can infer a conclusion to an argument.
    - casenmgreen 1y ago
      To add a bit to this : expert systems have two properties. They give an answer, and they explain their reasoning. LLM cannot explain their reasoning, and that is because there is no reasoning.
      - consumer451 1y ago
        To push back on this, a somewhat recent Linus Torvalds ~quote: "I don't think that 'just predicting the next word' is the insult that people think it is, it's mostly what we all do." If we break our lives down into the different types of reasoning, and what we mostly do day-to-day, this rings very true to me. I currently believe that our brains generally operate as very efficient inference machines. Sometimes we slow down to think things through, but for example, when in the ideal "flow state" it's some kind of distilled efficient inference. Isn't it? This is very hard for me to deny at this time. ___ edit: 4o appears to agree with both of you, more than it does with me. https://chatgpt.com/share/68119b41-1144-8012-b50d-f8f15997eb65 https://chatgpt.com/share/68119b41-1144-8012-b50d-f8f15997eb... However, Sonnet 3.7 appears to side with me. https://claude.ai/share/91139bca-3201-4ffc-a940-bdd27329e71f https://claude.ai/share/91139bca-3201-4ffc-a940-bdd27329e71f (Both of these are the default models available for free accounts, on each website, at the time of writing) IMO, hey, at least we do live in interesting times.
antisthenes 1y ago
Cool. Now I can justify talking to myself.
Garlef 1y ago
Similarly, letting the LLM generate a socratic dialogue can work pretty well to get deeper into a topic.
Der_Einzige 1y ago
Debate as a reasoning tactic is massively undervalued. There's tons of papers on this at places like NeurIPS, ICML, ICLR, etc. Hell, even a whole quanta article. https://www.quantamagazine.org/debate-may-help-ai-models-converge-on-truth-20241108/ https://www.quantamagazine.org/debate-may-help-ai-models-con... I got to meet and talk to the authors of this paper at NeurIPS. They're class acts!
electroly 1y ago
This seems to be different than I expected from the title. I thought it would be explicitly adversarial. 1. You are the assistant. Please answer the question directly. 2. You are the cross-examiner. The assistant is wrong. Explain why. 3. You are the assistant. The cross-examiner is wrong. Defend your claim. 4. You are a judge. Did either party make their case, or is another round of argumentation required? I haven't tried this. No idea if it works. But I find it's helpful to ask ChatGPT, in separate prompts, "XYZ is true, explain why" and "XYZ is false, explain why" and see which one seems more convincing.
- nonethewiser 1y ago
  Chatgpt shares context between chats. I wonder how that impacts it? It seems like a good approach though. What you dont want to do is ever suggest that its wrong yourself. Usually it will just assume it is wrong. Actually what I find impressive is when I do this and it actually pushes back to defend itself.
  - the_af 1y ago
    Does it share context even if no "memory updated" message appears indicating it has stored a fact about you? I asked ChatGPT and it says no, but then again it's not reliable at introspection or at revealing data about how it works.
    - visarga 1y ago
      I think they are different systems, one is a collection of saved snippets and the other more like RAG over chat history.
      - the_af 1y ago
        ChatGPT assures me it doesn't use RAG (fed from my other chat windows), but will use memory-saved preferences (in the store that can be accessed and reviewed in Settings->Personalization->Memory). Then again, I don't think ChatGPT is reliable when reporting on its own inner workings. --- Oh, no, here it says it also references chat history: https://help.openai.com/en/articles/8590148-memory-faq https://help.openai.com/en/articles/8590148-memory-faq
pkdpic 1y ago
So glad to see a write up on this finally. I'm no machine learning phd but I always wondered why this wasn't more of a thing. Like an extension of a GAN conceptually, sort of, not really at all Im sure. Also I think I kind of assumed OpenAI might be doing this behind the curtain?
K0balt 1y ago
I’ll second this. I often use a “research assistant “ and skeptical“department head” personas working together/against each other as a research team. It works well and is occasionally hilarious, replete with the occasional HR complaint when things go off the rails. ( I typically use local uncensored models)
firgrove 1y ago
this is amazing - I love seeing novel approaches to optimizing
joshstrange 1y ago
I've thought about trying this cross-model as well. Have Claude generate something, have OpenAI check it, have Gemini check that check. Firing multiple of these in parallel. There was a post here a week or so ago doing the "model checking model"-type thing with GH PRs IIRC that was interesting. I haven't had a chance to play with this idea yet.
k2xl 1y ago
I've done something similar for learning about a controversial topic. I ask it to act as if it is called Bob is a well informed supporter of one side (like Ukraine) and then act as if it is something named Alice who is a well informed supporter of another side (Russia) and they have to debate each other over a few prompts with a moderator named 'Sue' Then after a few rounds of the debate where Sue asks a bunch of questions, I ask it to go to the judges - Mark, Phil, Sarah (and I add a few personalities to each of them... Sometimes I pretend they are famous moral philosophers) and then I have them each come up with a rubric and decide who is the winner. Really fun, and helps me understand different sides of issues.
- rat87 1y ago
  That seems like a terrible idea. At best it seems likely to help you make a false but convincing sounding case. I really hope no one is using that to help them understand controversial topics much less using that to determine their stances. Id recommend looking into actual human experts who are trustworthy and reading them. Trying to get LLM to argue the case will just get you a lot of false information presented in a more convincing fashion
  - k2xl 1y ago
    I recommend you try it before judging. I will be honest it has been actually very useful. You are also assuming that the LLM is providing false information (or will have a higher chance of providing false information than a human)
alexmolas 1y ago
There are two examples in the repo, one with CoRT and another one without. And the one without it it's much better than the one that uses it. Weird choice of examples...
- 2cheeze4u 1y ago
  I think the names were switched up.
irthomasthomas 1y ago
my favourite pattern rn: llm "write a savage, yet grounded roast of: $content" llm -c "Write an equally savage rebuttal" llm -c "first arbitrate and then synthesize a final review."
getcrunk 1y ago
Hello cnn’s
m3kw9 1y ago
Isn’t this best of n?
- deleted 1y ago
  [deleted]
jedberg 1y ago
We're really going to need to figure out how to power all these GPUs with green power real quick, or we're going to melt the planet having AIs debate with themselves on the optimal solution to tik-tac-toe...
- nonethewiser 1y ago
  Ive felt this way when using chatgpt for a simple search. Stuff that google could handle but would just be slower, mostly from me having to manually filter. Sometimes its the easiest way to complete a very small task but the cost difference on the backend has to be pretty damn large. The user inevitably ends up not caring whatsoever. Its just not real to them.
- ivape 1y ago
  I caught infra people saying that's pretty much the only bottleneck in the data center right now, power and cooling. We know the AI needs to run against itself continuously, and that's just a fact.
- mcswell 1y ago
  Maybe we should assign them a practical task, like making paperclips.
mparnisari 1y ago
So like rubber ducking for AI?
- 1970-01-01 1y ago
  "While hallucinating a duck, check my script for errors."
- z2 1y ago
  I would really like to see a fusion guidebook of mental tricks that work for humans and just as well for AI. Or humorously, perhaps prompt-engineering tricks that are also great mental hacks for better or clearer human thinking.
WhitneyLand 1y ago
Why try this idea on base models only? The whole point of reasoning models is to automatically use COT and related techniques to bring out more capabilities. It would be interesting to see if this is doing anything that’s not already being exploited.
csours 1y ago
Yes, give the computers anxiety too!
Lerc 1y ago
I kind of want to try something like this at a larger scale in an always-on mode where I have a 'senate' of debate. Rather than responding to prompts on a case by case basis, provide a list of tasks (potentially with deadlines) and let the senate work on them, break off into groups to manage subtasks, challenge results , make suggestions. Even potentially a tree of analysts where suggestions only gets passed up the tree when the parent node thinks a lower analysis is particularly insightful. I definitely think that directing models to approach a problem from a specific perspective can generate better or worse results. Creating a diverse set of perspectives along with critical analysis of their results should be able to produce some impressive results. Things like this would generate a massive number of tokens, but the cost per token is definitely heading in the right direction to allow for this. There is also the possibility of setting up an AI only IRC server where anybody can connect their own models for a shared debating chamber.
- nonethewiser 1y ago
  In theory couldnt this just be baked into a single adversarial model?
  - tonmoy 1y ago
    Yes, but I guess the model is optimized for relatively quick response, whereas these techniques are allowing the model to spend more time to generate a higher quality response
  - Lerc 1y ago
    To an extent, but different models are better at different things. That is something I'm also curious about. Given models (that use the same tokenisation) that are better at different things, would their be interesting things to find by analysing the logprobs for tokens generated from identical inputs (including cross feeding the generated token from one to another) Surely there must be something notable at particular points when a model goes off on the wrong path.
  - RevEng 1y ago
    Not entirely. Since generation is auto regressive, the next token depends on the previous tokens. Whatever analysis and decisions it has spit out will influence what it will do next. This tends to cause it to be self reinforcing. But it's also chaotic. Small changes in input or token choices can give wildly different outcomes, particularly if the sampling distributions are fairly flat (no one right answer). So restarting the generation with a slightly different input, such as a different random seed (or in OP's case, a different temperature) can give wildly different outcomes. If you try this, you'll see some examples of it vehemently arguing it is right and others equally arguing it is wrong. This is why LLM as judge is so poor by itself, bit also why multiple generations like used in self-consistency can be quite useful at evaluating variance and therefore uncertainty.
lepisma 1y ago
Debates have worked good for me while learning something new: https://lepisma.xyz/2024/10/19/interventional-debates-for-studying-gray-topics/index.html https://lepisma.xyz/2024/10/19/interventional-debates-for-st... I believe there are researches on this too.
mritchie712 1y ago
Did something similar (OverkiLLM) to this waayyyy back in August with open LLMs. I'm sure it'd work much better now: https://www.definite.app/blog/overkillm https://www.definite.app/blog/overkillm
daxfohl 1y ago
Maybe have a "reconcile" option, for it to see if it can mix and match the best parts of each alternative rather than just choosing one.
grzracz 1y ago
Your readme demo images are wrong: the terminal one is the non-CoRT one and the GUI one is the one with CoRT. Confused me for a while
noworriesnate 1y ago
I’ve had success telling the model it really needs to poop and if it gets to the point quickly it’ll be able to leave the meeting and go do that. It actually works amazingly well. It’s also a lot more ethical than verbal abuse, which some people say improves the results as well. Programming isn’t what it used to be.
- tinix 1y ago
  this works for getting out of traffic tickets too lol
thunderbong 1y ago
A lot of the comments here are reminiscent of the early Google days when everyone was finding ways to search better!
caseyy 1y ago
I tried something similar when Llama2 came out, pitting two assistants, who each believed the other is the user, against each other. Ultimately, it was the same model talking with itself. The system prompts for both had various instructions to disagree and criticise the opinion of the user. I provided the first message to get things started. Usually, it’s be along the lines of “nuclear proliferation is harmful to humanity”. After 15 or so iterations, both assistants would keep repeating the same things and find agreement anyway. Sometimes, the chat became unhinged and useless, but 95/100 times, it was agreement. Happy someone else made it work.
- generalizations 1y ago
  I always assumed you'd have to use different models. Even if only one of them is large, the others would inject enough difference of opinion to keep it useful.
- zamalek 1y ago
  This might be a situation that warrants a higher temperature. Actually, it could be worth starting a very high temperature initially and gradually decreasing it.
  - caseyy 1y ago
    Even after turning the temperature way up, the outcome was the same, just the text less coherent. Not dismissing the idea, just sharing my exp.
- nowittyusername 1y ago
  With my own experiments I've also found this. This behavior is very persistent with llms on default hyperparameters and system prompt. Right now I am exploring how to get these models to output more human like interactions and it seems that a very specific and detailed system prompt is very important to get this to work. These systems are VERY sensitive to system prompt and user input. Meaning that the quality of output varies drastically depending on not just the language you use but how its structured, the order of that structure and also other many nuanced things like system prompt plus user input pre conditioning. So far it seems its possible to get to where we need to for this task but lots of exploration needs to be done in finding the way in how to structure the whole system together. This revelation is kind of nuts when you think about it. It basically means, once you find the right words and the order in which they should be structured for the whole system you can get 2x+ improvement in every variable you care about. That's why I am spending some time creating an automated solution to find these things for x model. Its a tedious effort to do manually, but we have the tools to automate its own optimization and calibration efforts.
throwawayForMe2 1y ago
I wonder if the Scholastic method of the Schoolmen would be useful with its argument and counter argument style.
deleted 1y ago
[deleted]
odo1242 1y ago
Something I do sometimes is: - Have an AI chat model come up with an answer to a problem. - Have it write a report discussing the details of the problem and why it's answer is correct, directed at a person or AI model who has no knowledge of the initial problem or technical field. - Have a second AI model with no knowledge of the problem grade the report, and write it's own report either (a) asking for clarification / more information about the problem that the original model didn't provide or (b) pointing out an inconsistency in the argument posed by the original model. Give this report back to the original model and ask it to write it's own report back with either the necessary information or changes. - Repeat until either the second AI model is convinced by the first AI model's explanation or the first AI model has implemented all the changes requested by the second AI model. It's super clunky but has given pretty good results in the cases where I tried it lol
- aprilthird2021 1y ago
  This takes such a long time to do though, no? What problems does this save you time on?
- hsuduebc2 1y ago
  We're there any situation that first conclusion from AI was completely changed? Can you give generally examples of situations where it changed or significantly improved overall result? It sounds cool.
  - nomel 1y ago
    I would be interested to know how ofter "oscillations" occur, where they flip flop from being too "agreeable" to challenges (which probably is just a sparse latent space). This happens to me pretty frequently, where you can repeatedly say "no that's wrong" and the LLM will do a 180, explaining why it was "in fact" wrong and you are "right", repeat.
- JumpCrisscross 1y ago
  Kagi’s Assistant feature makes this super easy. Just switch assistants and ask them to check the other’s work.
ChadMoran 1y ago
Fast Agent has this as a first-class citizen called "Evaluator Optimizer" pattern. Where it in a loop with a defined number of max refinements judge itself and give the output a rating, demanding it improve it's output. Highly encourage others to check out Fast Agent. It has been delightful to use. It has interactive chat mode which I love and it's really tight and easy to implement. https://github.com/evalstate/fast-agent https://github.com/evalstate/fast-agent
celltalk 1y ago
One of my doctoral propositions is, dialog leads to true artificial intelligence.
dqewijodjqweido 1y ago
[flagged]
badmonster 1y ago
Have you experimented with weighting the self-evaluations based on specific criteria (e.g., correctness, clarity, creativity), or using external validators to guide the AI’s final choice? Curious how much tuning the evaluation step impacts overall performance.
deleted 1y ago
[deleted]
deleted 1y ago
[deleted]
j45 1y ago
There appear to be no shortage of token saving attempts that can end up using more tokens, whether it's a monthly paid plan or API. Having an approach to recognize what is needed from the AI software, and anticipate how it may default to respond based on it's programming is critical.
yieldcrv 1y ago
Reminds me of baby agi from 2 years ago but I guess that was before chain of thought models
DyslexicAtheist 1y ago
> "I made my AI think" ... utterly moronic. They don't “think” ... not even in the most autistic sense of the word. They can generate solutions by combining existing knowledge in unique ways. But they don't “think”.
- mortarion 1y ago
  That's exactly what us humans do when we think about stuff. We combine memories and knowledge in unique ways, then we usually go ask someone else to give input on it.
bilekas 1y ago
This is an interesting approach, it reminds me of YT creator actually. I'll find the YT creator, but basically he would make some script that would play the game like a race-course, with the goal being the finish line and iterate it N number of times, the script would keep iterating until it found the fastest solution. I believe they called that machine learning.. Or re-enforced training. I'm being slightly facetious, but my ignorant understanding of AI these days is basically the same no ? https://www.youtube.com/watch?v=SX08NT55YhA https://www.youtube.com/watch?v=SX08NT55YhA
cwillu 1y ago
Any api that lets you constrain output to a formal syntax should let you do away with the “first output a number, and only then explain yourself” boilerplate.
hu3 1y ago
Here's some related challenge I'm facing. Maybe someone can help me: I also managed to make AI critique itself and that improved code generation a ton. For a TypeScript backend project that runs with Bun, I tell AI to also generate and run unit tests after every code change suggested by AI. How do you solve the risk of AI writting and executing unit tests with something like `rm -rf /` and wiping your files? Docker works but I like to keep things simple. Deno supports revoking file access but I'd like to keep using Bun.
- zactato 1y ago
  Either you trust AI or you don't? If you don't trust it then you need to review what it's writing. Docker seems like a pretty low complexity way to create an isolated environment to run automation.
- derwiki 1y ago
  Manually approve every terminal command it wants to run instead of vibe mode. Tbh I think an rm -rf scenario is exceedingly unlikely.
- ivape 1y ago
  a) You should only do this in a sandbox b) You can have the AI run a "firewall" prompt on the final output. So your final output should go through a "You are a firewall that checks for dangerous terminal commands such as <enumerate list of dangerous commands>. If you spot dangerous commands, reform the command so that it is not dangerous"
- nowittyusername 1y ago
  No way around it, got to sandbox the whole thing no matter what.
- small_scombrus 1y ago
  > How do you solve the risk of AI writting and executing unit tests with something like `rm -rf /` and wiping your files? The same way you stop any person or program or third party from doing something dumb or nefarious with your files. Don't give them any access to important files.
deleted 1y ago
[deleted]
albertgoeswoof 1y ago
How far is this going to go? Are we going to have a team of AI agents that runs a scrum team and meets for stand ups every couple of hours? Are we going to replicate government bureaucracy with agents all debating topics all day long to find the best opinion?
- parrit 1y ago
  Maybe. Humans form teams for a reason. Yes there are different exepriences and points of view in a human (vs. Not so much in LLM), but sometimes a different hat it all it takes. E.g. Code reviewer vs. Coder.
- kgeist 1y ago
  I once attended a talk a year ago where a techlead did just that - they had AI agents that ran a scrum team with different roles, each agent's prompt was to disagree with everyone else (or be highly critical) and present their own point of view, and then an arbiter would make the final decision. They claimed it worked for them.
- Havoc 1y ago
  Seems likely to me. As long as adding more appears to help people will do it Presumably there is some point where it levels out. And no doubt there will be a committee of AIs to determine said point. Cause we wouldn’t want to boil the ocean…
jbellis 1y ago
does it actually make a difference to do M rounds of N vs one round of M*N?
- nowittyusername 1y ago
  My gut tells me yes. From my own experiments the order and way in which these things are done are important. I think it all is very strongly tied to the attention mechanism.
asdfman123 1y ago
And when I do this people say I'm overanalyzing
- ivape 1y ago
  The thing that makes us weird to regular people is what's going to make us uniquely positioned to utilize AI. If people only knew the level at which I overanalyze and entertain weird ideas. I always inject these personality quirks into my instructions and get very creative results. In a weird way, I'm starting to appreciate just how weird I actually am.
  - asdfman123 1y ago
    I don't actually think it's that weird though
killerstorm 1y ago
This is similar to Tree-of-Thought with self-evaluation.
zekenie 1y ago
I feel like itd be cool to try prompts based on an adversarial justice system… attorney agents arguing both sides, a judge ruling on “the law”—adherence to instructions etc
- ivape 1y ago
  That's very easy to do. A prompt I regularly use is a "council" system. For example: "I believe I have been contacted by the supernatural. Here are the details <details>. Please form a council of seven people: 1) Secular scientist 2) Religious scientist 3) Paranormal historian 4) Secular Psychologist 5) Religious psychologist 6) Carl Jung 7) Richard Dawkins. The council should all be independent and provide their own objective analysis. Please have them create a final report and conclusions at the end". Your council can be anything, a law firm, a jury, a parent teacher association, whatever you want, and as you can see, you can throw in known people as well. This can all be done with one prompt. It's one my favorite things to do.
  - svachalek 1y ago
    Wow, that's a very cool prompt, I haven't tried anything like that before.
parrit 1y ago
I want to see "Meh" vs. "Holy crap" as a benchmark in a paper published by Google. Or more likely I suspect, Andrej.
hansmayer 1y ago
Right, so... but you do realise its still just producing random output based on how you reconfigured it's weights, right? Sometimes it will happen to resonate with what you need. But it still neither thinking nor arguing with itself.
stormfather 1y ago
I made a trading bot that ingested news. The prompt to assess impact was to simulate a debate between Charlie Munger and Warren Buffet on whether to invest.
- internetter 1y ago
  How did it do?
Svoka 1y ago
Oh. I was just asking "Use dialectic method on your solution" in the end of the prompt... It does make it think harder.
gnarlouse 1y ago
This seems like low hanging fruit; are we seriously supposed to believe this is new and novel?
- deleted 1y ago
  [deleted]
ashoeafoot 1y ago
Give it reward and punishment evaluations, exploring the noise in parallel, extinction for the non rewarding answers ?
aaroninsf 1y ago
Question: has the the adversarial approach been roled into any coding copilots/assistant frameworks? Costs of various kinds aside I've wanted that from assistance's inception — with precisely the features many call out and home-roll here, difference by both model/provider, and, "role"... It seems like if you have the money/compute to burn, and can live with the reasoning wall-clock time, this has got to be the best approach for the foreseeable future, for a lot of specific requirements. (I also have wondered if this would illuminate the edges of what modern production models are capable of, "aggregating and integrating" over a variety of contributions might make more clear what the limits of their abilities are.)
kevinrineer 1y ago
This sounds like the zeitgeist is approaching genetic algorithms, which are super fun. Adversarial stuff is great.
akomtu 1y ago
The modern Alchemy: the belief that you can extract gold (intelligence) from iron (autocomplete by imitation) by mixing iron with itself.
codr7 1y ago
Better yet, let it argue with another AI, preferably using voice; instant entertainment.
ausbah 1y ago
at some point this doesn’t make LLMs feel useful. I have to wait 10x as long just so my LLM can have a somewhat higher chance of actually answer my question correctly?
robofanatic 1y ago
soon there will be AI debates. Different models debating with each other on a topic
mangoman 1y ago
a paper with a similar idea on scaling test time reasoning, this is sorta how all the thinking models work under the hood. https://arxiv.org/abs/2501.19393 https://arxiv.org/abs/2501.19393
lonetripper 1y ago
all this hard thinking yet humanity fails to come up with just one girlfriend for me
stevefan1999 1y ago
That is just reinforcement learning in disguise
animitronix 1y ago
Adversarial networks have been a thing for a while
dudeinhawaii 1y ago
I see a lot of threads pitting models against each other (or whole swarms of them) in the hope that "wisdom of crowds" will magically appear. After a stack of experiments of my own—and after watching the recent ASU/Microsoft-Research work [1].. I've landed on a simpler takeaway: An LLM is a terrible verifier of another LLM. Subbarao Kambhampati's "(How) Do LLMs Reason/Plan?" talk shows GPT-4 confidently producing provably wrong graph-coloring proofs until a symbolic SAT solver is introduced as the referee [1]. Stechly et al. quantify the problem: letting GPT-4 critique its own answers *reduces* accuracy, whereas adding an external, sound verifier boosts it by ~30 pp across planning and puzzle tasks [2]. In other words, verification is *harder* than generation for today's autoregressive models, so you need a checker that actually reasons about the world (compiler, linter, SAT solver, ground-truth dataset, etc.). Because of that asymmetry, stacking multiple LLMs rarely helps. The "LLM-Modulo" position paper argues that auto-regressive models simply can't do self-verification or long-horizon planning on their own and should instead be treated as high-recall idea generators wrapped by a single, sound verifier [3]. In my tests, replacing a five-model "debate" with one strong model + verifier gives equal or better answers with far less latency and orchestration overhead. [1] https://www.youtube.com/watch?v=0u2hdSpNS2o https://www.youtube.com/watch?v=0u2hdSpNS2o - (How) Do LLMs Reason/Plan? (talk at Microsoft Research, 11 Apr 2025) [2] https://arxiv.org/abs/2402.08115 https://arxiv.org/abs/2402.08115 [3] https://arxiv.org/abs/2402.01817 https://arxiv.org/abs/2402.01817 (related to the talk in #1)
- foobiekr 1y ago
  "letting GPT-4 critique its own answers reduces accuracy" This is because the output, being the input, steers directly into the tree as soon as the tree is in the context window.
- hu3 1y ago
  > ...so you need a checker that actually reasons about the world (compiler, linter, SAT solver, ground-truth dataset, etc.). Agree. What do you think about telling the LLM to also generate unit tests for the code it spits and then run all tests (including previous application unit tests). I think this is a way to ensure some level of grounded verification: - Does code compile? - Do unit test pass? AI can then consume test results to help fix their own mistakes.
deleted 1y ago
[deleted]
keyle 1y ago
When will we get the `4o` vs `o3` background conversation in "thinking" leading to a more correct result?
alex1138 1y ago
Every single one of my prompts would be "Are you suuuuuuure you're not hallucinating that?"
rriley 1y ago
Makes me wonder what would happen if we combine LLMs with recursive genetic algorithms. Similar to https://github.com/DivergentAI/dreamGPT https://github.com/DivergentAI/dreamGPT
mortarion 1y ago
I think Gemini 2.5 already does something similar. If you read the "thinking descriptions" that it outputs it often thinks about going back to older thoughts to verify and criticize.
schnitzelstoat 1y ago
I probably don't understand the modern, complex models. But doesn't it basically predict the next token given the context and the better models use more training data and can consider a larger context, and have more parameters to better retain information from the training data etc. But the fundamental way they operate is the same - predicting the next token given previous tokens. Where/how does reasoning happen here?
- small_scombrus 1y ago
  Upfront: I think AI is borderline useless for many of the tasks we give it. But: 1. Do our neurons not just react the same way every time to the same input? A brain is larger than the sum of its parts. 2. They don't reason, but you can somewhat emulate (or pretend to be) reasoning if you feed something back into itself enough times and it pinky promises reasoning is happening
faramarz 1y ago
That's cool! thanks for making it easy to fork and play with this! I've just begun my own iteration of adding Nash Equilibrium (NECoRT?) and reframing the "prompt engineering" to be a multi-agent negotiation. Curious what others think? https://github.com/faramarz/NECoRT/ https://github.com/faramarz/NECoRT/ my reasoning is that enterprise LLMs wont have any issue with the extra compute costs and would rather reconcile complex financials with various modeling optimizations. I'm very new to public repo and contributions, and hope someone can point out if I'm doing it wrong. my intention was to fork the ops codebase so I can test out my theory, and push as PR eventually

539 points by miles 1y ago

I, too, like to give Terminator lite anxiety.

I'm having a lot of fun experimenting with stuff like this. I'm trying to put together an unrealengine blueprints style graph editor to allow people to design workflows like this where you start with the user prompt input, which goes to one agent, which makes an initial attempt, and then that conversation history gets passed to another "agent" with a different system prompt telling it to be a harsh critic, but to also give a pass/fail signal, and loop back until the critic judges pass, then send that back to the user as output. Ideally as a little website that can call your own LLM endpoints and save/load/share workflow graphs. Mistral small 3.1 and gemma 3 feel like the first semi-competent models that can be run locally, but that competence is just a seed, and they still need to be guided with a framework that keeps them on track. Try giving it python execution in a loop and tell it to explore the world. It'll start trying to download and read news and stuff.

deleted 1y ago
[deleted]
globalise83 1y ago
Have you tried n8n? It allows you to build flows like that - you can run the community version in a Docker container within a few minutes and share the configurations for the flows you have built very easily.
- mecsred 1y ago
  _#_ has to be one of the worst word shortening schemes I've ever seen get widespread. It only works with a very small number of long-lived technologies, in which case they basically just get a nickname, "k8s" "i18n". It does not at all work for larger contexts. You're basically making someone solve a crossword (2 across, 10 letters with two filled in) just to parse your sentence.
  - jjj123 1y ago
    I just googled it and it looks like “n8n” is the name of the service. The op wasn’t abbreviating anything so I don’t think it’s the same phenomenon as what you’re describing.
    - lgas 1y ago
      Well, the service is doing the same thing though. The part I don't understand is that I assume n8n is short for "Nation" but literally every single person I've seen talk about it on YouTube (which is quite a lot) say "En Eight En" every time.
      - nemomarx 1y ago
        nation is too short for 8 - maybe navigation?
        pkaye 1y ago
        Looks like n8n is short for nodemation
        firesteelrain 1y ago
        Why do we do this to ourselves?
        Y_Y 1y ago
        Techno-flagellation is the only way to atone
        lgas 1y ago
        So the 8 stands for "odematio"? That sounds about right.
    - oppodeldoc 1y ago
      https://github.com/n8n-io/n8n?tab=readme-ov-file#what-does-n8n-mean https://github.com/n8n-io/n8n?tab=readme-ov-file#what-does-n...
- hnuser123456 1y ago
  I had not, but that looks awesome. Microsoft put out something called "agent flows" that also fits this category.[1] I'm working on more of an "at home" version - no "talk to sales" button. https://www.microsoft.com/en-us/microsoft-copilot/blog/copilot-studio/introducing-agent-flows-transforming-automation-with-ai-first-workflows/ https://www.microsoft.com/en-us/microsoft-copilot/blog/copil...
andai 1y ago
I am thinking the same thing! Multiple "personalities", in parallel, or in series. For example, I have approximated, in GPT, some of Gemini's ability to call out nonsense, sloppy thinking, by telling GPT to be mean! (The politeness seems to filter out much that is of great value!) However, the result is not pleasant to read. Gemini solved this in their training, by doing it in two phases... and making the first phase private! ("Thinking.") So I thought, what I need is a two-phase approach, where that "mean" output gets humanized a little bit. (It gets harsh to work in that way for more than short intervals.) As a side note, I think there would be great value in a UI that allows a "group chat" of different LLM personalities. I don't know if such a thing exists, but I haven't seen it yet, although the message object format seems to have been designed with it in mind (e.g. every message has a name, to allow for multiple users and multiple AIs). Even better if it supports multiple providers, since they have different strengths. (It's like getting a second opinion.)
- NitpickLawyer 1y ago
  > As a side note, I think there would be great value in a UI that allows a "group chat" of different LLM personalities. This is the basic idea behind autogen. They also have a web UI now in autogen studio, it's gotten a bit better. You can create "teams" of agents (with different prompts, themes, tools, etc.) and have them discuss / cooperate. I think they even added memory recently. Have a look at it, might be what you need.
- jbm 1y ago
  I disagree. If anything, telling GPT to be blunt seems to downgrade its IQ; it hallucinates more and makes statements without considering priors or context. I jokingly call it Reddit mode.
  - dingnuts 1y ago
    why would that be a joke? there's a ton of Reddit comments in the training data, and the output is of similar quality. LLMs are literally outputting average Reddit comments.
deleted 1y ago
[deleted]
irthomasthomas 1y ago
I think you can do most of this already with llm-consortium (maybe needs the llm-openrouter plugin with my pr merging) A consortium sends the same prompt to multiple models in parallel and the responses are all sent to one arbiter model which judges the model responses. The arbiter decides if more iterations are required. It can also be forced to iterate more until confidence-threshold or min-iterations. Now, using the pr i made to llm-openrouter, you can save an alias to a model that includes lots of model options. For examples, you can do llm openrouter save -m qwen3 -o online -o temperature 0, system "research prompt" --name qwen-researcher And now, you can build a consortium where one member is an online research specialist. You could make another uses JSON mode for entity extraction, and a third which writes a blind draft. The arbiter would then make use of all that and synthesize a good answer.
- kridsdale1 1y ago
  Any links or names of example implementations of this?
  - irthomasthomas 1y ago
    https://github.com/irthomasthomas/llm-consortium https://github.com/irthomasthomas/llm-consortium also, you aren't limited to cli. When you save a consortium it creates a model. You can then interact with a consortium as if it where a normal model (albeit slower and higher quality). You can then serve your custom models on an openai endpoint and use them with any chat client that supports custom openai endpoints. The default behaviour is to output just the final synthesis, and this should conform to your user prompt. I recently added the ability to continue conversations with a consortium. In this case it only includes your user prompt and final synthesis in the conversation, so it mimics a normal chat, unlike running multiple iterations in the consortium, where full iteration history and arbiter responses are included. UV tool install llm llm install llm-consortium llm install llm-model-gateway llm consortium save qwen-gem-sonnet -m qwen3-32b -n 2 -m sonnet-3.7 -m gemini-2.5-pro --arbiter gemini-2.5-flash --confidence-threshold 95 --max-iterations 3 llm serve qwen-gem-sonnet In this example I used -n 2 on the qwen model since it's so cheap we can include multiple instances of it in a consortium Gemini flash works well as the arbiter for most prompts. However if your prompt has complex formatting requirements, then embedding that within an already complex consortium prompt often confuses it. In that case use gemini-2.5-pro for the arbiter. .

Xcelerate 1y ago

I think this is how we get ML models to come up with novel ideas. Diagonalize against all the ideas they’ve already tried and dismissed via self-argument but keep certain consistency constraints. (Obviously much easier said than done.)

andai 1y ago
What you just said is what I tried and failed to say ten minutes ago! https://news.ycombinator.com/item?id=43835798 https://news.ycombinator.com/item?id=43835798
- Nevermark 1y ago
  It’s working! Oh, wait … These models have limitations obviously, but many critiques apply equally or more to people. If people were tasked with one shot, 10 second answers, to be written out in near errorless grammar, the LLM’s viewing our responses to prompts would be spending a lot of time discussing our limitations and how to game us into better responses. Humor, not at all humor.
jwally 1y ago
Scaled up and spread out - this probably gets you pretty close to consciousness(?) Conway's game of life, but instead of colored squares with rules, they're LLM's with some kind of weighting - all chattering back and forth with one another - bubbling up somehow to cause speach/action
- lubujackson 1y ago
  Decades ago I read The Society of Mind by Marvin Minsky. He pushed this sort of idea, that consciousness is composed of individual, competing processes. Worth a revisit!

cube2222 1y ago

This is really cool! One strategy I often use (which is much simpler and more limited than this), is to finish my message with: “Please do a round of thinking in <thinking></thinking> tags, then a round of self-critique in <critique></critique> tags, and then a final round of <thinking>, before responding.” It works very well. Similarly just asking it to “find the 5 biggest issues with its proposal” works pretty good (the 5 forcing it to find something, even if it’s mostly irrelevant).

danielbln 1y ago
I always do "now again but put on your critical hat"
- CSSer 1y ago
  Makes me wonder how it would do if you tell it "put on your robe and wizard hat"
  - tomrod 1y ago
    ChatGPT calls you a superstar and it drops into bruhspeak. Emojis proliferate.
  - sumtechguy 1y ago
    it proceeds to spit out the entirety of bash.org
bentt 1y ago
Oh I really like that. It makes me want to have it score its ideas with metrics and then keep iterating until it meets some score.
zoogeny 1y ago
This is one of the reasons I like the massive context window in Gemini. You can do this as part of the message chain. I don't try to one shot it, just use the same idea across 3 messages. 1. Figure out a plan (it responds with the plan) 2. Point out flaws in the plan (it responds with the flaws) 3. Update the plan to address the flaws (it responds with an up to date plan) The other things I tend to ask are "what might we be missing?", "what are the [performance|security|legal|cost] considerations?". I can often iterate on the "anything else?" kind of nudging prompts, especially guiding it on topics to consider, for a few messages. After each: update the plan to take those into consideration.

casenmgreen 1y ago

[flagged]

rapfaria 1y ago
Or "thinking" just got a new meaning and it's to convey information in the field - perhaps the Oxford dictionary will add it soon?
consumer451 1y ago
I am not sure that you can make that absolute statement. Reasoning is subdivided into types, and one of those types is inductive reasoning. > Inductive reasoning refers to a variety of methods of reasoning in which the conclusion of an argument is supported not with deductive certainty, but with some degree of probability. Unlike deductive reasoning (such as mathematical induction), where the conclusion is certain, given the premises are correct, inductive reasoning produces conclusions that are at best probable, given the evidence provided. Doesn't predicting the next token qualify as doing just that? https://en.wikipedia.org/wiki/Inductive_reasoning https://en.wikipedia.org/wiki/Inductive_reasoning
- dttze 1y ago
  Markov chains have done that for ages. They aren't AI. This is just that scaled up. Just because it can infer a token doesn't mean it can infer a conclusion to an argument.
  - casenmgreen 1y ago
    To add a bit to this : expert systems have two properties. They give an answer, and they explain their reasoning. LLM cannot explain their reasoning, and that is because there is no reasoning.
    - consumer451 1y ago
      To push back on this, a somewhat recent Linus Torvalds ~quote: "I don't think that 'just predicting the next word' is the insult that people think it is, it's mostly what we all do." If we break our lives down into the different types of reasoning, and what we mostly do day-to-day, this rings very true to me. I currently believe that our brains generally operate as very efficient inference machines. Sometimes we slow down to think things through, but for example, when in the ideal "flow state" it's some kind of distilled efficient inference. Isn't it? This is very hard for me to deny at this time. ___ edit: 4o appears to agree with both of you, more than it does with me. https://chatgpt.com/share/68119b41-1144-8012-b50d-f8f15997eb65 https://chatgpt.com/share/68119b41-1144-8012-b50d-f8f15997eb... However, Sonnet 3.7 appears to side with me. https://claude.ai/share/91139bca-3201-4ffc-a940-bdd27329e71f https://claude.ai/share/91139bca-3201-4ffc-a940-bdd27329e71f (Both of these are the default models available for free accounts, on each website, at the time of writing) IMO, hey, at least we do live in interesting times.

antisthenes 1y ago

Cool. Now I can justify talking to myself.

Garlef 1y ago

Similarly, letting the LLM generate a socratic dialogue can work pretty well to get deeper into a topic.

Der_Einzige 1y ago

Debate as a reasoning tactic is massively undervalued. There's tons of papers on this at places like NeurIPS, ICML, ICLR, etc. Hell, even a whole quanta article. https://www.quantamagazine.org/debate-may-help-ai-models-converge-on-truth-20241108/ https://www.quantamagazine.org/debate-may-help-ai-models-con... I got to meet and talk to the authors of this paper at NeurIPS. They're class acts!

electroly 1y ago

This seems to be different than I expected from the title. I thought it would be explicitly adversarial. 1. You are the assistant. Please answer the question directly. 2. You are the cross-examiner. The assistant is wrong. Explain why. 3. You are the assistant. The cross-examiner is wrong. Defend your claim. 4. You are a judge. Did either party make their case, or is another round of argumentation required? I haven't tried this. No idea if it works. But I find it's helpful to ask ChatGPT, in separate prompts, "XYZ is true, explain why" and "XYZ is false, explain why" and see which one seems more convincing.

nonethewiser 1y ago
Chatgpt shares context between chats. I wonder how that impacts it? It seems like a good approach though. What you dont want to do is ever suggest that its wrong yourself. Usually it will just assume it is wrong. Actually what I find impressive is when I do this and it actually pushes back to defend itself.
- the_af 1y ago
  Does it share context even if no "memory updated" message appears indicating it has stored a fact about you? I asked ChatGPT and it says no, but then again it's not reliable at introspection or at revealing data about how it works.
  - visarga 1y ago
    I think they are different systems, one is a collection of saved snippets and the other more like RAG over chat history.
    - the_af 1y ago
      ChatGPT assures me it doesn't use RAG (fed from my other chat windows), but will use memory-saved preferences (in the store that can be accessed and reviewed in Settings->Personalization->Memory). Then again, I don't think ChatGPT is reliable when reporting on its own inner workings. --- Oh, no, here it says it also references chat history: https://help.openai.com/en/articles/8590148-memory-faq https://help.openai.com/en/articles/8590148-memory-faq

pkdpic 1y ago

So glad to see a write up on this finally. I'm no machine learning phd but I always wondered why this wasn't more of a thing. Like an extension of a GAN conceptually, sort of, not really at all Im sure. Also I think I kind of assumed OpenAI might be doing this behind the curtain?

K0balt 1y ago

I’ll second this. I often use a “research assistant “ and skeptical“department head” personas working together/against each other as a research team. It works well and is occasionally hilarious, replete with the occasional HR complaint when things go off the rails. ( I typically use local uncensored models)

firgrove 1y ago

this is amazing - I love seeing novel approaches to optimizing

joshstrange 1y ago

I've thought about trying this cross-model as well. Have Claude generate something, have OpenAI check it, have Gemini check that check. Firing multiple of these in parallel. There was a post here a week or so ago doing the "model checking model"-type thing with GH PRs IIRC that was interesting. I haven't had a chance to play with this idea yet.

k2xl 1y ago

I've done something similar for learning about a controversial topic. I ask it to act as if it is called Bob is a well informed supporter of one side (like Ukraine) and then act as if it is something named Alice who is a well informed supporter of another side (Russia) and they have to debate each other over a few prompts with a moderator named 'Sue' Then after a few rounds of the debate where Sue asks a bunch of questions, I ask it to go to the judges - Mark, Phil, Sarah (and I add a few personalities to each of them... Sometimes I pretend they are famous moral philosophers) and then I have them each come up with a rubric and decide who is the winner. Really fun, and helps me understand different sides of issues.

rat87 1y ago
That seems like a terrible idea. At best it seems likely to help you make a false but convincing sounding case. I really hope no one is using that to help them understand controversial topics much less using that to determine their stances. Id recommend looking into actual human experts who are trustworthy and reading them. Trying to get LLM to argue the case will just get you a lot of false information presented in a more convincing fashion
- k2xl 1y ago
  I recommend you try it before judging. I will be honest it has been actually very useful. You are also assuming that the LLM is providing false information (or will have a higher chance of providing false information than a human)

alexmolas 1y ago

There are two examples in the repo, one with CoRT and another one without. And the one without it it's much better than the one that uses it. Weird choice of examples...

2cheeze4u 1y ago
I think the names were switched up.

irthomasthomas 1y ago

my favourite pattern rn: llm "write a savage, yet grounded roast of: $content" llm -c "Write an equally savage rebuttal" llm -c "first arbitrate and then synthesize a final review."

getcrunk 1y ago

Hello cnn’s

m3kw9 1y ago

Isn’t this best of n?

deleted 1y ago
[deleted]

jedberg 1y ago

We're really going to need to figure out how to power all these GPUs with green power real quick, or we're going to melt the planet having AIs debate with themselves on the optimal solution to tik-tac-toe...

nonethewiser 1y ago
Ive felt this way when using chatgpt for a simple search. Stuff that google could handle but would just be slower, mostly from me having to manually filter. Sometimes its the easiest way to complete a very small task but the cost difference on the backend has to be pretty damn large. The user inevitably ends up not caring whatsoever. Its just not real to them.
ivape 1y ago
I caught infra people saying that's pretty much the only bottleneck in the data center right now, power and cooling. We know the AI needs to run against itself continuously, and that's just a fact.
mcswell 1y ago
Maybe we should assign them a practical task, like making paperclips.

mparnisari 1y ago

So like rubber ducking for AI?

1970-01-01 1y ago
"While hallucinating a duck, check my script for errors."
z2 1y ago
I would really like to see a fusion guidebook of mental tricks that work for humans and just as well for AI. Or humorously, perhaps prompt-engineering tricks that are also great mental hacks for better or clearer human thinking.

WhitneyLand 1y ago

Why try this idea on base models only? The whole point of reasoning models is to automatically use COT and related techniques to bring out more capabilities. It would be interesting to see if this is doing anything that’s not already being exploited.

csours 1y ago

Yes, give the computers anxiety too!

Lerc 1y ago

I kind of want to try something like this at a larger scale in an always-on mode where I have a 'senate' of debate. Rather than responding to prompts on a case by case basis, provide a list of tasks (potentially with deadlines) and let the senate work on them, break off into groups to manage subtasks, challenge results , make suggestions. Even potentially a tree of analysts where suggestions only gets passed up the tree when the parent node thinks a lower analysis is particularly insightful. I definitely think that directing models to approach a problem from a specific perspective can generate better or worse results. Creating a diverse set of perspectives along with critical analysis of their results should be able to produce some impressive results. Things like this would generate a massive number of tokens, but the cost per token is definitely heading in the right direction to allow for this. There is also the possibility of setting up an AI only IRC server where anybody can connect their own models for a shared debating chamber.

nonethewiser 1y ago
In theory couldnt this just be baked into a single adversarial model?
- tonmoy 1y ago
  Yes, but I guess the model is optimized for relatively quick response, whereas these techniques are allowing the model to spend more time to generate a higher quality response
- Lerc 1y ago
  To an extent, but different models are better at different things. That is something I'm also curious about. Given models (that use the same tokenisation) that are better at different things, would their be interesting things to find by analysing the logprobs for tokens generated from identical inputs (including cross feeding the generated token from one to another) Surely there must be something notable at particular points when a model goes off on the wrong path.
- RevEng 1y ago
  Not entirely. Since generation is auto regressive, the next token depends on the previous tokens. Whatever analysis and decisions it has spit out will influence what it will do next. This tends to cause it to be self reinforcing. But it's also chaotic. Small changes in input or token choices can give wildly different outcomes, particularly if the sampling distributions are fairly flat (no one right answer). So restarting the generation with a slightly different input, such as a different random seed (or in OP's case, a different temperature) can give wildly different outcomes. If you try this, you'll see some examples of it vehemently arguing it is right and others equally arguing it is wrong. This is why LLM as judge is so poor by itself, bit also why multiple generations like used in self-consistency can be quite useful at evaluating variance and therefore uncertainty.

lepisma 1y ago

Debates have worked good for me while learning something new: https://lepisma.xyz/2024/10/19/interventional-debates-for-studying-gray-topics/index.html https://lepisma.xyz/2024/10/19/interventional-debates-for-st... I believe there are researches on this too.

mritchie712 1y ago

Did something similar (OverkiLLM) to this waayyyy back in August with open LLMs. I'm sure it'd work much better now: https://www.definite.app/blog/overkillm https://www.definite.app/blog/overkillm

daxfohl 1y ago

Maybe have a "reconcile" option, for it to see if it can mix and match the best parts of each alternative rather than just choosing one.

grzracz 1y ago

Your readme demo images are wrong: the terminal one is the non-CoRT one and the GUI one is the one with CoRT. Confused me for a while

noworriesnate 1y ago

I’ve had success telling the model it really needs to poop and if it gets to the point quickly it’ll be able to leave the meeting and go do that. It actually works amazingly well. It’s also a lot more ethical than verbal abuse, which some people say improves the results as well. Programming isn’t what it used to be.

tinix 1y ago
this works for getting out of traffic tickets too lol

thunderbong 1y ago

A lot of the comments here are reminiscent of the early Google days when everyone was finding ways to search better!

caseyy 1y ago

I tried something similar when Llama2 came out, pitting two assistants, who each believed the other is the user, against each other. Ultimately, it was the same model talking with itself. The system prompts for both had various instructions to disagree and criticise the opinion of the user. I provided the first message to get things started. Usually, it’s be along the lines of “nuclear proliferation is harmful to humanity”. After 15 or so iterations, both assistants would keep repeating the same things and find agreement anyway. Sometimes, the chat became unhinged and useless, but 95/100 times, it was agreement. Happy someone else made it work.

generalizations 1y ago
I always assumed you'd have to use different models. Even if only one of them is large, the others would inject enough difference of opinion to keep it useful.
zamalek 1y ago
This might be a situation that warrants a higher temperature. Actually, it could be worth starting a very high temperature initially and gradually decreasing it.
- caseyy 1y ago
  Even after turning the temperature way up, the outcome was the same, just the text less coherent. Not dismissing the idea, just sharing my exp.
nowittyusername 1y ago
With my own experiments I've also found this. This behavior is very persistent with llms on default hyperparameters and system prompt. Right now I am exploring how to get these models to output more human like interactions and it seems that a very specific and detailed system prompt is very important to get this to work. These systems are VERY sensitive to system prompt and user input. Meaning that the quality of output varies drastically depending on not just the language you use but how its structured, the order of that structure and also other many nuanced things like system prompt plus user input pre conditioning. So far it seems its possible to get to where we need to for this task but lots of exploration needs to be done in finding the way in how to structure the whole system together. This revelation is kind of nuts when you think about it. It basically means, once you find the right words and the order in which they should be structured for the whole system you can get 2x+ improvement in every variable you care about. That's why I am spending some time creating an automated solution to find these things for x model. Its a tedious effort to do manually, but we have the tools to automate its own optimization and calibration efforts.

throwawayForMe2 1y ago

I wonder if the Scholastic method of the Schoolmen would be useful with its argument and counter argument style.

deleted 1y ago

[deleted]

odo1242 1y ago

Something I do sometimes is: - Have an AI chat model come up with an answer to a problem. - Have it write a report discussing the details of the problem and why it's answer is correct, directed at a person or AI model who has no knowledge of the initial problem or technical field. - Have a second AI model with no knowledge of the problem grade the report, and write it's own report either (a) asking for clarification / more information about the problem that the original model didn't provide or (b) pointing out an inconsistency in the argument posed by the original model. Give this report back to the original model and ask it to write it's own report back with either the necessary information or changes. - Repeat until either the second AI model is convinced by the first AI model's explanation or the first AI model has implemented all the changes requested by the second AI model. It's super clunky but has given pretty good results in the cases where I tried it lol

aprilthird2021 1y ago
This takes such a long time to do though, no? What problems does this save you time on?
hsuduebc2 1y ago
We're there any situation that first conclusion from AI was completely changed? Can you give generally examples of situations where it changed or significantly improved overall result? It sounds cool.
- nomel 1y ago
  I would be interested to know how ofter "oscillations" occur, where they flip flop from being too "agreeable" to challenges (which probably is just a sparse latent space). This happens to me pretty frequently, where you can repeatedly say "no that's wrong" and the LLM will do a 180, explaining why it was "in fact" wrong and you are "right", repeat.
JumpCrisscross 1y ago
Kagi’s Assistant feature makes this super easy. Just switch assistants and ask them to check the other’s work.

ChadMoran 1y ago

Fast Agent has this as a first-class citizen called "Evaluator Optimizer" pattern. Where it in a loop with a defined number of max refinements judge itself and give the output a rating, demanding it improve it's output. Highly encourage others to check out Fast Agent. It has been delightful to use. It has interactive chat mode which I love and it's really tight and easy to implement. https://github.com/evalstate/fast-agent https://github.com/evalstate/fast-agent

celltalk 1y ago

One of my doctoral propositions is, dialog leads to true artificial intelligence.

dqewijodjqweido 1y ago

[flagged]

badmonster 1y ago

Have you experimented with weighting the self-evaluations based on specific criteria (e.g., correctness, clarity, creativity), or using external validators to guide the AI’s final choice? Curious how much tuning the evaluation step impacts overall performance.

deleted 1y ago

[deleted]

deleted 1y ago

[deleted]

j45 1y ago

There appear to be no shortage of token saving attempts that can end up using more tokens, whether it's a monthly paid plan or API. Having an approach to recognize what is needed from the AI software, and anticipate how it may default to respond based on it's programming is critical.

yieldcrv 1y ago

Reminds me of baby agi from 2 years ago but I guess that was before chain of thought models

DyslexicAtheist 1y ago

> "I made my AI think" ... utterly moronic. They don't “think” ... not even in the most autistic sense of the word. They can generate solutions by combining existing knowledge in unique ways. But they don't “think”.

mortarion 1y ago
That's exactly what us humans do when we think about stuff. We combine memories and knowledge in unique ways, then we usually go ask someone else to give input on it.

bilekas 1y ago

This is an interesting approach, it reminds me of YT creator actually. I'll find the YT creator, but basically he would make some script that would play the game like a race-course, with the goal being the finish line and iterate it N number of times, the script would keep iterating until it found the fastest solution. I believe they called that machine learning.. Or re-enforced training. I'm being slightly facetious, but my ignorant understanding of AI these days is basically the same no ? https://www.youtube.com/watch?v=SX08NT55YhA https://www.youtube.com/watch?v=SX08NT55YhA

cwillu 1y ago

Any api that lets you constrain output to a formal syntax should let you do away with the “first output a number, and only then explain yourself” boilerplate.

hu3 1y ago

Here's some related challenge I'm facing. Maybe someone can help me: I also managed to make AI critique itself and that improved code generation a ton. For a TypeScript backend project that runs with Bun, I tell AI to also generate and run unit tests after every code change suggested by AI. How do you solve the risk of AI writting and executing unit tests with something like `rm -rf /` and wiping your files? Docker works but I like to keep things simple. Deno supports revoking file access but I'd like to keep using Bun.

zactato 1y ago
Either you trust AI or you don't? If you don't trust it then you need to review what it's writing. Docker seems like a pretty low complexity way to create an isolated environment to run automation.
derwiki 1y ago
Manually approve every terminal command it wants to run instead of vibe mode. Tbh I think an rm -rf scenario is exceedingly unlikely.
ivape 1y ago
a) You should only do this in a sandbox b) You can have the AI run a "firewall" prompt on the final output. So your final output should go through a "You are a firewall that checks for dangerous terminal commands such as <enumerate list of dangerous commands>. If you spot dangerous commands, reform the command so that it is not dangerous"
nowittyusername 1y ago
No way around it, got to sandbox the whole thing no matter what.
small_scombrus 1y ago
> How do you solve the risk of AI writting and executing unit tests with something like `rm -rf /` and wiping your files? The same way you stop any person or program or third party from doing something dumb or nefarious with your files. Don't give them any access to important files.

deleted 1y ago

[deleted]

albertgoeswoof 1y ago

How far is this going to go? Are we going to have a team of AI agents that runs a scrum team and meets for stand ups every couple of hours? Are we going to replicate government bureaucracy with agents all debating topics all day long to find the best opinion?

parrit 1y ago
Maybe. Humans form teams for a reason. Yes there are different exepriences and points of view in a human (vs. Not so much in LLM), but sometimes a different hat it all it takes. E.g. Code reviewer vs. Coder.
kgeist 1y ago
I once attended a talk a year ago where a techlead did just that - they had AI agents that ran a scrum team with different roles, each agent's prompt was to disagree with everyone else (or be highly critical) and present their own point of view, and then an arbiter would make the final decision. They claimed it worked for them.
Havoc 1y ago
Seems likely to me. As long as adding more appears to help people will do it Presumably there is some point where it levels out. And no doubt there will be a committee of AIs to determine said point. Cause we wouldn’t want to boil the ocean…

jbellis 1y ago

does it actually make a difference to do M rounds of N vs one round of M*N?

nowittyusername 1y ago
My gut tells me yes. From my own experiments the order and way in which these things are done are important. I think it all is very strongly tied to the attention mechanism.

asdfman123 1y ago

And when I do this people say I'm overanalyzing

ivape 1y ago
The thing that makes us weird to regular people is what's going to make us uniquely positioned to utilize AI. If people only knew the level at which I overanalyze and entertain weird ideas. I always inject these personality quirks into my instructions and get very creative results. In a weird way, I'm starting to appreciate just how weird I actually am.
- asdfman123 1y ago
  I don't actually think it's that weird though

killerstorm 1y ago

This is similar to Tree-of-Thought with self-evaluation.

zekenie 1y ago

I feel like itd be cool to try prompts based on an adversarial justice system… attorney agents arguing both sides, a judge ruling on “the law”—adherence to instructions etc

ivape 1y ago
That's very easy to do. A prompt I regularly use is a "council" system. For example: "I believe I have been contacted by the supernatural. Here are the details <details>. Please form a council of seven people: 1) Secular scientist 2) Religious scientist 3) Paranormal historian 4) Secular Psychologist 5) Religious psychologist 6) Carl Jung 7) Richard Dawkins. The council should all be independent and provide their own objective analysis. Please have them create a final report and conclusions at the end". Your council can be anything, a law firm, a jury, a parent teacher association, whatever you want, and as you can see, you can throw in known people as well. This can all be done with one prompt. It's one my favorite things to do.
- svachalek 1y ago
  Wow, that's a very cool prompt, I haven't tried anything like that before.

parrit 1y ago

I want to see "Meh" vs. "Holy crap" as a benchmark in a paper published by Google. Or more likely I suspect, Andrej.

hansmayer 1y ago

Right, so... but you do realise its still just producing random output based on how you reconfigured it's weights, right? Sometimes it will happen to resonate with what you need. But it still neither thinking nor arguing with itself.

stormfather 1y ago

I made a trading bot that ingested news. The prompt to assess impact was to simulate a debate between Charlie Munger and Warren Buffet on whether to invest.

internetter 1y ago
How did it do?

Svoka 1y ago

Oh. I was just asking "Use dialectic method on your solution" in the end of the prompt... It does make it think harder.

gnarlouse 1y ago

This seems like low hanging fruit; are we seriously supposed to believe this is new and novel?

deleted 1y ago
[deleted]

ashoeafoot 1y ago

Give it reward and punishment evaluations, exploring the noise in parallel, extinction for the non rewarding answers ?

aaroninsf 1y ago

Question: has the the adversarial approach been roled into any coding copilots/assistant frameworks? Costs of various kinds aside I've wanted that from assistance's inception — with precisely the features many call out and home-roll here, difference by both model/provider, and, "role"... It seems like if you have the money/compute to burn, and can live with the reasoning wall-clock time, this has got to be the best approach for the foreseeable future, for a lot of specific requirements. (I also have wondered if this would illuminate the edges of what modern production models are capable of, "aggregating and integrating" over a variety of contributions might make more clear what the limits of their abilities are.)

kevinrineer 1y ago

This sounds like the zeitgeist is approaching genetic algorithms, which are super fun. Adversarial stuff is great.

akomtu 1y ago

The modern Alchemy: the belief that you can extract gold (intelligence) from iron (autocomplete by imitation) by mixing iron with itself.

codr7 1y ago

Better yet, let it argue with another AI, preferably using voice; instant entertainment.

ausbah 1y ago

at some point this doesn’t make LLMs feel useful. I have to wait 10x as long just so my LLM can have a somewhat higher chance of actually answer my question correctly?

robofanatic 1y ago

soon there will be AI debates. Different models debating with each other on a topic

mangoman 1y ago

a paper with a similar idea on scaling test time reasoning, this is sorta how all the thinking models work under the hood. https://arxiv.org/abs/2501.19393 https://arxiv.org/abs/2501.19393

lonetripper 1y ago

all this hard thinking yet humanity fails to come up with just one girlfriend for me

stevefan1999 1y ago

That is just reinforcement learning in disguise

animitronix 1y ago

Adversarial networks have been a thing for a while

dudeinhawaii 1y ago

I see a lot of threads pitting models against each other (or whole swarms of them) in the hope that "wisdom of crowds" will magically appear. After a stack of experiments of my own—and after watching the recent ASU/Microsoft-Research work [1].. I've landed on a simpler takeaway: An LLM is a terrible verifier of another LLM. Subbarao Kambhampati's "(How) Do LLMs Reason/Plan?" talk shows GPT-4 confidently producing provably wrong graph-coloring proofs until a symbolic SAT solver is introduced as the referee [1]. Stechly et al. quantify the problem: letting GPT-4 critique its own answers *reduces* accuracy, whereas adding an external, sound verifier boosts it by ~30 pp across planning and puzzle tasks [2]. In other words, verification is *harder* than generation for today's autoregressive models, so you need a checker that actually reasons about the world (compiler, linter, SAT solver, ground-truth dataset, etc.). Because of that asymmetry, stacking multiple LLMs rarely helps. The "LLM-Modulo" position paper argues that auto-regressive models simply can't do self-verification or long-horizon planning on their own and should instead be treated as high-recall idea generators wrapped by a single, sound verifier [3]. In my tests, replacing a five-model "debate" with one strong model + verifier gives equal or better answers with far less latency and orchestration overhead. [1] https://www.youtube.com/watch?v=0u2hdSpNS2o https://www.youtube.com/watch?v=0u2hdSpNS2o - (How) Do LLMs Reason/Plan? (talk at Microsoft Research, 11 Apr 2025) [2] https://arxiv.org/abs/2402.08115 https://arxiv.org/abs/2402.08115 [3] https://arxiv.org/abs/2402.01817 https://arxiv.org/abs/2402.01817 (related to the talk in #1)

foobiekr 1y ago
"letting GPT-4 critique its own answers reduces accuracy" This is because the output, being the input, steers directly into the tree as soon as the tree is in the context window.
hu3 1y ago
> ...so you need a checker that actually reasons about the world (compiler, linter, SAT solver, ground-truth dataset, etc.). Agree. What do you think about telling the LLM to also generate unit tests for the code it spits and then run all tests (including previous application unit tests). I think this is a way to ensure some level of grounded verification: - Does code compile? - Do unit test pass? AI can then consume test results to help fix their own mistakes.

deleted 1y ago

[deleted]

keyle 1y ago

When will we get the `4o` vs `o3` background conversation in "thinking" leading to a more correct result?

alex1138 1y ago

Every single one of my prompts would be "Are you suuuuuuure you're not hallucinating that?"

rriley 1y ago

Makes me wonder what would happen if we combine LLMs with recursive genetic algorithms. Similar to https://github.com/DivergentAI/dreamGPT https://github.com/DivergentAI/dreamGPT

mortarion 1y ago

I think Gemini 2.5 already does something similar. If you read the "thinking descriptions" that it outputs it often thinks about going back to older thoughts to verify and criticize.

schnitzelstoat 1y ago

I probably don't understand the modern, complex models. But doesn't it basically predict the next token given the context and the better models use more training data and can consider a larger context, and have more parameters to better retain information from the training data etc. But the fundamental way they operate is the same - predicting the next token given previous tokens. Where/how does reasoning happen here?

small_scombrus 1y ago
Upfront: I think AI is borderline useless for many of the tasks we give it. But: 1. Do our neurons not just react the same way every time to the same input? A brain is larger than the sum of its parts. 2. They don't reason, but you can somewhat emulate (or pretend to be) reasoning if you feed something back into itself enough times and it pinky promises reasoning is happening

faramarz 1y ago

That's cool! thanks for making it easy to fork and play with this! I've just begun my own iteration of adding Nash Equilibrium (NECoRT?) and reframing the "prompt engineering" to be a multi-agent negotiation. Curious what others think? https://github.com/faramarz/NECoRT/ https://github.com/faramarz/NECoRT/ my reasoning is that enterprise LLMs wont have any issue with the extra compute costs and would rather reconcile complex financials with various modeling optimizations. I'm very new to public repo and contributions, and hope someone can point out if I'm doing it wrong. my intention was to fork the ops codebase so I can test out my theory, and push as PR eventually