13 ms·
Cognitive Behaviors That Enable Self-Improving Reasoners
- deleted 2y ago[deleted]
- robocat 2y agoHow much has our knowledge of AI training techniques helped to discover how to train people to think better?
- sanxiyn 2y agoSo far, I don't think we found anything interesting, yet.
- vasco 2y agoWe've had knowledge of how to eat better to not get extreme scenarios like obesity and look at the effectiveness of that. Until you have a pill that makes you think better only the motivated will do it, and in this case the motivated could already do it.
- PeterStuer 2y agoYou seem to imply the 'motivated' can not improve. I'd say the motivated often reap the rewards of innovations more so than the average, as they were pushing the boundaries in the first place. Having a dishwasher or a robot vacuum does not make me lazy. It allows me to do more productive things.
- vasco 2y agoI guess I was focusing on the most obvious cases. If a pill makes everyone skinny you'll notice much more that there's no more huge people than that skinny people's body fat went from an average of 13% to 11% or some such. I do agree the motivated have more likelihood to improve, but on a societal level I tend to think first of raising the bottom of the group. I agree with your points.
- atwrk 2y agoAs someone with an educational background I actually often ask myself the opposite: Why don't AI techniques almost never seem to use the knowledge we have about human learning to train better AI?
- HPsquared 2y agoMaybe an area worth exploring, if you think there's something to it!
- bongripper 2y ago[dead]
- tdeck 2y agoWasnt that exactly what they tried to do for the first 30 or so years, but it wasn't very fruitful?
- owenpalmer 2y ago> four key cognitive behaviors -- verification, backtracking, subgoal setting, and backward chaining -- that both expert human problem solvers and successful language models employ. As we make AI better, perhaps we'll inadvertently find ways to make HI (human intelligence) better too. I had a personal experience with this when I was studying for an exam recently. As I read over practice questions, I spoke aloud, replicating the reasoning methods/personality of Deepseek R1. By spending a lot of time reading long verbose R1 outputs, I've essentially fine-tuned my brain for reasoning tasks. I believe this method contributed to my excellent score on that exam.
- jdpage 2y agoThis is a well-known approach: verbalizing your thought process (either by speaking aloud, or by writing) is something that's long established as a good tactic for making sure that you're actually thinking through something, rather than glossing over it. Ironically, I've seen people bemoaning that use of AI will rob people of that. I agree that there's potential here, though, and do genuinely hope that we find ways to make human intelligence better as we're going about AI research. Even pessimistically, I think we'll at least surface approaches that people use without thinking about, which is on its own a good thing, because once you know you're doing something, it becomes a lot easier to train yourself to do it better.
- ilrwbwrkhv 2y agoThis is one of the secrets of the top British universities. They do a lot of debating in small groups. Even their papers are read out loud
- crooked-v 2y ago> Ironically, I've seen people bemoaning that use of AI will rob people of that. There's that quote from Socrates, recorded by Plato: > For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory. Their trust in writing, produced by external characters which are no part of themselves, will discourage the use of their own memory within them.
- idiotsecant 2y agoI sometimes see these reddit threads of people talking about the experience of having an internal monologue. I have no such monologue, at least not one that is accessible to the part of my mind that calls itself 'me', but I have often wondered if that monologue is something like a 'chain of thought'. I feel like maybe without access to that 'idea feed' maybe my planning and executive functioning is less effective than some other people. I do find myself quite more effective with those sort of tasks when I do a little 'chain of thought' notepad. I also suspect I spend less time ruminating and second-guessing myself and other anxious behaviours that I imagine would come with having someone talking in your ear all day, but that's probably off topic.
- kla-s 2y agoGenuine question, how does multi step reasoning work for you then? Like eg if you have some math problem that's trivially to solve individually but needs multiple steps, lets say 16 * 3 + 5? How does 16 * 3 = 48 land in some 'register' of your brain (short term memory), so that you can then add 5 to get to 53? Maybe 16 * 3 + 5 is to easy for you and you'll just 'see' it but the question still stands, just choose a more complex problem. Isn't the same meta process at play when thinking about more fuzzy topics?
- cgriswald 2y agoI believe I have an internal narrator but I’m not certain exactly what others mean by that so I don’t know for sure. However the way I think about math is different than the way I plan my day or other things. In my case, it is very much like I have registers that would hold the result of 16 x 3 in it so I can add the 5 to it later. I have a certain number of registers and with effort like repeating what Ive already solved I could temporarily create more. It also feels somewhat physical, as if the register is an actual box or has a “location” or like I’ve put the answer down on the desk like a part of something I’m building. Perhaps not coincidentally I am one of the many people who have a “calendar shape” for the months.
- crooked-v 2y agoNot that poster, but for me it's directly manipulating numbers (for example, "16×3 + 5" turns into "10×3 + 6×3 + 5" into "30 + 18 + 5" into "30 + 10 + 8 + 5" into "40 + 13" into "53"). There's no language involved, though in some cases I might use some spatial reasoning by doing something like associating given chunks of an equation with different fingers.
- nickpsecurity 2y ago"models primed with incorrect solutions containing proper reasoning patterns achieve comparable performance to those trained on correct solutions" One of the parts most worth a replication study.
- meindnoch 2y agoAt this point I can't tell from the title whether it's a self-help psychology fad or an LLM paper.
- miksik 2y ago> four key cognitive behaviors -- verification, backtracking, subgoal setting, and backward chaining -- that both expert human problem solvers and successful language models employ. Based on what have they claimed that such methods are used by expert human problem solvers?
- th0ma5 2y agoOnce you suspend disbelief of AI all of this pseudoscience becomes just as plausible.
- miksik 2y agoI'm actually interested whether there is some larger explanation on these study methods. Maybe there's something worth integrating with myself and get more efficient at learning.
- th0ma5 2y agoI think everyone wants that but there really is no silver Bullet in explaining or understanding how to make this technology more effective or more accurate. The companies themselves can radically change the models, behavior intentionally and unintentionally. There is no guarantee that any insights that you gain today will be applicable in the future.
- glass_door 2y agoDoes this also mean giving better system prompts that encourage this behaviour also substantially help?
- astrange 2y agoIn my experience models aren't very good at following such prompts. Smart "non-reasoning" models like Claude 3.5 could, but would generate so much text when thinking they ran out of context.
- spwa4 2y agoTrue, but a problem is that self-improving AI leads to a somewhat troubling mode of thinking. AIs switch to an internal babbling type language that makes no sense but clearly still conveys meaning to the AIs, then think in that language (if it's a language, though not sure what else it could be) and then produce correct results. Worse, when you use multiple agents to get AI LLMs talking to one another, all AI agents switch to this internal language and they make progress despite no human understanding what hell is happening. This seems very bad. Illustration: > How many r in strawberry? I'm asked how many r in strawberry. I can just spell the word and a;dklsjaw; a;ewjraqwpeouypaads;lq qepwiouryaqeopw qewrpoiuyoiauysdqw145124rfa.nkjlwh ;45a8345a894ya4a q4p58q45jaq;lkjas;dlfkja;j <answer>There are 3 (three) r's in strawberry</answer>
- hnuser123456 2y agoThey are not going to invent a new language by themselves, they by definition can't even "think" in terms of languages they haven't seen. It does not occur to them that the language they use may be suboptimal. And surely, any better ways of thinking can still be described in English. It seems more likely there will be a gradual transition from us teaching LLMs how to reason, to LLMs being able to actually gobble and process enough data to learn more effective ways to reason, which it can then "teach" us. But that's just the LLM reflecting the way it was trained and aligned.
- theptip 2y agoI’ve heard this described as talking in “Neuralese”. It seems plausible that this will be the most dense language for model-internal dialog (or presumably inter-LLM dialog assuming they share the same weights). You will penalize this inasmuch as your alignment strategy depends on Deliberative Alignment. But at some point I assume that will come with a real capability cost as Neuralese can be more conceptually dense.
- kittikitti 2y ago``think'' In the abstract they use different characters for double quotes here.
- juped 2y agoThat's LaTeX syntax for opening and closing quotes. Though their rendered paper doesn't render them that way for some reason!