3 ms·
Artificial Intelligence Learns to Reason
- pmarreck 2y agoSummary for the time-pressed: Large Reasoning Models (LRMs) are a new type of AI system designed to perform specific types of reasoning. They generate “chains of thought” to solve problems, mimicking human reasoning processes. While LRMs have shown impressive performance on benchmarks, their robustness, trustworthiness, and interpretability remain uncertain, raising questions about their ability to handle real-world problems.
- a3w 2y agoSome experiments spoke of pattern matching instead of reasoning: if in math formulae, you altered a variable name, solving suddenly failed. Could the example of "solve count of siblings" from the paper be implicit connections in the model, instead of reasoning?
- MPSimmons 2y agoNot the topic of this piece, but every time I use reasoning models to do any serious analysis, all I can think is how in 1990, if you showed this to literally any person on the planet, they would have agreed that it was AGI, and we still argue over that for some reason.
- AndrewOMartin 2y agoBecause we know how it works, and we've seen how it fails.
- Incipient 2y agoAt least for me, my litmus test (today) would require an AGI model to respond with "huh I really don't know" instead of confidently making up garbage when it, in fact, has no clue. I don't think I'd have expected differently even in 1990.
- inetknght 2y ago> 1. Julia has two sisters. That means there are three girls in total (Julia + two more). False. Julie could be the name of a boy. > 2. Julia also has one brother, named Martin. OK > 3. Altogether, there are four siblings: three girls and one boy (Martin). Where did you identify the third girl? > 4. From Martin’s perspective, his sisters are all three of the girls (Julia and her two sisters). Here's where the answer comes from (Julia and her two sisters). More directly: "How many sisters does her brother Martin have?" > 5. Therefore, Martin has three sisters. OK. So the reasoning might have come to the "right" answer but the way it arrived at the answer was incorrect.
- aezart 2y agoJulia is referred to as "her" in the puzzle, so it's reasonable to assume she's a girl.