8 ms·
Exactly! > It uses two interdependent recurrent modules: a *high-level module* for abstract, slow planning and a *low-level module* for rapid, detailed computa
by diwank 1y ago
Exactly!
> It uses two interdependent recurrent modules: a *high-level module* for abstract, slow planning and a *low-level module* for rapid, detailed computations. This structure enables HRM to achieve significant computational depth while maintaining training stability and efficiency, even with minimal parameters (27 million) and small datasets (~1,000 examples).
> HRM outperforms state-of-the-art CoT models on challenging benchmarks like Sudoku-Extreme, Maze-Hard, and the Abstraction and Reasoning Corpus (ARC-AGI), where CoT methods fail entirely. For instance, it solves 96% of Sudoku puzzles and achieves 40.3% accuracy on ARC-AGI-2, surpassing larger models like Claude 3.7 and DeepSeek R1.
Erm what? How? Needs a computer and sitting down.
- mkagenius 1y agoIs it talking about fine tuning existing models with 1000 examples to beat them in those tasks?
- cs702 1y agoYeah, that was pretty much my reaction. I will need time on a computer too. The repo is at https://github.com/sapientinc/HRM https://github.com/sapientinc/HRM . I love it when authors publish working code. It's usually a good sign. If the code does what the authors claim, no one can argue with it!
- diwank 1y agoSame! Guan’s work on sample packing during finetuning has become a staple. His openchat code is also super simple and easy to understand.