6 ms·
I think it's a little early (even in these AI times) to call HRM a counterexample of the bitter lesson. I think it's quite a bit more likely for HRM to scale e
by benlivengood 1y ago
I think it's a little early (even in these AI times) to call HRM a counterexample of the bitter lesson.
I think it's quite a bit more likely for HRM to scale embarrassingly far and outstrip the tons of RLHF and distillation that's been invested in for transformers, more of a bitter lesson 2.0 than anything else.
- turnsout 1y agoI didn't know about HRM until this article, but it looks incredibly impressive, even in its current state. A 27M parameter model that can be trained to perform difficult domain-specific tasks with better results than flagship reasoning models? That is compelling.