3 ms·
test-time training/RL is definitely the right approach for math AI in the future. It is probably one of only a few ways to spend an obscene amounts of compute a
by Davidzheng 2y ago
test-time training/RL is definitely the right approach for math AI in the future. It is probably one of only a few ways to spend an obscene amounts of compute at a given problem (think 10^5 gpus for a few days) and has hopes of making progress when test-time inference scaling may not at first (think if you try to do MCTS on a go position with a bad value/policy net). Alphaproof already did this but nice to see it done again--good results!
- Davidzheng 2y agoactually I think the interesting thing is to see how much of the boosted performance can be distilled back into the LLM at small sizes. Then you can emulate really how alphazero works b/c you have a policy improver (test time rl with similar problems). and we get to see just how strong a small net can be theoretically (say 32B or something)
- eru 2y agoThis might be trivial, or deep, depending: We have done a lot of that improving historically by publishing research and textbooks. I can solve (some) problems today in minutes that would have stumped Isaac Newton for a life time (or at least a few weeks). Of course, you are hinting at a more general distillation, I suspect.
- InvidFlower 2y agoWe've already seen Qwen's new QWQ 32B (not distilled) model doing impressive things on benchmarks. It'll definitely be interesting to see how just good small models can get. When combined with rag and large context window for expanded knowledge, might be able to get pretty far.
- brookst 2y agoA little like the T2 read only versus learning mode switch: operating on intrinsic knowledge versus able to reason improvements.