3 ms·
They are highly effective pattern matchers. You change the pattern, it won't work. I don't remember who, but most likely @tszzl (roon), commented on x that they
by ankit219 2y ago
They are highly effective pattern matchers. You change the pattern, it won't work. I don't remember who, but most likely @tszzl (roon), commented on x that they still trained the traditional way, and there is no test time compute (TTC) or Montecarlo Tree search (like Alpha Go) in o1 or o3. If that is true, then it's still predicting the next word based on it's training data. Likely to follow the most probable path - which comes directly from the training itself - even for the slight variations. Encouragingly, if TTC hasnt been explored, there is a long runway for performance improvements.
The other reason this seems hard to guess is because we don't know how much of what we are asking is in the training data. It would perform on some tasks, while fail at others even though those are similar.
- x_may 2y agoI believe they are using scalable TTC. The o3 announcement released accuracy numbers for high and low compute usage, which I feel would be hard to do in the same model without TTC. I also believe that the 200$ subscription they offer is just them allowing the TTC to go for longer before forcing it to answer. If what you say is true, though, I agree that there is a huge headroom for TTC to improve results if the huggingface experiments on 1/3B models are anything to go off.
- ankit219 2y agoThe other comment posted YT videos where Open AI researchers are talking about TTC. So, I am wrong. That $200 subscription is just because the number of tokens generated are huge when CoT is involved. Usually inference output is capped at 2000-4000 tokens (max of ~8192) or so, but they cannot do it with o1 and all the thinking tokens involved. This is true with all the approaches - next token prediction, TTC with beam/lookahead search, or MCTS + TTC. If you specify the output token range as high and induce a model to think before it answers, you will get better results on smaller/local models too. > huge headroom for TTC to improve results ...1B/3B models Absolutely. How this is productized remains to be seen. I have high hopes with MCTS and Iterative Preference Learning, but it is harder to implement. Not sure if Open AI has done that. Though Deepmind's results are unbelievably good [1]. [1]:https://arxiv.org/pdf/2405.00451v2 https://arxiv.org/pdf/2405.00451v2
- whimsicalism 2y agottc is an incredibly broad term and it is broadening as the hype spreads. people are now calling CoT “TTC” because they are spending compute on reasoning tokens before answering
- HarHarVeryFunny 2y agoYes, and HuggingFace have published this outlining some of the potential ways to use TTC, including but not limited to tree search, showing TTC performance gains from LLama. https://huggingface.co/spaces/HuggingFaceH4/blogpost-scaling-test-time-compute https://huggingface.co/spaces/HuggingFaceH4/blogpost-scaling...
- e1g 2y agoI recently watched two interviews with OpenAI researchers where they describe that the breakthrough of o-series (unlike GPT series) is to focus on test time compute as they are designed to “think” more specifically to avoid pattern matching. Noam Brown https://youtu.be/OoL8K_AFqkw?si=ocIS0YDXLvaX9Xb6&t=195 https://youtu.be/OoL8K_AFqkw?si=ocIS0YDXLvaX9Xb6&t=195 and Mark Chen https://youtu.be/kO192K7_FaQ?si=moWiwYChj65osLGy https://youtu.be/kO192K7_FaQ?si=moWiwYChj65osLGy
- deleted 2y ago[deleted]
- ankit219 2y agoThank you, this is helpful. The post on X was seemingly wrong.
- mmmore 2y agoThe comment was likely that there's no explicit search. In o1, the model has learned how to search using its context. Presumably they do this by RLing over long reasoning strings/internal monologues.
- HarHarVeryFunny 2y agoOpenAI have openly stated that o1 & o3 are using test time compute, and released a log scale graph indicating linear performance gains for exponential compute usage. https://openai.com/index/learning-to-reason-with-llms/ https://openai.com/index/learning-to-reason-with-llms/ They only confirm that the model/system is doing chain of thought, but the exponential factor and origin of reasoning gains likely comes from TREE of thoughts (number of branches/compute goes up exponentially with depth), essentially doing tree search over different reasoning chains. I assume roon's identity is well known inside OpenAI (he's an employee), so I wouldn't expect him to be leaking implementation details on twitter.