3 ms·
To be fair, o1 is a major breakthrough in the field. If other AI labs can't crack scaling useful inference compute, OpenAI will maintain a big lead.
by lordswork 2y ago
To be fair, o1 is a major breakthrough in the field. If other AI labs can't crack scaling useful inference compute, OpenAI will maintain a big lead.
- lolinder 2y agoIsn't o1 just applying last year's Tree of Thoughts paper in production? Is there any reason to believe that the other companies will struggle to implement their own? https://github.com/princeton-nlp/tree-of-thought-llm https://github.com/princeton-nlp/tree-of-thought-llm
- impossiblefork 2y agoI don't think it's tree of thoughts at all. I think it's as they say: reinforcement learning applied to cause it to generate a relatively long 'reasoning trace' of some kind from which the answer is obtained through summarisation. I think it's likely a cleverly simplified version of QuietSTaR, with no thought tokens, just one big generation to which the RL is applied. The way I believe it's trained in practice is as follows: they have a bunch of examples, some at the edge of GPT-4s ability to answer, some beyond it, some that GPT-4 can answer if you're lucky with the randomness. Then they give it one of these prompts, generate a fairly long text, maybe 3x the length of the answer, and summarize that to produce the final answer. Then they use REINFORCE to reward the generated texts that increase the probability of the summary being correct.
- WJW 2y agoNot be be nitpicky, but being the first to deploy recent academic research papers to production should count as a breakthrough IMHO.
- lordswork 2y agoThere seems to be several components involved: tree of thoughts, MCTS, RL, high quality chain of thought data, possibly multiple models.
- njtransit 2y agoo1 seems like it’s basically 4o with some chain of thought bolted on. Personally, I don’t consider chain of thought a breakthrough, let alone a major one.
- lordswork 2y agoIt's much more than CoT. I suspect it will take other labs some time to replicate.
- petesergeant 2y ago> o1 is a major breakthrough Is it? I feel like if you don't care about the cost it's pretty easily replicable on any other LLM, just with a lang-chain sort of approach
- lordswork 2y agoThen why hasn't it been done?
- tim333 2y agoSomeone gave the various models an IQ test and the previous ones scored 80-90 so a bit dim compared to humans, and o1 got 120, quite bright for a human. https://www.reddit.com/r/ClaudeAI/comments/1fhwfyl/openai_1o_gets_120_iq_on_norway_mensa_iq_test https://www.reddit.com/r/ClaudeAI/comments/1fhwfyl/openai_1o... this may have consequences for how useful it is.
- aunty_helen 2y agoCoT can be _easily_ achieved using langgraph in a similar manner. There’s no “scaling of inference” it’s just prompting, all the way down.
- lordswork 2y agoIt's not just CoT