3 ms·
o1, which you seem to be referring to, is at best slightly better at specific tasks than Sonnet 3.5 while being completely unviable economically for production
by maeil 2y ago
o1, which you seem to be referring to, is at best slightly better at specific tasks than Sonnet 3.5 while being completely unviable economically for production use, despite the amount of subsidizing OpenAI has been doing to make them lose a cool $5 billion this year. GPT has barely improved over the last 1.5 years [1].
[1] https://aider.chat/assets/models-over-time.svg https://aider.chat/assets/models-over-time.svg
- mrbungie 2y agoYeah, I know. I didn't say it was incredibly better in every bench, but there are reasoning and planning benchmarks it does much better (not just slightly) than Sonnet 3.5 and the rest right now, and that still that makes their offering SOTA vs the others, even if its not in terms of cost. I put sources in another sibling comment, and as I said in both comments, I think that gap will close. I don't expect OAI to keep their moat much longer, especially with the people they've lost. PS: I say all of this as a constant critic of OAI: both how they behave and their offerings. But credit where credit is due, I don't see out-of-the-box alternatives to OAI's rlhf'd CoT at any price point right now.