3 ms·
The open source/weights models so far have proved that openAI doesn't have some special magic sauce. I m confident we ll soon have a model from Meta or others t
by seydor 2y ago
The open source/weights models so far have proved that openAI doesn't have some special magic sauce. I m confident we ll soon have a model from Meta or others that s close to this level of reasoning. [Also consider that some of their top researchers have departed]
On a cursory look, it looks like the chain of thought is a long series of chains of thought balanced on each step, with a small backtracking added whenever a negative result occurs, sort of like solving a maze.
- zamalek 2y agoI suspect that the largest limiting factor for a competing model will be the dataset. Unless they somehow used GPT4 to generate the dataset somehow, this is an extremely novel dataset to have to build.
- j_maffe 2y agoThey almost definitely used existing models for generating it. The human feedback part, however, is the expensive aspect.
- tarruda 2y agoI would love to see Meta releasing CoT specialized model as a LoRa we can apply to existing 3.1 models
- niutech 2y agoIsn't it what Reflection 70b (https://news.ycombinator.com/item?id=41459781 https://news.ycombinator.com/item?id=41459781) does on top of Llama 3.1?
- valine 2y agoReflection 70B is a scam. The creator was just routing requests to Claude. https://old.reddit.com/r/LocalLLaMA/comments/1fc98fu/confirmed_reflection_70bs_official_api_is_sonnet/ https://old.reddit.com/r/LocalLLaMA/comments/1fc98fu/confirm...