3 ms·
Supposedly 10T scale. Literally the next big thing. A bit like what OpenAI tried with GPT-4.5 - but Anthropic actually made it work with MoE, reasoning, tool us
by ACCount37 5mo ago
Supposedly 10T scale. Literally the next big thing. A bit like what OpenAI tried with GPT-4.5 - but Anthropic actually made it work with MoE, reasoning, tool use, RLVR, etc.
It matters because the "g factor" of today's LLMs is at least in part a function of raw scale. Larger models are just smarter - assuming you can handle the training and inference at this increased scale.
- AnimalMuppet 5mo agoSo, realistically, how much further can this go? How many more orders of magnitude?
- ACCount37 5mo ago100T definitely. 1Q maybe. Beyond that, we need new architectures or new types of inference hardware - maybe both. By some estimates, 100T is about what the human brain has when used to its fullest.