3 ms·

No one is bitter lesson pilled anymore. Everyone is pivoting to neurosymbolic systems. It looks like Gary Marcus was right.

nl 4mo ago
> No one is bitter lesson pilled anymore. Will the 10T parameter Mythos model be released this month or next month? They better soon because it is generally accepted that one of the reasons GPT 5.5 is better at hard tasks than Opus is because of its parameter size - and that Opus 4.8 remains competitive only be scaling test-time compute (see how many more tokens it uses than GPT 5.5) https://www.reddit.com/r/LLM/comments/1sz8bjz/parameter_estimate/ https://www.reddit.com/r/LLM/comments/1sz8bjz/parameter_esti...
- irthomasthomas 4mo ago
  Why ask me? Anyway, Mythos is not 10T. Anthropic confirmed the training run was under 10^26 flops. You can't train 10T to chincilla and stay under 10^26. Anthropic also confirmed they will not release Mythos, only a "Mythos-class" model, whatever that means.
  - nl 4mo ago
    > Anthropic confirmed the training run was under 10^26 flops. You can't train 10T to chincilla and stay under 10^26. I don't think Anthropic have said anything of the sort. Microsoft published it as 6.1*10^27 FLOPs[1] Elon has claimed the are also training a 10T model because "Some catching up to do"[2] [1] https://x.com/scaling01/status/2061897540161728791 https://x.com/scaling01/status/2061897540161728791 [2] https://x.com/elonmusk/status/2041754402239975479 https://x.com/elonmusk/status/2041754402239975479
    - irthomasthomas 4mo ago
      I must have confused mythos with opus 4.7. One of their recent model cards confirmed that training flops was under the EO reporting requirement of 10^26 flops.
wild_egg 4mo ago
How is neurosymbolic not aligned with the bitter lesson? The bitter lesson is completely agnostic to architecture.
- irthomasthomas 4mo ago
  I should have stressed the symbolic part. Everyone has pivoted to symbolic systems like claude code and codex. They would no invest so heavily in such systems if they thought llms would deliver agi soon.
  - jubilanti 4mo ago
    That's not what symbolic means.

nl 4mo ago

> No one is bitter lesson pilled anymore. Will the 10T parameter Mythos model be released this month or next month? They better soon because it is generally accepted that one of the reasons GPT 5.5 is better at hard tasks than Opus is because of its parameter size - and that Opus 4.8 remains competitive only be scaling test-time compute (see how many more tokens it uses than GPT 5.5) https://www.reddit.com/r/LLM/comments/1sz8bjz/parameter_estimate/ https://www.reddit.com/r/LLM/comments/1sz8bjz/parameter_esti...

irthomasthomas 4mo ago
Why ask me? Anyway, Mythos is not 10T. Anthropic confirmed the training run was under 10^26 flops. You can't train 10T to chincilla and stay under 10^26. Anthropic also confirmed they will not release Mythos, only a "Mythos-class" model, whatever that means.
- nl 4mo ago
  > Anthropic confirmed the training run was under 10^26 flops. You can't train 10T to chincilla and stay under 10^26. I don't think Anthropic have said anything of the sort. Microsoft published it as 6.1*10^27 FLOPs[1] Elon has claimed the are also training a 10T model because "Some catching up to do"[2] [1] https://x.com/scaling01/status/2061897540161728791 https://x.com/scaling01/status/2061897540161728791 [2] https://x.com/elonmusk/status/2041754402239975479 https://x.com/elonmusk/status/2041754402239975479
  - irthomasthomas 4mo ago
    I must have confused mythos with opus 4.7. One of their recent model cards confirmed that training flops was under the EO reporting requirement of 10^26 flops.

wild_egg 4mo ago

How is neurosymbolic not aligned with the bitter lesson? The bitter lesson is completely agnostic to architecture.

irthomasthomas 4mo ago
I should have stressed the symbolic part. Everyone has pivoted to symbolic systems like claude code and codex. They would no invest so heavily in such systems if they thought llms would deliver agi soon.
- jubilanti 4mo ago
  That's not what symbolic means.