4 ms·
In what sense are the models flatlining?
by squidbeak 5mo ago
In what sense are the models flatlining?
- nicoburns 5mo agoIn the sense that the incremental improvements in capabilities that we've been seeing in recent models seem to taking exponentially growing amounts of compute to achieve.
- nl 5mo agoBut they don't? Mythos is a 10T model. Opus is a 5T model. That's not an exponentially growing amount of compute but it is achieving exponential improvements (eg from Mozilla: https://blog.mozilla.org/en/privacy-security/ai-security-zero-day-vulnerabilities/ https://blog.mozilla.org/en/privacy-security/ai-security-zer... )
- le-mark 5mo ago> but it is achieving exponential improvements “Exponential” used here is pure hyperbole. Can you justify it?
- coldtea 5mo agoCompute doesn't necessarily linerarly follow parameters. And with how many active parameters Mythos vs Opus gets its effectivenes from? Is it 1x or 2x? We don't know. We don't even know the parameters (it's more of rumor than confirmed 10T iirc). But even more so, who said the improvements are "exponential"? Mozilla's single metric, that doesn't even prove anything of the sort?
- nozzlegear 5mo ago> Mythos Ah yes, the marketing model that's ostensibly so powerful us mere mortals aren't allowed to use it. It's certainly led to exponential hype and speculation.
- _heimdall 5mo agoWasn't 4.6 Sonnet a 1T model? Parameters and compute are quite the same thing, but going from 1T to 5T to 10T is quite a ramp up.
- minitech 5mo agoI know parameters don’t translate directly like that (and that linear and exponential aren’t the only types of growth) but a doubling as a go-to example of “not exponential growth” is pretty funny.
- crthpl 5mo agowhere the heck did you get those parameter numbers from?
- nl 5mo agoSonnet and Opus are from Elon Musk (given the people he's hired it seems likely it is approximately true). Mythos is quite widely spoken about.