3 ms·
In the sense that the incremental improvements in capabilities that we've been seeing in recent models seem to taking exponentially growing amounts of compute t
by nicoburns 6mo ago
In the sense that the incremental improvements in capabilities that we've been seeing in recent models seem to taking exponentially growing amounts of compute to achieve.
- nl 6mo agoBut they don't? Mythos is a 10T model. Opus is a 5T model. That's not an exponentially growing amount of compute but it is achieving exponential improvements (eg from Mozilla: https://blog.mozilla.org/en/privacy-security/ai-security-zero-day-vulnerabilities/ https://blog.mozilla.org/en/privacy-security/ai-security-zer... )
- le-mark 6mo ago> but it is achieving exponential improvements “Exponential” used here is pure hyperbole. Can you justify it?
- coldtea 6mo agoCompute doesn't necessarily linerarly follow parameters. And with how many active parameters Mythos vs Opus gets its effectivenes from? Is it 1x or 2x? We don't know. We don't even know the parameters (it's more of rumor than confirmed 10T iirc). But even more so, who said the improvements are "exponential"? Mozilla's single metric, that doesn't even prove anything of the sort?
- nozzlegear 6mo ago> Mythos Ah yes, the marketing model that's ostensibly so powerful us mere mortals aren't allowed to use it. It's certainly led to exponential hype and speculation.
- _heimdall 6mo agoWasn't 4.6 Sonnet a 1T model? Parameters and compute are quite the same thing, but going from 1T to 5T to 10T is quite a ramp up.
- minitech 6mo agoI know parameters don’t translate directly like that (and that linear and exponential aren’t the only types of growth) but a doubling as a go-to example of “not exponential growth” is pretty funny.
- crthpl 6mo agowhere the heck did you get those parameter numbers from?
- nl 5mo agoSonnet and Opus are from Elon Musk (given the people he's hired it seems likely it is approximately true). Mythos is quite widely spoken about.