4 ms·
The hardware difference explains runtime performance differences, not task performance. Speculation is that the frontier models are all below 200B parameters b
by vlovich123 8mo ago
The hardware difference explains runtime performance differences, not task performance.
Speculation is that the frontier models are all below 200B parameters but a 2x size difference wouldn’t fully explain task performance differences
- ses1984 8mo agoWho would have thought ai labs with billions upon billions of r&d budget would have better models than a free alternative.
- deleted 8mo ago[deleted]
- NamlchakKhandro 8mo ago> The hardware difference explains runtime performance differences, not task performance. Yes it does.
- MrDrMcCoy 8mo agoCare to elaborate?
- nl 8mo ago> Speculation is that the frontier models are all below 200B parameters Some versions of some the models are around that size, which you might hit for example with the ChatGPT auto-router. But the frontier models are all over 1T parameters. Source: watch interview with people who have left one of the big three labs and now work at the Chinese labs and are talking about how to train 1T+ models.
- 827a 8mo agoHe's running a 35B parameter model. Frontier models are well over a trillion parameters at this point. Parameters = smarts. There are 1T+ open source models (e.g. GLM5), and they're actually getting to the point of being comparable with the closed source models; but you cannot remotely run them on any hardware available to us. Core speed/count and memory bandwidth determines your performance. Memory size determines your model size which determines your smarts. Broadly speaking.
- BoredomIsFun 8mo ago> There are 1T+ open source models (e.g. GLM5), GLM-5 is ~750B model.
- regularfry 8mo agoThe architecture is also important: there's a trade-off for MoE. There used to be a rough rule of thumb that a 35bxa3b model would be equivalent in smarts to an 11b dense model, give or take, but that's not been accurate for a while.
- BoredomIsFun 8mo agoCertainly not Opus. That beast feels very heavy - the coherence of longer form prose is usually a good marker, and it is able to spit 4000 words coherent short stories from a single shot.