3 ms·
Neat to see more models getting closer, thought it appears only one so far has exceeded GPT-3's 175B parameters. That said, what I'm really curious is how thos
by warning26 5y ago
Neat to see more models getting closer, thought it appears only one so far has exceeded GPT-3's 175B parameters.
That said, what I'm really curious is how those other models stack up against GPT-3 in terms of performance -- does anyone know of any comparisons?
- machiaweliczny 5y agoThere's Wu Dao 2.0 and Google has 2 models with 1T+ params.
- atty 5y agoFor clarity, i believe these are all mixture of expert models, where each input only sparsely activates some subset subset of the full model. This is why they were able to make such a big jump over the “dense” GPT3. Not really an apples-to-apples comparison.
- pyb 5y ago+1, does the new generation match or exceed GPT-3 in terms of relevance ? Is there a way for a non-AI-researcher to understand how the benchmarks measure this ? Bigger does not mean better.
- sillysaurusx 5y agoI’m surprised that no one has answered for three hours! The answer is at https://github.com/kingoflolz/mesh-transformer-jax https://github.com/kingoflolz/mesh-transformer-jax It has detailed comparisons and a full breakdown of the performance, courtesy of Eleuther.
- 6gvONxR4sf7o 5y agoI was so frustrated when that was first announced because it didn't include those metrics, and everyone ate it up like the models were equivalent.
- 6gvONxR4sf7o 5y agoWhenever I've seen language modeling metrics, GPT-3's largest model has been at the top. If you see a writeup that doesn't include accuracy-type metrics, you're reading a sales pitch, not an honest comparison.