6 ms·
Sorry, I think you've misunderstood, they're saying that's exactly the point. Those weird metrics are what they're debunking, not supporting. Per my snippet fro
by ianbutler 3y ago
Sorry, I think you've misunderstood, they're saying that's exactly the point. Those weird metrics are what they're debunking, not supporting. Per my snippet from the abstract in my last comment.
Their point was a lot of papers use those weird metrics and it contributes to the appearance of emergent ability, when in reality its just the bad metrics.
Nothing you've said so far disagrees with either my understanding or the conclusion of the paper I linked.
- sigmoid10 3y agoI think you misunderstood. The authors created the very issue they are "debunking." They took GPT3, slapped on some random metrics and showed that these metrics don't show scaling behaviour correctly, while the original publication of GPT3 actually did it correctly in the first place.
- deleted 3y ago[deleted]