3 ms·
But why does GPT3 need to be so large?
by d13 4y ago
But why does GPT3 need to be so large?
- yazaddaruvala 4y agoDisclaimer: I’m not well versed in this field, but my basic understanding is that Diffusion can and is “fuzzy”. An image without “crisp” pixels that create lines is acceptable. A basketball with an extra/missing blue pixel is acceptable. With words, they either haven’t invented Diffusion yet, or the nature of the problem is too hard for Diffusion. With text you can’t have the model Diffuse to “teli ne abouf a fdog?” “The dod is graen and has stix leg5”. It’s just too obviously wrong. Meanwhile, the “fuzziness” allows Diffusion models to be smaller, compared with models that need precision.