Arguably as soon as you go beyond English, this term starts making more sense. For many languages - especially smaller ones - having a good universal model pret

Arguably as soon as you go beyond English, this term starts making more sense. For many languages - especially smaller ones - having a good universal model pretrained on large quantities of good data is literally quite "foundational"; it's very useful as a foundation to build other things on top of it, and it's sufficiently resource-intensive (and also getting appropriate data is labor-intensive and sometimes legally tricky) to make sense to make such models rarely, not forcing everyone to recreate one from scratch. Having a GPT-3 clone for a new language isn't really novel, but it's foundational work that helps a lot of other research.