From [1]: "What is a Foundation Model" In recent years, a new successful paradigm for building AI systems has emerged: Train one model on a huge amount of data

From [1]: "What is a Foundation Model" In recent years, a new successful paradigm for building AI systems has emerged: Train one model on a huge amount of data and adapt it to many applications. We call such a model a foundation model. From their big paper [2] "A foundation model is any model that is trained on broad data (generally using self-supervision at scale) that can be adapted (e.g., fine-tuned) to a wide range of downstream tasks; current examples include BERT [Devlin et al.2019], GPT-3 [Brown et al. 2020], and CLIP [Radford et al. 2021]." [1] https://crfm.stanford.edu/ https://crfm.stanford.edu/ [2] "On the Opportunities and Risks of Foundation Models", Bommasani/Hudson/et al