2 ms·
There's a few things at play here. LLMs - OpenAI, Google Brain, Meta FAIR, HuggingFace and others are now routinely training models with the entire corpus of t
by nojvek 4y ago
There's a few things at play here.
LLMs - OpenAI, Google Brain, Meta FAIR, HuggingFace and others are now routinely training models with the entire corpus of the internet in a few months. The models are getting larger and more efficient.
Diffusion models - MidJourney, StableDiffusion, Dall-E and it's control net cousins - Trained on terabytes of images, almost entire corpus of internet.
Same with voice and other multimodal models.
The transformer algorithm is magical but we're just getting started.
There are now multiple competing players who can drop millions of dollars on compute and have access to internet sized datasets.
The compute, the datasets, the algorithms, the human reinforcement loops, all are getting better week over week. Millions of users are interacting with these systems daily. A large subset even paying for it.
There is the gold rush.