43 ms·
There is a recent Yannic Kilcher interview about LAION. > LAION-5B: 5 billion image-text-pairs dataset (with the authors) https://www.youtube.com/watch?v=AIOE
by visarga 5y ago
There is a recent Yannic Kilcher interview about LAION.
> LAION-5B: 5 billion image-text-pairs dataset (with the authors)
https://www.youtube.com/watch?v=AIOE1l1W0Tw https://www.youtube.com/watch?v=AIOE1l1W0Tw
A nice recent result (DeepMind) is that you can either make the dataset 4x larger or the network 4x larger to get the same result. So a large dataset could create a more efficient/smaller model and in turn it could be easier to distribute and use.
https://www.deepmind.com/publications/an-empirical-analysis-of-compute-optimal-large-language-model-training https://www.deepmind.com/publications/an-empirical-analysis-...