3 ms·
Diminishing returns? Have you read the Gato, Palm, Stable Diffusion, etc. papers? Progress is racing ahead. Nothing is stalling... the only thing stopping progr
by adamsmith143 4y ago
Diminishing returns? Have you read the Gato, Palm, Stable Diffusion, etc. papers? Progress is racing ahead. Nothing is stalling... the only thing stopping progress from accelerating even faster is data.
- random314 4y agoHe is talking about Deep learning at FB
- mirker 4y agoMany of these scaling patterns are logarithmic with respect to data size. You can only double the dataset size so many times that it’s really not clear the scaling will continue.
- lostmsu 4y agoLow data modes are also progressing quite fast. There is Dreamer and more recent papers based on RL in learned world models.