3 ms·
> Note in particular that "like a function that can do the same computation with fewer inputs" maps very well to GPT-3 - it can complete many interesting tasks
by abernard1 6y ago
> Note in particular that "like a function that can do the same computation with fewer inputs" maps very well to GPT-3 - it can complete many interesting tasks by just having a few samples provided to it, instead of having to fine-tune it with more training.
The reason it doesn't need more training is because it's already trained itself with millions of lifetimes of human data and encoded that in the parameters!
Humans aren't born trained with data. The fact that we're throwing more and more data at this problem is crazy. The compression ratio of GPT-3 is worse than GPT-2.
- MauranKilom 6y ago> The reason it doesn't need more training is because it's already trained itself with millions of lifetimes of human data and encoded that in the parameters! You know what else is trained by the experiences of thousands of individual (and billions of collective) human lifetimes of data? And several trillions of non-human ones? > Humans aren't born trained with data. That's either very wrong or about to evolve into a no true scotsman regarding what counts as data.