4 ms·
>All without having been explicitly trained in those tasks. Can you elaborate on what kind of training data was used here? I'm curious.
by BenoitEssiambre 6y ago
>All without having been explicitly trained in those tasks.
Can you elaborate on what kind of training data was used here? I'm curious.
- visarga 6y ago500 billion word pieces (it splits words into few-character long pieces), which comes down to 100-200B words, scraped off the internet. It used mostly CommonCrawl as training data. It's like the proverbial witch's pot where they put everything in and out comes the magic.
- YeGoblynQueenne 6y agoYes, that's an apt metaphor. Magic, indeed.