3 ms·
500 billion word pieces (it splits words into few-character long pieces), which comes down to 100-200B words, scraped off the internet. It used mostly CommonCra
by visarga 6y ago
500 billion word pieces (it splits words into few-character long pieces), which comes down to 100-200B words, scraped off the internet. It used mostly CommonCrawl as training data.
It's like the proverbial witch's pot where they put everything in and out comes the magic.
- YeGoblynQueenne 6y agoYes, that's an apt metaphor. Magic, indeed.