3 ms·
Where are any LLMs going to get data from as they become more ubiquitous and humans produce less publicly accessible original and thoughtful content? The whole
by berkle4455 4y ago
Where are any LLMs going to get data from as they become
more ubiquitous and humans produce less publicly accessible original and thoughtful content?
The whole thing is a plateaued feedback loop.
- TillE 4y agoIt'd be cool to have an LLM that's trained almost exclusively on books from good publishers, and other select sources. Working out licensing deals would be a challenge, of course.
- whimsicalism 4y agoCorpora is likely too small. It would just be an "LM"
- whimsicalism 4y agoProbably from multiple modalities as well as extending the sequence lookback length further and further. They have low perplexity now, but the perplexity possible when predicting the next word on page 365 of a book where you can attend over the last 364 pages will allow even more complexity to emerge.