3 ms·
> LLMs have an additional problem related to training data. We are already throwing all the data we can get our hands on at them. I think it's plausible we'll
by jebarker 2y ago
> LLMs have an additional problem related to training data. We are already throwing all the data we can get our hands on at them.
I think it's plausible we'll see a breakthrough in data efficiency that helps here. Humans are an existence proof that language is learnable through much less data and, in theory, facts should only need to be seen once. LLMs in their current form seem very data inefficient.
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- dartos 2y agoYou can’t predict breakthroughs. We probably will see a breakthrough, but there’s no more reason to believe it’ll happen tomorrow than 100 years from now.
- jebarker 2y agoI didn't make a prediction, let alone a timeline - I just said it's plausible. I suppose I did say "we'll see" so I'm implicitly predicting within a few decades.