4 ms·
Wait until people start training AI on AI output because (a) that's the only source of information left with a large enough corpus size and (b) all other source
by unxdfa 4y ago
Wait until people start training AI on AI output because (a) that's the only source of information left with a large enough corpus size and (b) all other sources are polluted.
There's no way of asserting accuracy when we reach that point because the very nature of information and scientific thinking will be corrupted.
- valine 4y agoThere is evidence to suggest that LLMs have recently passed a threshold where their performance can be improved by training on their own output. https://arxiv.org/abs/2210.11610 https://arxiv.org/abs/2210.11610 I expect your concerns are likely unwarranted.
- danaris 4y agoHow does this do anything but validate their concerns? They're not saying "if we train LLMs on their own output, they'll get slower or worse", they're saying "if we train LLMs on their own output, it gets even harder to actually derive sourcing for what they produce."