3 ms·
Alteratively if they trained the model on synthetic data, filtered to avoid duplication, then no copyrighted material would be seen by the model. For example tu
by visarga 2y ago
Alteratively if they trained the model on synthetic data, filtered to avoid duplication, then no copyrighted material would be seen by the model. For example turn an article into QA pairs, or summarize across multiple sources of text.
- thesz 2y ago> trained the model on synthetic data You get knowledge collapse [1] this way. [1] https://arxiv.org/abs/2404.03502 https://arxiv.org/abs/2404.03502
- ben_w 2y agoYou can get knowledge collapse. But even though there are counter-examples (e.g. learning Go from self-play based only on the rules), IMO — based on how often people were already doing this with buggy human-written software[0][1] — most people don't think about this in the right way and will therefore treat these things as magical oracles when they shouldn't. No silver bullets. [0] https://en.wikipedia.org/wiki/British_Post_Office_scandal https://en.wikipedia.org/wiki/British_Post_Office_scandal [1] https://en.wikipedia.org/wiki/Computer_says_no https://en.wikipedia.org/wiki/Computer_says_no