4 ms·
> trained the model on synthetic data You get knowledge collapse [1] this way. [1] https://arxiv.org/abs/2404.03502 https://arxiv.org/abs/2404.03502
by thesz 2y ago
> trained the model on synthetic data
You get knowledge collapse [1] this way.
[1] https://arxiv.org/abs/2404.03502 https://arxiv.org/abs/2404.03502
- ben_w 2y agoYou can get knowledge collapse. But even though there are counter-examples (e.g. learning Go from self-play based only on the rules), IMO — based on how often people were already doing this with buggy human-written software[0][1] — most people don't think about this in the right way and will therefore treat these things as magical oracles when they shouldn't. No silver bullets. [0] https://en.wikipedia.org/wiki/British_Post_Office_scandal https://en.wikipedia.org/wiki/British_Post_Office_scandal [1] https://en.wikipedia.org/wiki/Computer_says_no https://en.wikipedia.org/wiki/Computer_says_no