2 ms·
You can get knowledge collapse. But even though there are counter-examples (e.g. learning Go from self-play based only on the rules), IMO — based on how often
by ben_w 2y ago
You can get knowledge collapse.
But even though there are counter-examples (e.g. learning Go from self-play based only on the rules), IMO — based on how often people were already doing this with buggy human-written software[0][1] — most people don't think about this in the right way and will therefore treat these things as magical oracles when they shouldn't.
No silver bullets.
[0] https://en.wikipedia.org/wiki/British_Post_Office_scandal https://en.wikipedia.org/wiki/British_Post_Office_scandal
[1] https://en.wikipedia.org/wiki/Computer_says_no https://en.wikipedia.org/wiki/Computer_says_no