> This result feels like creepy perpetual motion. It’s like they’re teaching the AI ethics by making it write an ethics textbook and then read the textbook it j

> This result feels like creepy perpetual motion. It’s like they’re teaching the AI ethics by making it write an ethics textbook and then read the textbook it just wrote. Is this a free lunch? Shouldn’t it be impossible for the AI to teach itself any more ethics than it started out with? Isn't this kind of feedback loop precisely what's predicted at the heart of the Singularity? A machine learning system reaching the point where it's able to improve itself without further human guidance? "Large Language Models Can Self-Improve" (https://arxiv.org/abs/2210.11610 https://arxiv.org/abs/2210.11610) describes something similar, increasing an LLM's performance by fine tuning it on its own 'high confidence' answers. I'd expect there to be a limit to the amount of extra improvement possible using this method, the question is whether this improvement is enough to push the system past the performance of the researchers that created it. (As an aside, is there a theoretical limit to how intelligent a system can be for a given quantity of training data? Of course, we'd need a rigorous definition of 'intelligence' and 'data'...)