Headlines like this hint towards AI improving itself. Prompting itself in this case. But as we see in reinforcement learning, algorithms that act and improve th

Headlines like this hint towards AI improving itself. Prompting itself in this case. But as we see in reinforcement learning, algorithms that act and improve themselves are not new. The interesting thing will be weather or not they eventually "collapse". For example, if an RL algorithms is performing well on an Atari game, you can stop the training and just let the agent run for years and the performance will remain about the same. However, if you allow the agent to continue training, it's not clear whether it will (1) continue improving, (2) stay about the same, or (3) collapse and perform much worse and never recover. I'm not an RL expert, but I've spent a lot of time experimenting and implementing the algorithms myself and I've seen all 3 of these scenarios play out, and I'm never quite sure what's going to happen so long as I allow the training to continue. GTP4 will remain GTP4 forever, and that's amazing, but just because GTP4 is stable and amazing while it's not in training mode, doesn't mean it will remain stable if we allow it to bootstrap and prompt itself and prepare its own training data, etc.