Worse, you need only plug a chatbot into itself, with some kind of basic bash script and very simple "goal prompt", and suddenly you get an agent with long term

Worse, you need only plug a chatbot into itself, with some kind of basic bash script and very simple "goal prompt", and suddenly you get an agent with long term context. You could do that today. I don't think people realize how close these generic undirected intelligences are to unpredictable complex behavior. A sobering intuition pump: https://www.lesswrong.com/posts/kpPnReyBC54KESiSn/optimality-is-the-tiger-and-agents-are-its-teeth https://www.lesswrong.com/posts/kpPnReyBC54KESiSn/optimality...