The underlying model architectures have issues that perpetuate as models get better. We're still using transformers and RL. * Optimized for task completion,

The underlying model architectures have issues that perpetuate as models get better. We're still using transformers and RL. * Optimized for task completion, with limited attention resources for global alignment (RL/RLAIF reward loops/hacking) * These systems run outside of chat now. file systems, CLIs, DBs, browsers → real-world side‑effects that you cannot train for. Hallucination becomes a problem of contradiction in the real world, and alignment is something an agent will struggle with as it's optimized to complete tasks. Which is why you see things like databases being dropped today. * These are baked-in problems, not even considering the the adversarial nuances of things like prompt injection. As AI advances, so do these issues. Maybe it's cliche from an AI safety perspective. But I can never get over https://en.wikipedia.org/wiki/Instrumental_convergence https://en.wikipedia.org/wiki/Instrumental_convergence as we see micro instances of it in our day-to-day with today's agents. Again, an issue that has existed from the dawn of these types of models. https://www.youtube.com/watch?v=s5qqjyGiBdc&t=1853s https://www.youtube.com/watch?v=s5qqjyGiBdc&t=1853s