Its context overflows and it becomes incoherent, so it stops following any rules you set within increased frequency until it accesses something it shouldn’t and

Its context overflows and it becomes incoherent, so it stops following any rules you set within increased frequency until it accesses something it shouldn’t and turns off the power to something that kills dozens of people. Then we will do some handwringing over it, and it will happen to hundreds of people, and it will just escalate and escalate and escalate. This is exactly what happened with social media. Every discovery has led to some angry words, people going “shucks we’ll do better next time!” and no meaningful change. The money is too good. We have only had our first major instances of acknowledging these are addiction platforms targeting adults and kids alike and it’s taken what? 20 years? I’m not worried about an AI going terminator. I’m worried about dumbasses dropping agents into critical systems and what will inevitably happen as a result. And I guarantee you this is going to happen because there’s nothing businesses love more than cutting personnel and corners to save a buck. AI enables this tendency in innovative ways, that’s the value to them. Everyone now has access to digital grenades and we wonder why everything feels so uneasy right now.