Huh. I hadn't seen those observations. They're both obviously correct. Yes, absolutely, AI will reach superhuman persuasion before it reaches AGI. Yes, that's

Huh. I hadn't seen those observations. They're both obviously correct. Yes, absolutely, AI will reach superhuman persuasion before it reaches AGI. Yes, that's an extraordinary threat. The reason it's a threat is that it's a weapon without self-guidance or direction: a paperclip maximizer, as it were. A gray-goo problem. I still think Sam has access to exactly this, and panic about it is what caused his firing. Whether his use of his persuasion-weapon has played a role in what ensued is not as clear. I get that huge swathes of OpenAI appear to have been persuaded, but I don't think this weapon applies as much to them, nor do I think it is unique and different in nature from what already exists. It's a way of short-cutting the process of persuasion and coming out with the answer right away. Everything superhuman persuasion can offer, has already been deployed by trial-and-error using tools like Google Analytics and whatever metrics Facebook offered, in recent (and not-so-recent) years. Just because you can now push a button and get the killer argument to sway a demographic doesn't mean people haven't got to that argument more slowly in previous years, whether it's watching the results of persuasion campaigns in click-through feedback… or issuing radio broadcasts and observing the results. Back in the day there was a man named George Wallace who spoke of the things he'd done in public service, and nobody cared, and then he began talking of something else 'and they stomped the floor'. AI persuasion is nothing more than a short-cut for getting to 'and they stomped the floor', and there's more than one way to elicit that.