Thanks for clarifying. I strongly agree with your paragraphs 2-5, but I draw the opposite conclusion. Many alignment researchers don’t think that solving promp

Thanks for clarifying. I strongly agree with your paragraphs 2-5, but I draw the opposite conclusion. Many alignment researchers don’t think that solving prompt security will be similar to hard alignment challenges, I suspect it will at least somewhat (both requiring strong interpretability). Either way, it’s clearly a necessary precursor as you say. Most people I know that take AGI risk seriously _are_ terrified of how cavalier the companies like Microsoft are being. Nadela’s “I want everyone to know we made Google dance” line was frankly chilling. However, where I diverge from you is your final paragraph. Until very recently, as Hinton himself said, pretty much nobody credible thought this stuff was going to happen in our lifetimes, and the EA movement was considered kooky for putting money into AGI risk. If most people think that the worst that could happen is some AI saying racist/biased stuff, some hacks, maybe some wars - that is business as usual for humanity. It’s not going to get anyone to change what they are doing. And the justification for fixing prompt security is just like fixing IoT security; a dumpster fire that nobody cares enough about to do anything. If people, now, discovered an asteroid hurtling towards us, you’d hope they drop petty wars and unite (at least somewhat) to save the planet. I don’t happen to put P(doom) that high, but hopefully that illustrates why I think it’s important to discuss doom now. Put differently, political movements and the Overton Window take decades to effect change; we might not have decades until AGI takes off.