I'm really not convinced we're close to AGI, at least in the traditional sense of it that LessWrong / the rationalists seem to be so worried about. I'm not conv

I'm really not convinced we're close to AGI, at least in the traditional sense of it that LessWrong / the rationalists seem to be so worried about. I'm not convinced that our current approaches will actually lead to AGI in that sense. However, I am worried about where AI is taking us. Because what I think I see happening is that we don't _need_ AI takeoff for this to change society for the worse. Just look at the Microsoft OpenAI stuff for example. We don't need the AI to prevent its shutdown with grey goo and nanobots. The incentive structure around this whole thing means that even as stupid and sociopathic as Sydney is, they _won't_ shut it off. It doesn't matter that they _could_. On the topic of alignment, the whole idea that _Microsoft_ of all companies thought that they could just feed GPT a good prompt and get the behavior they wanted is hilarious. We clearly have no idea how to get AIs to do what we want. It would be so funny that this was the approach of the absolute mind-boggling stupidity of it wasn't so horrifying. I'm not worried about the AIs manufacturing a plague, or turning us all into paperclips. I'm worried that our systems of government and business will have no ability whatsoever to deal with the effects of these systems in a way that doesn't result in harm. And I don't even mean direct harm, from people losing their jobs (although that certainly seems like it's on the horizon). I just mean the same enshittification of everything that we've been experiencing for the last 10 or 20 years, at rapidly accelerating paces, further increasing the wealth gap and leading us towards a shitty, boring, unfulfilling dystopia. It's not AI alignment I'm worried about, but human alignment. Looking back, it's miraculous that we somehow managed to dodge the bullet of nuclear MAD. Imagining the same thing playing out like that today is laughable. I've been relatively positive about AI until the last few weeks, but watching the absolute shitshow with Microsoft has really made me cynical about the whole project.