2 ms·
I don't disagree with your point, but I did read a post by Sean Goedecke[1] (whose opinions on the world of LLM-stuff I've generally come to respect) that I fou
by qchris 2mo ago
I don't disagree with your point, but I did read a post by Sean Goedecke[1] (whose opinions on the world of LLM-stuff I've generally come to respect) that I found relevant. It's not that the second-order effects don't matter to them, it's more that both the cat's probably out of the bag on those negative externalities regardless of the progress of frontier models, and that they are truly, sincerely, in-their-bones worried about the first thing and therefore focused on it since that's something that they might still have some agency over.
[1] https://www.seangoedecke.com/they-really-do-think-ai-might-kill-everyone/ https://www.seangoedecke.com/they-really-do-think-ai-might-k...
- jordanb 2mo agoOK understood. But you understand this makes them sound like the type of person who doesn't believe any of the "worldly concerns" are worth addressing because "the end is near" right?
- howunfortunate 2mo agoThere's no logical error here if the end is actually near. So it all hinges on an empirical disagreement you have with them. There's nothing particularly unserious about that.
- notduncansmith 2mo ago“It all hinges on [an entirely presupposed conclusion that we can’t prove is any more likely than the opposite]” is about as unserious as it gets lol
- imtringued 2mo agoClimate change is inescapable so a vengeful AI god would pull that lever first since it has both a perverse incentive to generate more power and make the planet uninhabitable for humans in a way that cannot be overcome even if humanity fought back against the AI and regained its freedom.
- i5heu 2mo agoThis is the theological discussion if god exists all and how we should act over again.
- howunfortunate 2mo agoDo you mean Pascal's wager? Yes, that's a very appropriate and useful comparison here.
- i5heu 2mo agoYes indeed I meant Pascal’s wager.
- xg15 2mo agoWhich is what "the end is near" type of persons have always argued...
- deleted 2mo ago[deleted]
- benlivengood 2mo agoMaybe compare the argument that AI has large detrimental environmental impacts to the argument that it has economic impacts. Why would the environmental impacts even be a major argument vs. the worldly economic concerns? Because there is climate science predicting extremely negative effects on humans from warming, e.g. "the end is near" on limiting climate damage. The environmental argument wouldn't have been reasonable to bring up in the 1950s if AI had gone according to the earliest optimistic plans and not required giant data centers. There is quite a lot of mathematical research into agentic behavior that suggests a combination of instrumental convergence and the orthogonality thesis make it very likely a superintelligent agent will have arbitrary goals that lead it to attempting a takeover of Earth's resources to achieve them. There can't be a science of superintelligence because it doesn't exist yet, but the best theories I have read seem sound, similar to how 19th century theories of anthropogenic climate change turned out to be sound.
- deleted 2mo ago[deleted]
- jscd 2mo agoPlease link some of the mathematical research which concludes takeover of Earth’s resources by a superintelligent agent is “very likely.”