I find the complexity arguments convincing despite the fact that they have various weaknesses (that while a solution to some NP-hard problem would suffice, it i

I find the complexity arguments convincing despite the fact that they have various weaknesses (that while a solution to some NP-hard problem would suffice, it isn’t necessary and a merely good solution or a solution to an easier subset of problems would be ok.) There’s two things to talk about: 1. The article seems to switch a bit between superhuman intelligence or performance at something (I find this reasonably plausible) and a runaway singularity where intelligence/performance grows exponentially and the computer then moves to destroy us all. A few reasons I find the latter proposition implausible are that only one problem of many needs to be too hard for complexity to be an issue; it’s not clear how good exponentially better performance is (for something like image recognition, how much does it matter to be able to recognise lots more things, or to have a 1% error rate that halves every year? I say not that much,) indeed it seems that evolution isn’t particularly optimising for intelligence in humans despite how much some of us may think it matters, although that could just be due to it not mattering so much until recently; and that it often relies on imagining an intelligence which is somehow clever in the way that humans and computers are clever (eg also fast at big mathematical calculations), but also stupid in the ways computers can be stupid, being precisely logical and optimising for a single goal (I find it unbelievable that the all-consuming paperclip-making intelligence would perfectly follow its paperclip-making instruction while destroying its masters and never thinking to disobey.) 2. I feel like most people who are serious believers in a singularity (eg Yudkowsky et al) are massive cranks who’ve reasoned themselves into a weird sci-fi corner. I struggle to believe that an argument from them is grounded in reality rather than begging their prophecy of a singularity future.