4 ms·
It seems most people who take a position against the feasibility of machine superintelligence are basically making an argument-from-lack-of-imagination: "I can'
by leot 11y ago
It seems most people who take a position against the feasibility of machine superintelligence are basically making an argument-from-lack-of-imagination: "I can't think up a plausible way that bad things could happen, therefore bad things won't happen." This is then backed up by an argument-from-authority: "And I've been in this field for X decades, so you should trust my ability to imagine this kind of thing."
We have seen arguments like this throughout history. Often, what motivates them is the writer's own failures and consequent life narrative: "I couldn't do Y, so Y isn't possible right now."
- at-fates-hands 11y agoI completely agree with your points. It's strange to me to think as humans we will remain at the top of the food chain forever. As such, it would seem a fault of our genetic makeup thinking nothing would capable of knocking us off that top rung of the evolutionary ladder. It's like we can't fathom building something that would undermine our own existence, or downplay the notion that AI would have some sort of bad intentions since we built it.
- leot 11y ago"Bad intentions" aren't even necessary -- out of control AGI will probably look more like "oops, I hit >rm -rf /". AFAIK, nearly every piece of software ever deployed has had bugs that humans have had to subsequently patch. AGI must not, even though it would probably be much easier to build it if it did.
- TeMPOraL 11y ago> AGI must not, even though it would probably be much easier to build it if it did. Which is exactly what raises the issue to the level of an existential threat - it's easier to build an Unfriendly AGI than a Friendly one - which means the first GAI ever built is likely to be Unfriendly - and it takes only one impatient team / person unleashing an UGAI on the world to kill us all.