4 ms·
As a principal I tend to lower my conviction of any opinion I hold the more that people I intellectually respect disagree with me. I think to say with confiden
by kypro 3y ago
As a principal I tend to lower my conviction of any opinion I hold the more that people I intellectually respect disagree with me.
I think to say with confidence that we're doomed or that there is no risk to the AI system currently in development would be intellectually arrogant. I'm not even convinced you necessarily need AGI before AI becomes a risk to humanity. An advanced enough auto-GPT bot given a bad goal might be all you need. Cripple enough infrastructure and humanity is going to be starving to death within weeks.
Personally I think humanity is probably okay for another 5 - 10 years, but after that I have almost zero certainty. It seems to me that it's quite possible we could have AGI by the end of the decade if we get another breakthrough or two.
Even near-term I have no strong conviction. I have a decent amount of experience building and working with neural nets, but my experience is so far from scale and complexity companies like OpenAI are working at my ability to predict even what might come out in the next 1-2 years is very low. I'd basically just have to assume the next GPT is going to be somewhere between 10-50% jump from what we got this year. But what if it's 200% and the trend is moving exponentially?
- fl7305 3y ago> Cripple enough infrastructure and humanity is going to be starving to death within weeks. Just the threat of a malicious AGI that people take seriously enough would cause ourselves to pull the plug on the Internet and all computers until they can be disinfected and new filters put in place. It doesn't matter if the AGI is capable of actually inflicting damage or not.
- rzl1235 3y agoAh yes, just "pull the plug". That's a hilarious idea, are you sure the AGI that was trying to destroy humanity didn't give you that one?
- reverius42 3y agoWe couldn’t even do coordinated lockdowns for a few weeks to prevent millions of Covid deaths, and you think we’ll have a coordinated cessation of computer and internet use?
- fl7305 3y agoI live in Europe. Germany, France and other large countries became virtual ghost towns overnight as it became illegal to leave your home for other than critical errands. Wasn't it the same in big parts of the US? I remember seeing videos of a helicopter flying over LA without cars or people out on the streets. But sure, the whole world wouldn't coordinate to shut down.
- rzl1235 3y ago[dead]
- mike_hearn 3y ago> An advanced enough auto-GPT bot given a bad goal might be all you need. Cripple enough infrastructure and humanity is going to be starving to death within weeks. AI risk enthusiasts tend to define "bad goal" as one that sounds reasonable but has unexpected side effects. The real risk to infrastructure and physical safety though is that people will use AI to assist in hacking during a hot war. In this case the goal is only bad from the perspective of one side of the conflict, and the solutions are the same in any event because human hackers can also do a lot of damage, at least in theory. Our computing infrastructure hasn't ever really been tested in a hot war between sophisticated powers, the closest is currently Russia v Ukraine where electricity outages and the like have been caused by hackers but the damage seems minor compared to that inflicted by conventional warfare. So this is maybe a "hopeful" outcome - the much touted cyberwarfare has fizzled so far. So the argument would have to become that AI hackers will be vastly better at disruption than human hackers are. And I guess I can sort of believe that in principle one day but currently they are very far away from this sort of capability.
- deleted 3y ago[deleted]