3 ms·

> I cannot see why a superintelligence wouldn't be able to put together Stuxnet^10 How will this Stuxnet^10 be able to do orders of magnitude more damage, thou

by gary_0 3y ago

> I cannot see why a superintelligence wouldn't be able to put together Stuxnet^10 How will this Stuxnet^10 be able to do orders of magnitude more damage, though? Shut down power plants? Launch ze nukes? The "shut down power plants" issue already exists and people have been aware since the Y2K scare, and now due to cyberwarfare concerns. Such problems are solved by not being dumb, and making sure there are enough air-gaps and humans-in-the-loop that an errant computer command can't wreak havoc (or fire ze missiles). I guess your Stuxnet^10 could do damage by immobilizing the entire Tesla fleet, but once again, that's a dumb design issue with networked cars, not a vulnerability you need a super-AI to exploit. And I personally don't (and would never) drive a remotely-updatable car myself for exactly this reason. These are all problems that are easily solved by not doing anything monumentally stupid engineering-wise. The lesson we need to take to heart isn't Terminator, it's Chernobyl.

TomaszZielinski 3y ago
According to the Wikipedia article [1], Stuxnet not only infected the Iranian infrastructure but apparently also a Russian nuclear plant. And so air gaps & co. didn't help against a skilled, determined but still human-level attacker. But then I read the Wikipedia article more closely, and, for instance, it turns out that Stuxnet components were signed with stolen certificates so that Windows recognized them as legitimate. Which means that at best someone had not only write Stuxnet, but also get those certs. And even if it's easy in comparison, there had to be at least a remote hack involved, if not a physical one. And then there was the risk of those certs being revoked. Etc., etc. So, even if I'm more pessimistic than you, the lesson for me here is that if you start peeling the onion layers, it's all not so simple to pull off. [1] https://en.wikipedia.org/wiki/Stuxnet https://en.wikipedia.org/wiki/Stuxnet

by gary_0 3y ago

TomaszZielinski 3y ago

According to the Wikipedia article [1], Stuxnet not only infected the Iranian infrastructure but apparently also a Russian nuclear plant. And so air gaps & co. didn't help against a skilled, determined but still human-level attacker. But then I read the Wikipedia article more closely, and, for instance, it turns out that Stuxnet components were signed with stolen certificates so that Windows recognized them as legitimate. Which means that at best someone had not only write Stuxnet, but also get those certs. And even if it's easy in comparison, there had to be at least a remote hack involved, if not a physical one. And then there was the risk of those certs being revoked. Etc., etc. So, even if I'm more pessimistic than you, the lesson for me here is that if you start peeling the onion layers, it's all not so simple to pull off. [1] https://en.wikipedia.org/wiki/Stuxnet https://en.wikipedia.org/wiki/Stuxnet