4 ms·
It may be superfluous to point this out but this seems to be talking about malicious uses of narrow AI, rather than malicious strong AI. The only defense agains
by taneq 9y ago
It may be superfluous to point this out but this seems to be talking about malicious uses of narrow AI, rather than malicious strong AI. The only defense against a malicious strong AI is, of course, a friendly strong AI.
- tree_of_item 9y agoAt the moment "strong AI" is science fiction so what could you possibly say about it? How do you even know what the "only defense" is?
- edanm 9y agoA super-virus that has a 100% mortality rate, is incredibly infectious, and stays dormant for 1 year before killing the host. There's a fiction (I literally just made it up). Is it impossible to say anything about it? A bomb that is 1,000 times for powerful than a nuclear weapon. Again, currently science fiction. Can't say anything about that either? The fact that it doesn't exist right now doesn't mean we can't say anything about it, and the fact that it's potentially dangerous means we should be trying, at least, to think about it. Assuming it really is as dangerous as some people claim, do you really want to wait around until it's no longer science fiction, and only then, after it's too late, start thinking about it?
- tree_of_item 9y agoA "superintelligent" AI that is to a human as a human is to a fly. Is it impossible to say anything about the world after it's been built? Yes. It's not the fact that it doesn't exist, it's the fact that not only is it not clear that it's even physically possible, anything you come up with can just be handwaved away by this mystical property known as "intelligence". Your two examples don't have this problem so they're not similar. Most of the literature I've seen about AGI is about theories on how to build it, or theories on how to build it safely (e.g. so it has stable preferences that align with human values). Not stuff on "how to defend yourself against a rogue AGI", because there really is nothing you can say about that.
- taneq 9y ago> Is it impossible to say anything about the world after it's been built? Well, actually no. It's possible to say one thing (and as far as I'm aware, only this thing): You'd better have an even smarter friendly AI on your side, or you're almost certainly boned.
- shkkmo 9y ago> You'd better have an even smarter friendly AI on your side, or you're almost certainly boned. Are you sure? Was the world better off with both the US and Russia having nukes? Perhaps, or perhaps we got incredibly lucky that we didn't destroy ourselves. It is quite possible that a world where we have a single malicious AI is better than a world where a malicious AI and a friendly AI are locked in a battle to the death.
- taneq 9y agoYou're missing the point that an active, malicious strong AI is a nuke that's already been detonated.
- shkkmo 9y agoThe problem may not have been your point (that we can't reason about strong AI), but the way you justified it. Strong AI is not hard to reason about because it is science fiction. As edanm pointed out, it is entirely possible to reason about science fiction. The difference is that the examples edanm points out are extensions or modifications of things that do exist and that provides a basis from which we can reasonably extrapolate. We can't reason about strong AI precisely because we don't have as good of an existing example from which we can extrapolate. The best we have is the human mind. The extrapolation from the human mind is precisely the basis for argument that "friendly strong AI" is the best/only response to to "malicious strong AI" since the thing that keeps malicious humans in check is "good" humans. We don't even know if or to what degree 'super-intelligence' is possible. For all we know, there may be fundamental trade offs between the different skills necessary for general intelligence that may provide functional limits to the degree to which intelligence can be super human.
- jonkiddy 9y agoThis made me think of a riddle I once heard recently from a high school student. "What is sexually transmitted, 100% fatal, and literally everyone you know is infected? ...Life."
- shkkmo 9y agoExcept life isn't sexually transmitted. There is no "receiver" of life in the sexual process. Life is propagated through sexual reproduction. There is also no "infection". The joke, as told properly, is: "What is spread through sex, 100% fatal and literally everyone you know has had it?"
- nradov 9y agoA JDAM through it's power supply would be an effective defense against a malicious strong AI.
- whiskers08xmt 9y agoA strong AI which limits itself to one machine, isn't very strong at all.
- adrianN 9y agoOnly if you recognize its malicious intent before it convinced you to implant the fancy new chips that it invented right into your brain.
- landryraccoon 9y agoLions and Bears : Humans aren't a threat. Sharp teeth through the neck are an effective defense against a malicious human. If the hypothetical malicious AI gets built, your JDAMs will be about as advanced by comparison as a pointy stick against an M-16.
- nradov 9y agoWhatever. Let me know when someone invents an AI as smart as a lab mouse. Until then this is all just pointless idle speculation.
- landryraccoon 9y agoThat's fair, but arguing something isn't possible is very different from arguing that when it happens you can solve it with a bomb. The conversation is about a hypothetical strong AI. You can say strong AI isn't possible, like you can say warp drives aren't possible, and it's still valid to discuss what it might be like if they were possible.
- platz 9y agoyou're missing the point assuming we're talking about strong AI - when the paper is more addressing malevolent uses of weak AI.
- debt 9y agoOf course.