10 ms·
The best defense against malicious AI is AI
- etiam 9y agoI'm following the original title for the post for now, but frankly I'm unhappy with almost everything about the style of it. I would encourage changing it.
- Houshalter 9y agoI would go with "Kaggle Hosts Adversarial Machine Learning Competition".
- etiam 9y agoI like it. Accurate and descriptive.
- deleted 9y ago[deleted]
- TempleOS 9y agoYOU ARE THE SOOPTIEDST ATHEOISDT MONKEY RETARD NIGGER.
- melling 9y agoAre we still wasting our time talking about evil AI at this premature time? Can someone dig up Andrew Ng’s comments on this?
- arcanus 9y agoThe ever-timely: 'Fearing a rise of killer robots is like worrying about overpopulation on Mars.'
- narrator 9y agoKiller robots are already here : https://www.youtube.com/watch?v=MXm-ofKLB3M https://www.youtube.com/watch?v=MXm-ofKLB3M "Generally, the vehicle will have a set of sensors to observe the environment, and will either autonomously make decisions about its behavior or pass the information to a human operator at a different location who will control the vehicle through teleoperation."
- backpropaganda 9y agoThis is a strawman. Elon's idea and what Andrew is countering is that killer robots can be unintentionally created by someone trying to do good. Evil robots could always be created by evil actors, and there is nothing interesting here.
- narrator 9y agoLook at the ransomware and DDOS botnet epidemics though. If these programs were to infect and control self-driving cars all sorts of science fiction level bad stuff could happen.
- skgoa 9y agoYes, but that is a computer security issue (and a damn important one!), not an ML issue.
- Ace17 9y agoNot saying that creating military killer robots counts as "trying to do good", however ... if we're talking about a terminator-like scenario, let's not forget the robots from the movie initially are human-designed military appliances.
- rytill 9y agoPossibly by someone trying to do good. More likely by someone just following their rational incentives. Very few people are 'evil' actors but most follow incentives.
- c_shu 9y agoit's kindof irrelevant to andrew ng's comments. It's about something like Cyberattacks. Something can be done very soon like tricking spam filters
- booleandilemma 9y agoBoolean algebra was developed by George Boole in 1847. Konrad Zuse built the Z1 computer in 1936, almost 90 years later. Just because the technology isn't here yet doesn't mean we can't start discussing the theory and its implications.
- QAPereo 9y agoEspecially when the impact of strong AI will make the impact of automobiles look downright trivial by comparison.
- backpropaganda 9y agoThis isn't Elon Musk-style evil AI. This is the more prosaic idea of hackers being able to hack AI systems. For instance, a hacker can put stuff on a stop sign (which won't be visible to humans) to make a Tesla car think it's not a stop sign, and therefore cause an accident. I recommend reading up about it. This is the kind of stuff that can actually happen today.
- Houshalter 9y agoThere are practical applications too. For instance, getting around automated filters like copyright or porn detection. Making adversarial captchas that fool computers but not humans.
- rimliu 9y agoIf something looks like a stop sign to the human but like something else to the AI, then "I" stands for "Idiocy" not "Intelligence".
- ben_w 9y agoYour brain is not immune to the problem, it's just hard to automate the creation of optical (and audio, and presumably all other sensory) illusions when we don't have a synapse resolution connectome of the relevant bits of your brain. Examples include That Dress, duck-or-rabbit, stereotypes, "garden path sentences", and most film special effects.
- rimliu 9y agoI am talking about cases like these: https://blog.openai.com/robust-adversarial-inputs/ https://blog.openai.com/robust-adversarial-inputs/ http://www.popsci.com/byzantine-science-deceiving-artificial-intelligence http://www.popsci.com/byzantine-science-deceiving-artificial...
- ben_w 9y agoIronically your second link starts with Clever Hans, which is another example of my point. Machines, even organic ones like our brains, are not magically able to know objective reality, and the failure itself isn't idiocy — the rate of failure (and things like metecognition about the possibility of failure) is (at last part of) the intelligence-idiocy spectrum.
- deleted 9y ago[deleted]
- astrojams 9y agoWould strategies used by AI's that play imperfect information games like Poker be useful for winning a contest like this?
- inopinatus 9y agoThose who have not seen WarGames are clearly doomed to reinvent it. Just after I finish this game of "Falken's Maze"...
- rch 9y agoNo kidding - what else is a potentially hostile autonomous agent if not everybody.
- prodtorok 9y agoThe Oracle. She possesses the power of foresight, which she uses to advise and guide the humans attempting to fight the Matrix
- examplar 9y agothe best defence against AI is to reduce the # of bits for algorithm to use
- zitterbewegung 9y agoFor something that is more oriented toward patches and network defense see http://archive.darpa.mil/cybergrandchallenge/ http://archive.darpa.mil/cybergrandchallenge/ .
- dguido 9y agoI competed in that! It was a fun contest and if you want to read more about it, we have a section on our blog dedicated to it with lots of rich technical detail: https://blog.trailofbits.com/category/cyber-grand-challenge/ https://blog.trailofbits.com/category/cyber-grand-challenge/ I am sorry to admit, though, that it did not involve AI or machine learning in the slightest. Most of it was about melding the capabilities of fast, dumb dynamic testing like fuzzing together with the deep analytical capabilities of more advanced program analyses like symbolic execution.
- zitterbewegung 9y agoI was there so I probably saw you. I remembered that the competitors used symbolic execution . My comment should have been more general and say automated .
- hueving 9y agoThe sports-caster style commentary was hilariously awkward at times. A room full of people staring a stage full of server racks with a projector showing basic shapes being spewed between locations. Then off on another screen a technical person trying to explain to the Science channel person what the hell was going on with the different types of exploits and code execution graphs. It's completely indescribable and epically nerdy. +10, would attend again. :)
- zitterbewegung 9y agoIt was awkward at times, the visualizations were cool but didn't really help you understand what was occuring. But, given their content and also to get people not very technical excited? That is a heruclean task for anyone. But I hope DARPA tries again to do it. They could attempt to do better and make it interesting. OR they could embrace the awkwardness and own it. Make it hilarious, make it MEMEy, make it like Tim and Eric.
- zeep 9y agolet's hope that this good AI won't turn bad once it's too late to switch to something else...
- strictnein 9y agoOn a related note, at Blackhat this year there's a presentation on using Machine Learning based malware detection to train your malware to evade ML based malware detection: https://www.blackhat.com/us-17/briefings/schedule/index.html#bot-vs-bot-for-evading-machine-learning-malware-detection-6461 https://www.blackhat.com/us-17/briefings/schedule/index.html...
- s-brody 9y agoWell, if there were no AI, there were no malicious AI also.
- jerry40 9y agoA name of the topic reminds me "Watchbird" novel written by Sheckley
- wruza 9y agoOr, The City and the Stars by Arthur C. Clarke, though it is was not exactly computer AI. That book still makes me goosebumps every time I remember it.
- gsg 9y agoSo... the only way to stop a bad guy with a GAN is a good guy with a GAN?
- deleted 9y ago[deleted]
- bem94 9y agoDepending on ones level of idealism, the best defense against malicious AI is actually educating programmers / hackers / engineers on the implications of their work and to have a sense of decency & foresight. It baffles me that we just accept the kind of malice inflicted on people by programmers because "someone will always do it". As a profession / collection of skilled persons, we should really be better than that. Obviously, one cannot see the future, nor would we want to be paralized by fear of doing anything. But there is a certain minimum requirement for collective responsiblity which I really don't think we are meeting at the moment.
- skate22 9y agoEthics can get pretty subjective, so I think it would be hard to teach a uniform view of decency. Also I hate to sounds pessimistic, but there are a lot of smart people who do much worse things than create malicious software when money is at stake. I think educating the people is a step in the right direction, but I fear it will fall short of your ideal.
- bem94 9y ago>> I think it would be hard to teach a uniform view of decency Ofcourse, but we don't avoid studying moral philosophy because we can't find a uniform model. Nor does not being able to find a uniform model mean that there are not things worth learning from it. >> there are a lot of smart people who do much worse things than create malicious software when money is at stake Agreed. Two wrongs do not make a right, and as engineers in the know, we have a duty to stand up to such people when we have the chance. >> I fear it will fall short of your ideal. Me too, but the alternative is worse in my opinion.
- amelius 9y agoThe problem is that if engineers in the US don't take the job then someone somewhere in, say, Asia will. Ethical rules are not the same everywhere. You can already tell this by looking at experimentation with stem-cells, cloning, et cetera, which is happening faster in Asia than in the US because of regulations. Also, refraining from building these systems means that other countries get ahead of us. Besides being an economic disadvantage, this could also threaten security. So I do agree that engineers should adhere to ethical guidelines, but they should be considered in a global context.
- kerkeslager 9y agoYet another thing predicted decades ago by William Gibson in Neuromancer.
- pmlnr 9y agoCame here to comment "Wintermute?"
- dovdovdov 9y agoThat's how Skynet got started.
- sigzero 9y agoThis reminds me of a cyberpunk story.
- nullpage 9y agoThis reminds me of the Halo series lore: http://halo.wikia.com/wiki/Offensive_Bias http://halo.wikia.com/wiki/Offensive_Bias http://halo.wikia.com/wiki/032_Mendicant_Bias http://halo.wikia.com/wiki/032_Mendicant_Bias
- okreallywtf 9y agoSure Xerxes seemed like a good idea but we all saw how that worked out...[1] [1] http://shodan.wikia.com/wiki/XERXES http://shodan.wikia.com/wiki/XERXES
- dave7 9y agoReminds me of Person of Interest. I'm still amazed this show had a full run on mainstream TV. One of the best TV/movie treatments of "Hacker News topics" there has been. https://en.wikipedia.org/wiki/Person_of_Interest_%28TV_series%29 https://en.wikipedia.org/wiki/Person_of_Interest_%28TV_serie...