41 ms·

Preparing for Malicious Uses of AI

330 points by williamtrask 9y ago

akerro 9y ago
You can have 1000 of companies that act fair and don't use AI for malicious purposes, but there it that one company or community that doesn't... and then someone sends you gay porn with your face in it.
- extrememacaroni 9y ago
  Wow, you summed up the dangers of AI so concisely!
  - Pica_soO 9y ago
    Imature Joke in 3 .. 2 .. 1 .. Pentrationtest. Hey- at least i brought a keg of beer. PS: 1. Policymakers should collaborate closely with technical researchers to investigate, prevent, and mitigate potential malicious uses of AI. I never knew that research in Security was so deeply underfunded, that you had to write pleas into a paper. Jobsecurity research - its really that dangerous.
- ZenPsycho 9y ago
  that sending you gay porn is the worst thing you can imagine out of this is more terrifying to me than the AI.
  - akerro 9y ago
    Your username checks out!
    - deleted 9y ago
      [deleted]
- simias 9y ago
  I'm not too worried about that. The moment that type of technology becomes widely available is the moment this type of blackmail loses all edge. You might even actually start doing gay porn IRL and people will assume that it's been "deepfaked". The corollary is a little more worrying: any kind of incriminating document about a politician or public figure will be dismissed as a fake immediately. I mean, they already do that, but that'll be even harder to figure out what's real and what's not. That "grab them by the pussy" tape? Obviously fake. I mean, you don't even see the guy talking, just the audio, how gullible can you be? That girl running away from the napalm bombing? Obviously fake. I mean you're going to tell me that all of her clothes burned but she's still fit enough to run? Everybody around her wears clothes. Come on man, are you new here? That chinese guy standing in front of a military tank with groceries? Come on, I can do a more convincing fake in 10 seconds on my smartphone. There, look, I just did. We have a brave new world ahead of us where you won't be able to trust anything you see or hear through any media, no matter how convincing it seems. That's pretty terrifying IMO. I remember a while ago stumbling upon a conspiracy theory forum where people were claiming that a video of an interview with Julian Assange was a fake because there were a few strange visual artifacts around his face sometimes. Given that the quality of the video was very good and the oddities were rather minor (possibly encoding artifacts) I dismissed it as the usual tinfoil hattery. I think in the future I won't be so sure anymore. I'm not sure if the technology to make such a good quality fake already exists but it's probably a matter of years before we get there. If some people with too much time on their hands manage to make somewhat convincing porn montages for free on the internet what can big three letter agencies do? What does the state of the art look like? What will it look like 10 years from now?
  - tjr225 9y ago
    > We have a brave new world ahead of us where you won't be able to trust anything you see or hear through any media, no matter how convincing it seems. That's pretty terrifying IMO. Anyone can correct me if I'm being a bit dramatic but personally, it feels like we are very much already there.
    - nugi 9y ago
      That would has always been here. From embellishing town sayers, to the Hurst empire, media has always been The tool for mass manipulation.
Timothycquinn 9y ago
Reading this makes me think it's the opening to a sci-fi movie. Sure is damn scary that it's actual fact. I wonder how much of society will deny AI just like they are with human generated climate change?
- jimmy1 9y ago
  The difference is we don't need AI. We need the planet.
  - eliben 9y ago
    I wonder if we really don't need AI. It's not like we have amazing ideas on how to save the planet and reverse global warming realistically. Sure, there _are_ ideas but the costs seem comically high. Maybe AI could help us do that, and make cold fusion work while it's at it ;-) [I'm only half joking, actually]
  - michaelmcmillan 9y ago
    We need AI to keep the planet.
igorkraw 9y ago
Glad to see more "mainstream" recognition of the political threat of AI supercharged surveillance and possibly manipulation
appleflaxen 9y ago
In case it saves anyone time: the real meat in this link is the arxiv paper: The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation https://arxiv.org/abs/1802.07228 https://arxiv.org/abs/1802.07228
bitL 9y ago
It's already too late.
mtkd 9y ago
It's out of the bag now - we just have to hope the blue team can defend against regimes where the best maths talent more likely ends up building military apps than doggy photo filters
taneq 9y ago
It may be superfluous to point this out but this seems to be talking about malicious uses of narrow AI, rather than malicious strong AI. The only defense against a malicious strong AI is, of course, a friendly strong AI.
- tree_of_item 9y ago
  At the moment "strong AI" is science fiction so what could you possibly say about it? How do you even know what the "only defense" is?
  - edanm 9y ago
    A super-virus that has a 100% mortality rate, is incredibly infectious, and stays dormant for 1 year before killing the host. There's a fiction (I literally just made it up). Is it impossible to say anything about it? A bomb that is 1,000 times for powerful than a nuclear weapon. Again, currently science fiction. Can't say anything about that either? The fact that it doesn't exist right now doesn't mean we can't say anything about it, and the fact that it's potentially dangerous means we should be trying, at least, to think about it. Assuming it really is as dangerous as some people claim, do you really want to wait around until it's no longer science fiction, and only then, after it's too late, start thinking about it?
    - tree_of_item 9y ago
      A "superintelligent" AI that is to a human as a human is to a fly. Is it impossible to say anything about the world after it's been built? Yes. It's not the fact that it doesn't exist, it's the fact that not only is it not clear that it's even physically possible, anything you come up with can just be handwaved away by this mystical property known as "intelligence". Your two examples don't have this problem so they're not similar. Most of the literature I've seen about AGI is about theories on how to build it, or theories on how to build it safely (e.g. so it has stable preferences that align with human values). Not stuff on "how to defend yourself against a rogue AGI", because there really is nothing you can say about that.
matte_black 9y ago
How will we stop deepfakes?
- akerro 9y ago
  Dont post your pictures online.
  - falcolas 9y ago
    That only works if all your friends and people who tag you in photos online cooperate. That is, simply abstaining from posting pictures online doesn't work.
  - matte_black 9y ago
    Will not work. Someone else can easily take your picture and even do it from a distance with a telephoto.
  - pixl97 9y ago
    How many cameras have have you walked by already today?
    - matte_black 9y ago
      Makes me wonder if snapping a photo of a person without their consent then immediately going home and making deep fake porn about them could constitute as digital rape.
      - nugi 9y ago
        It can be charged under slander laws iirc, but without physical assult, not rape. Many harassment laws may also cover this. But if the picture was taken in public, you may not have many rights. Using 'rape' to mean any unwanted sexual communication is a terrible thing, and seems to be getting more common.
  - bitL 9y ago
    You need 100-10000 of good ones to get any good results for now. Maybe once they employ GANs to the mix... Well, I guess we'll see with the next gen of NVidias...
zucchini_head 9y ago
I find it quite funny, rather intruiging, that we seem to have gone full circle on trusted sources of information. Historically, a face-to-face meeting was considered as the ultimate legitimate and trustworthy way. Not story or rumors or witnessing, since the courts say people can be "decieved", "traumatised", etc. Then came microphones, cameras, CCTV in the 20th century, and then they became the ultimate trusted sources of information. And due to AI and it's rapidly increasing misuse by enormous conglomerates, it will be very soon when videos are never trusted but rather treated as comedic rumor and folklore, and we will go back again to how it always was. ...until replicants come. I'm saddened that there are actual "smart" people who waste their days to work on these malicious forms of AI, be it Google's almost entire arsenal, or anything. However, i'm not surprised they do, but it is still sad.
- VMG 9y ago
  There is and will be a place for strong crypto for trust, however probably not on standard commodity hardware.
  - PeterisP 9y ago
    Crypto works for trust of computing systems - it can make some claims about the transformations of the captured data or about the integrity of the software handling these transformations, but it fundamentally can't make solid claims about reality that's supposedly being captured. You can't put crypto between reality and a sensor. At best, crypto can give you a statement like "something possessing the particular secret X claims to have sensed this data at this point of time" combined with "there's a device with secret X that has it's software/firmware integrity verified and signed by entity Y". Crypto can't ensure that the secrets on the device aren't actually leaked by the manufacturer to enable "verifying" of arbitrary data outside of that device, and there's always the option to simply ensure that the sensor "sees" what you want; a camera and all the crypto on it can't tell whether it's pointed at a real event, at a staged event, or at a sophisticated optical device projecting arbitrary photoshopped data.
- rm_-rf_slash 9y ago
  Your comment reminds me of a silly little graph I saw posted to reddit once, basically stating that the prevalence of things like miracles and witchcraft was high throughout human history until the development of the camera, where it stayed low until the development of Photoshop.
strawcomb 9y ago
I would suggest that we shouldn't ever rely on digital archiving of important information. There should always be a copy of the information in analog, that can be dated & verified with analog methods.
- mwaitjmp 9y ago
  This is one feature a blockchain excels at. People have stored the Bitcoin white paper in the blockchain. Anyone can then download it, and verify it is the untouched original.
  - jl6 9y ago
    You don’t really need the huge overhead of a block chain for that. You just need a hash and some redundant storage.
    - SXX 9y ago
      Block chain still handy to verify that no one tried to rewrite history and pretrnd hash was different in first place. And hashes on larger documents are never safe from collisions.
    - sethgecko 9y ago
      And to trust the guy who is storing the hash?
      - drusepth 9y ago
        You could always put the hash on the blockchain. :)
  - strawcomb 9y ago
    Going fully digital with info, verifying with blockchain is all well and good, until a malicious actor forbids/prevents people from using blockchain somehow. if you've gone fully digital at that stage, you're in trouble
  - 9y ago
viach 9y ago
I was thinking recently, isn't any usage of Strong AI is in fact the same as slavery and, hence, immoral?
- Santosh83 9y ago
  According to most humans no, because while it may possess a mind, it does not have a soul and hence can be used and exploited. This is the argument we have given ourselves over millennia to use highly intelligent animals and of course human slavery was based on the myth that Africans were sub-human and slavery was good for them. Why do you think this time will be any different?
  - pacala 9y ago
    String AI will have a soul.
  - viach 9y ago
    Because it's 2018 and everyone is looking so cool and we've got our history lessons and we fight inequality and all this stuff? Or, do you mean, this is just another circle of the spiral of human biases?
  - nradov 9y ago
    In the Roman empire they didn't even bother making a myth that their slaves were sub-human. It was simply a matter of might makes right: we conquered your nation so now we own you.
    - bloak 9y ago
      I think perhaps you're confusing Roman history with some other period of human history, perhaps on a different continent. In the Roman empire there were slaves who taught philosophy, slaves who managed large estates, etcetera, and people could both sell themselves into slavery and be freed from slavery. In fact, selling oneself into slavery was a popular route to becoming a Roman citizen. If you don't want to read a history book, for which I wouldn't blame you, you might nevertheless enjoy Robert Harris's Cicero trilogy, which gives a fairly accurate impression of Roman society (or so claim many reviewers more competent than me). It's a truly amazing period of human history, when life was so modern in some ways, and yet so different from today in other ways, and the world was small enough for an individual to change the course of history.
nukeop 9y ago
We're already seeing lots of malicious uses of AI, for example: - User profiling - De-anonymization - Mining and correllating data from purchased databases of user info - "Pre-crime" prediction that influence real decisions - Changing insurance rates, credit scores, and so on based on decisions of completely opaque AI systems that use data from unknown sources
announcerman 9y ago
I really enjoy the example set by ClarifAI, the ability to search terrabytes of video with the help of tags is going to be a very nice boon for any totalitarian regime in the future.
Santosh83 9y ago
When AI becomes practically unrecognisable from human, it will get really interesting in finding ways and means to stop it from being used for conning.
Mizza 9y ago
The disturbing thing about this paper to me - flashy though it may be - is what they left out rather than what they kept in. OpenAI appears to only be thinking of the crimes-against-individuals segment of malicious AI, rather then the crimes-against-humanity type of malicious AI that the surveillance advertising corporations who are supporting OpenAI are building. I am far, far less worried about an assassin's drones using AI to find a politician in a crowd than I am about Facebook using pictures of me that other people have posted and tagged me in, so that my face is used to track my movements, and the movements of every other human on the planet, everywhere we go, and selling that information to everybody who wants a copy, and giving it away at the request of the local police. I'm more concerned about Google using AI to mine every conversation I've ever had or my browsing history to classify me as a dissident before I apply for a visa to travel to China or the United States, or as a deadbeat before I apply for a bank loan, or sick before I apply for insurance, or as unrehabilitatable before I apply for parole. The hackers-on-steroids narrative is a smokescreen for fully automated corporate fascism.
- macawfish 9y ago
  I took a class Sociology class in college called "Killing". Not a single time did we once ever talk about "authoritative" or "legitimate" killings of any sort. It turned out that this class was specifically meant for 'Criminal Justice' majors, people who were going on to be police officers and prosecutors. We didn't ever talk about war, the death penalty, genocide, eugenics, abortion, differential reproductive success, famine, none of it. The whole class was about homicide. I was disturbed and appalled.
  - dabbledash 9y ago
    One of these things is not like the others...
  - cryoshon 9y ago
    there are many experiences like this. the worst part: those people will leave the class thinking they have the whole story. the part you mention won't ever enter their mind-- it isn't under the heading of "killing" because "killing" only happens to the groups covered in class.
jcadam 9y ago
Evil startup idea gleaned from paper: Use AI/ML to scour a sales prospect's online persona (social media) and build a 'vulnerability profile' and generate targeted, personalized cold-emails (or even phone calls eventually). Also identifies 'levers' for a particular person that can be used to influence a buying decision. May even pre-qualify leads for you and tell you when not to waste your time :) I mean, a good sales person already does a lot of this, but it's time-consuming. Imagine if you could automate this process.
- meterplech 9y ago
  https://bombora.com/ https://bombora.com/
- no1youknowz 9y ago
  It's not an idea. You are already 5 years too late. You don't need to scour online personas. Companies such as Acxiom and FullContact are just two of many which house identities of 10s of millions of individuals. Then all that's needed to do is plug into twitters firehose and facebooks stream and voila. Real-time data based on what they are thinking/doing online. Oh and connect to an RTB stream and get real-time where they are clicking. With enough data, you could forecast where they are going next. Finally, have AI compose the content in the sales funnel to secure that conversion. I can guarantee you, this is actively being worked on as I type. Within 5 years, the entire marketing aspect will be mostly automated.
  - jcadam 9y ago
    > It's not an idea. You are already 5 years too late. Story of my life ;)
- notabee 9y ago
  Cambridge Analytica already did this for the past U.S. election. It had an app that told canvassers which doors were worth knocking on and what sort of dialogue to use depending on things like the person's estimated neuroticism and other traits. It'll only get more sophisticated.
paulintrognon 9y ago
If you are interested in this area of research, checkout Rob Miles' channel (the guy from Computerphile) => https://www.youtube.com/channel/UCLB7AzTwc6VFZrBsO2ucBMg https://www.youtube.com/channel/UCLB7AzTwc6VFZrBsO2ucBMg
deleted 9y ago
[deleted]
keymone 9y ago
what's up with whitepapers that look like powerpoint marketing material? first couple pages look like a shady ICO "whitepaper".
- sudouser 9y ago
  ico the video game, uh?
zitterbewegung 9y ago
I am wondering does anyone have a survey or list of AI exploits or malicious actions done on production services or systems ? For example like if a misclassified image that would target a image recognition system (such as Clarfai)? I have only seen papers that have theoretical attacks so far.
deleted 9y ago
[deleted]
3pt14159 9y ago
Reading the comments on this article make me realize how unserious most engineers and software developers are taking this. These threats are real. The weaponization of everything is actually happening. Since I wrote about it a month ago (Self-Crashing Cars) a number of people have reached out including people with actual insight into the military aspect of it. Militaries around the world are getting ready for true AI enabled weapon systems and there building deterrence strategies for mass casualty cyber attack (including nuclear weapons response), whether its from hacked industrial plants or cars it doesn't matter. They're actually talking about the weaponization of cars at the Munich security conference. We need to stop burying our head in the sand and write to our politicians about this threat. I know it sounds crazy but it's real. As an aside, my main complaint about the people that truly understand this the inability / unwillingness to accept that the act of subverting systems capable of mass destruction via cyber attack amounts to cyber weapons of mass destruction. We need to assemble all the work / treaties / regulations / research we put into securing nukes into securing AI / robotics and for the identical reasons. We know how this ends otherwise. We need wide-spread government funding and we need to communicate what these things are in language that our governments understand. Not saying something that is true just because it sounds weird is counterproductive.
- evoloution 9y ago
  I think you are absolutely correct. I think that people who are more entrenched in the field are even subconsciously unwilling to accept what is to come knowing that the unavoidable regulation will limit their creativity and just make their lives more miserable. But they know...
- AndrewKemendo 9y ago
  You can trust that people in the US Military are taking this very seriously. I know because I, alongside LtGen Shanahan and Eric Schmidt briefed Secretary Mattis personally about these issues and is part of my ongoing work within the IC/DoD. You're correct though that most line DL engineers don't have these issues at top of mind. I don't know any ML researcher worth their salt that hasn't thought about them, but the tendency is to brush the concerns off until we're closer to more generalized systems. That is not a wholly unreasonable position for many reasons, especially given the history of hype around AI, but I'd like to see more discussion happening between junior and mid-level engineers about these things - and especially more work being done about human-machine interfaces in the AI context, of which very little design thought put in.
jeffreyrogers 9y ago
It's nice to see a discussion of AI risk that addresses concrete scenarios. A lot of the forecasted doom (in other reports) resorts to handwavy arguments, but rarely goes into specifics. The examples they've given[0] seem plausible enough to me (except the persuasive ads one). [0]: Persuasive ads, vulnerability discovery and exploitation, hacking robots (this one is only tangentially AI related), and AI-augmented surveillance.
Bhilai 9y ago
I found this to be very fascinating read. I have heard of use of ML to detect, for instance spam or phishing emails, but I've never heard of attackers using ML models to generate phishing emails. How do you differentiate such a message from any other phishing attempt ? Thinking out loud, in the US, we have seen breaches of OPM, travel, healthcare and insurance companies where seemingly the only motive was to exfil data. Many of these attempts are attributed to state sponsored APT groups. Now that someone has all this data, the next potential move seems to be to train models over this data to understand habits and patterns, frequent locations and friends, and predict social and political leanings... I only have limited knowledge on this subject, but all this sounds plausible right ?
d0lph 9y ago
https://www.eff.org/files/2018/02/20/malicious_ai_report_final.pdf https://www.eff.org/files/2018/02/20/malicious_ai_report_fin... Still reading the paper and forming an opinion. But my initial thoughts are what exactly is new here that couldn't be done through some other means? I'm sure there will be interesting implications, but right now nothing seems particularly novel.
- platz 9y ago
  it is an injunction to policy makers, not a discovery of new methods.
  - d0lph 9y ago
    But is there really anything to report really? What's new that isn't doable with low level automation, I've been able to drive a browser with python for a while. > Human-like denial-of-service. Imitating human-like behavior (e.g. through human-speed click patterns and website navigation) It's already easy to simulate activity, clicking a random link on a page would be basically as good. Are there even DoS > Prioritising targets for cyber attacks using machine learning. Large datasets are used to identify victims more efficiently, e.g. by estimating personal wealth and willingness to pay based on online behavior. So, like, sum and sort all the spending data? But then again, maybe policy makers already didn't know what was possible? [edit] - formatting
    - platz 9y ago
      Ding ding ding
platz 9y ago
enjoying the brutalist-inspired design of the pdf
TuringNYC 9y ago
And then there is obscene "research" like this: https://arxiv.org/pdf/1611.04135v1.pdf https://arxiv.org/pdf/1611.04135v1.pdf "Automated Inference on Criminality using Face Images." How does stuff like this get past IRB?
hagbardceline 9y ago
I believe much of this can be dealt with, if we start RIGHT NOW to address a serious issue in our systems - the lack of a way to represent Morals and Ethics (the When and the Why) in the systems we are building. This needs to provide important input to, and thus shape the DIKW(D) pyramid. I've been doing some work with the IEEE on this - and I'm looking here on ycombinator to get some real-world feeedback on what people are thinking and concerned about. I have some (personal) ideas that might work to address the concerns I'm seeing. {NOTE Some of this is taken from a previous post I wrote (but kinda missed the thread developing, I was late I don't think anyone read it). It is useful for this thread, so a modification of that post follows.} First, I think you need a way to define 'ethics' and 'morals', with a 'ontology' and a 'epistemology' to derive a metaphysic for the system (and for my $.02, aesthetics arises from here). Until we can have a bit of rigor surrounding this, it's a challenge to discuss ethics in the context of an AI, and AI in the context of the metaphysical stance it takes towards the world. This is vital, as we need to define what 'malicious use' IS. This is still an area (as the thread demonstrates) of serious contention. Take sesame credit (a great primer, and even if you know all about it, it is still great to watch: https://www.youtube.com/watch?v=lHcTKWiZ8sI https://www.youtube.com/watch?v=lHcTKWiZ8sI ). Now here is a question for you: Is it wrong for the government to create a system that uses social pressure, rather than retributive justice or the reactive use of force, to promote social order and a 'better way of life'? Now, I'm not arguing for this (nor against for the purposes of this missive), but using it as a way to illustrate that different cultures, governmental systems, and societies, may have vastly different perspectives on the idea of a persons relationship viz a viz the state when it comes to something like privacy. I would suggest that transperancy in these decisions is a good idea. But right now we have no way to do that. I think the current way the industry is working - seemingly hell-bent on developing better, faster, more eficient, et al ways to engineer Epistemic engines and Ontologic frameworks in isolation is the root cause of the problem of malicious use. Even the analysis of potential threats (from the article referenced 'The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation' - I just skimmed it so I can keep up with this thread, please enlighten me if I'm missing something important) only pays lip service to this idea. In the executive Summary, it says: 'Promoting a culture of responsibility. AI researchers and the organisations that employ them are in a unique position to shape the security landscape of the AI-enabled world. We highlight the importance of education, ethical statements and standards, framings, norms, and expectations.' However in the 'Areas for Further research' section, I would point out that the questions are at a higher level of abstraction than the other areas, or discuss the narrative and not the problem. This might be due to the authors not having exposure to this area of research and development (such as the IEEE) - but I will concede that the fact that the note about the narrative shows that very few are aware of the work we are doing... This isn't pie-in-the-sky stuff, it has real-world use in areas other than life or death scenarios. To quickly illustrate - let's take race or gender bias (for example the Google '3 white kids' vs. '3 black kids' issue a while back in 2016). I think this is a metaphysical problem (application of Wisdom to determine correct action) that we mistake for an epistemic issue (it came from 'bad' encoding). This is another spin on kypro's concern about the consequences of AI deployment to enable the construction of a panopticon. This is about WISDOM - making wise choices - not about coding a faster epistemic engine or ontologic classifier. Next, after we get some rigor surrounding the ethical stances you consider 'good' vs. 'bad' (a vital piece that just isn't being discussed or defined) in the context of a metaphysic - you have to consider 'who' is using the system unethically. If it is the AI itself, then we have a different, but important issue - I'm going with 'you can use the AI to do wrong' as opposed to 'the AI is doing wrong' (for whatever reason, its own motivations, or it agrees with the evil or immoral users goals, perhaps, and acts in concert). Unless you have clarity here, it becomes extremely easy to befuddle, confuse, or mislead (innocently or not) questions regarding 'who'. - Who can answer for the 'Scope' or strategic context (CEO, Board of Directors, General Staff, Politburo, etc.) - Who in 'Organizational Concepts' or 'Planning' (Division Management, Program Management, Field commanders, etc) - Who in 'Architecture' or 'Schematic' (Project Management, Solution Architecture, Company commanders, etc) - Who in 'Engineering' or 'Blueprints' (Team Leaders, Chief Engineers, NCO's, etc.) - Who in 'Tools' or 'Config' (Individual contributors, Programmers, Soldiers, etc.) that constructed the AI. Then you need to ask which person, group, or combination (none dare call it conspiracy!) of these actors used the system in an unethical manner? Might 'enabled for use' be culpable as well - and is that a programmer, or an executive, or both? What I'm getting at here, is that there is both a lack of rigor in such questions (in general in this entire area), a challenge in defining ethical stances in context (which I argue requires a metaphysic), and a lack of clarity in understanding how such systems come to creation ('who' is only one interrogative that needs to be answered, after all). I would say that until and unless we have some sort of structure and standard to answer these questions, it might be beside the point to even ask... And not being able to ask leads us to some uncomfortable near-term consequences. If someone does use such a system unethically - can our system of retributive justice determine the particulars of: - where the crimes were committed (jurisdiction) - what went wrong - who to hold accountable - how it was accomplished (in a manner hopefully understandable by lawmakers, government/corporate/organizational leadership, other implementers, and one would think - the victims) - why it could be used this way - when could it happen again just for starters. The sum total of ignorance surrounding such a question points to a serious problem in how society overall - and then down to the individuals creating and using such tech - is dealing (or rather, not dealing) with this vital issue. We need to start talking along these lines in order to stake out the playing field for everyone NOW, so we actually might have time to address these things, before the alien pops right up and runs across the kitchen table.
vicpara 9y ago
I don't think these issues have anything to do with AI but 100% with humans. AI is used everywhere where money is to be made. Let's take ads. Why do we allow ads to happen all over the place (TV, radio, internet, magazines)? Because they feel stupid, harmless, how else can we sell products, etc? Well, then when a ML system is serving you ads, it's doing it so at the right time and will target you with the bang on content so that you buy exactly what it's needed. For eg. modern ML algos can pick up in which mood and mental state you are (out of 27 different states) by looking at how you read content online, what you read, how many tabs you open or how related the pages you visit are <yes, all cookie(-pool) based>. I know this because I build these things. It works incredibly well. It finds your weak spot, your passion, your habits, your indulgences and makes sure you're always tempted. It can tell more about yourself in a split second than you can think in an hour about who you really are. That bullshit you tell yourself about you doesn't matter because you are the rat the ML algo is baiting you until you bite. The problem is not that the AI is selling to the diabetic more cupcakes, the alcoholic booze and hipsters new gadgets like no other salesman. The idea of advertising is perverted itself from skin till bone marrow. How can you allow that to be done to the people? And we think this is how things are? OMFG.. AI is just a tool. WMD are another dead stupid idea only humans could come up with. Me country builds insane big weapons that can whack entire cities in split second just because it can & has more money & and we were first & we won the war & no reason. It also tries to make sure nobody can build it (like Iran) so they all kneel and kiss the ring. AI kind of levels the field. It's just a bigger nuclear bomb. My is bigger than yours taken to the next level. And that's just the beginning. AI makes it game on for more players than we like it to be. And it's not just AI (which is just math run by computers), it's mathematics, human genome research, physics, nuclear physics...all STEM subjects. You can come up with a weapon from any mix of these. There are only two ways out of it. The humanity dies or we learn how to all work together, chill our entitlement and find our common values regardless of religion, money, skin colour, gender or part of the world. A fart made in any part of the world can kill people everywhere. If humanity is to turn against itself because countries or corporations play a retarded game in AI/ pharma/ nuclear/fill_in_the_blank .. this only only accelerates a bit the process but the result will be the same. Let's ditch advertising, pharmaceuticals, OTC, bogus pills created without any fundamentals, weapon industries and all other nonsense industries or acts focused on massive profit at the cost of humanity. Let's ditch all kinds of waste and put people first for once. Let's ditch olympics where we try to prove that my country has it bigger than yours. Let's invest in education and health, inclusive societies where everyone works hard to solve humanity's tough problems. We keep building bigger guns but we as a society didn't grow much in empathy, inclusiveness or respect for the planet or other countries. People are killing people not AI. A machine gun is the safest thing in the world. No single machine gun on this universe loaded itself and started firing precisely to people or animals. Probably the best thing that can happen to this planet is to have humans vanish while there are still trees and animals around.

330 points by williamtrask 9y ago

You can have 1000 of companies that act fair and don't use AI for malicious purposes, but there it that one company or community that doesn't... and then someone sends you gay porn with your face in it.

extrememacaroni 9y ago
Wow, you summed up the dangers of AI so concisely!
- Pica_soO 9y ago
  Imature Joke in 3 .. 2 .. 1 .. Pentrationtest. Hey- at least i brought a keg of beer. PS: 1. Policymakers should collaborate closely with technical researchers to investigate, prevent, and mitigate potential malicious uses of AI. I never knew that research in Security was so deeply underfunded, that you had to write pleas into a paper. Jobsecurity research - its really that dangerous.
ZenPsycho 9y ago
that sending you gay porn is the worst thing you can imagine out of this is more terrifying to me than the AI.
- akerro 9y ago
  Your username checks out!
  - deleted 9y ago
    [deleted]
simias 9y ago
I'm not too worried about that. The moment that type of technology becomes widely available is the moment this type of blackmail loses all edge. You might even actually start doing gay porn IRL and people will assume that it's been "deepfaked". The corollary is a little more worrying: any kind of incriminating document about a politician or public figure will be dismissed as a fake immediately. I mean, they already do that, but that'll be even harder to figure out what's real and what's not. That "grab them by the pussy" tape? Obviously fake. I mean, you don't even see the guy talking, just the audio, how gullible can you be? That girl running away from the napalm bombing? Obviously fake. I mean you're going to tell me that all of her clothes burned but she's still fit enough to run? Everybody around her wears clothes. Come on man, are you new here? That chinese guy standing in front of a military tank with groceries? Come on, I can do a more convincing fake in 10 seconds on my smartphone. There, look, I just did. We have a brave new world ahead of us where you won't be able to trust anything you see or hear through any media, no matter how convincing it seems. That's pretty terrifying IMO. I remember a while ago stumbling upon a conspiracy theory forum where people were claiming that a video of an interview with Julian Assange was a fake because there were a few strange visual artifacts around his face sometimes. Given that the quality of the video was very good and the oddities were rather minor (possibly encoding artifacts) I dismissed it as the usual tinfoil hattery. I think in the future I won't be so sure anymore. I'm not sure if the technology to make such a good quality fake already exists but it's probably a matter of years before we get there. If some people with too much time on their hands manage to make somewhat convincing porn montages for free on the internet what can big three letter agencies do? What does the state of the art look like? What will it look like 10 years from now?
- tjr225 9y ago
  > We have a brave new world ahead of us where you won't be able to trust anything you see or hear through any media, no matter how convincing it seems. That's pretty terrifying IMO. Anyone can correct me if I'm being a bit dramatic but personally, it feels like we are very much already there.
  - nugi 9y ago
    That would has always been here. From embellishing town sayers, to the Hurst empire, media has always been The tool for mass manipulation.

Timothycquinn 9y ago

Reading this makes me think it's the opening to a sci-fi movie. Sure is damn scary that it's actual fact. I wonder how much of society will deny AI just like they are with human generated climate change?

jimmy1 9y ago
The difference is we don't need AI. We need the planet.
- eliben 9y ago
  I wonder if we really don't need AI. It's not like we have amazing ideas on how to save the planet and reverse global warming realistically. Sure, there _are_ ideas but the costs seem comically high. Maybe AI could help us do that, and make cold fusion work while it's at it ;-) [I'm only half joking, actually]
- michaelmcmillan 9y ago
  We need AI to keep the planet.

igorkraw 9y ago

Glad to see more "mainstream" recognition of the political threat of AI supercharged surveillance and possibly manipulation

appleflaxen 9y ago

In case it saves anyone time: the real meat in this link is the arxiv paper: The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation https://arxiv.org/abs/1802.07228 https://arxiv.org/abs/1802.07228

bitL 9y ago

It's already too late.

mtkd 9y ago

It's out of the bag now - we just have to hope the blue team can defend against regimes where the best maths talent more likely ends up building military apps than doggy photo filters

taneq 9y ago

It may be superfluous to point this out but this seems to be talking about malicious uses of narrow AI, rather than malicious strong AI. The only defense against a malicious strong AI is, of course, a friendly strong AI.

tree_of_item 9y ago
At the moment "strong AI" is science fiction so what could you possibly say about it? How do you even know what the "only defense" is?
- edanm 9y ago
  A super-virus that has a 100% mortality rate, is incredibly infectious, and stays dormant for 1 year before killing the host. There's a fiction (I literally just made it up). Is it impossible to say anything about it? A bomb that is 1,000 times for powerful than a nuclear weapon. Again, currently science fiction. Can't say anything about that either? The fact that it doesn't exist right now doesn't mean we can't say anything about it, and the fact that it's potentially dangerous means we should be trying, at least, to think about it. Assuming it really is as dangerous as some people claim, do you really want to wait around until it's no longer science fiction, and only then, after it's too late, start thinking about it?
  - tree_of_item 9y ago
    A "superintelligent" AI that is to a human as a human is to a fly. Is it impossible to say anything about the world after it's been built? Yes. It's not the fact that it doesn't exist, it's the fact that not only is it not clear that it's even physically possible, anything you come up with can just be handwaved away by this mystical property known as "intelligence". Your two examples don't have this problem so they're not similar. Most of the literature I've seen about AGI is about theories on how to build it, or theories on how to build it safely (e.g. so it has stable preferences that align with human values). Not stuff on "how to defend yourself against a rogue AGI", because there really is nothing you can say about that.

matte_black 9y ago

How will we stop deepfakes?

akerro 9y ago
Dont post your pictures online.
- falcolas 9y ago
  That only works if all your friends and people who tag you in photos online cooperate. That is, simply abstaining from posting pictures online doesn't work.
- matte_black 9y ago
  Will not work. Someone else can easily take your picture and even do it from a distance with a telephoto.
- pixl97 9y ago
  How many cameras have have you walked by already today?
  - matte_black 9y ago
    Makes me wonder if snapping a photo of a person without their consent then immediately going home and making deep fake porn about them could constitute as digital rape.
    - nugi 9y ago
      It can be charged under slander laws iirc, but without physical assult, not rape. Many harassment laws may also cover this. But if the picture was taken in public, you may not have many rights. Using 'rape' to mean any unwanted sexual communication is a terrible thing, and seems to be getting more common.
- bitL 9y ago
  You need 100-10000 of good ones to get any good results for now. Maybe once they employ GANs to the mix... Well, I guess we'll see with the next gen of NVidias...

zucchini_head 9y ago

I find it quite funny, rather intruiging, that we seem to have gone full circle on trusted sources of information. Historically, a face-to-face meeting was considered as the ultimate legitimate and trustworthy way. Not story or rumors or witnessing, since the courts say people can be "decieved", "traumatised", etc. Then came microphones, cameras, CCTV in the 20th century, and then they became the ultimate trusted sources of information. And due to AI and it's rapidly increasing misuse by enormous conglomerates, it will be very soon when videos are never trusted but rather treated as comedic rumor and folklore, and we will go back again to how it always was. ...until replicants come. I'm saddened that there are actual "smart" people who waste their days to work on these malicious forms of AI, be it Google's almost entire arsenal, or anything. However, i'm not surprised they do, but it is still sad.

VMG 9y ago
There is and will be a place for strong crypto for trust, however probably not on standard commodity hardware.
- PeterisP 9y ago
  Crypto works for trust of computing systems - it can make some claims about the transformations of the captured data or about the integrity of the software handling these transformations, but it fundamentally can't make solid claims about reality that's supposedly being captured. You can't put crypto between reality and a sensor. At best, crypto can give you a statement like "something possessing the particular secret X claims to have sensed this data at this point of time" combined with "there's a device with secret X that has it's software/firmware integrity verified and signed by entity Y". Crypto can't ensure that the secrets on the device aren't actually leaked by the manufacturer to enable "verifying" of arbitrary data outside of that device, and there's always the option to simply ensure that the sensor "sees" what you want; a camera and all the crypto on it can't tell whether it's pointed at a real event, at a staged event, or at a sophisticated optical device projecting arbitrary photoshopped data.
rm_-rf_slash 9y ago
Your comment reminds me of a silly little graph I saw posted to reddit once, basically stating that the prevalence of things like miracles and witchcraft was high throughout human history until the development of the camera, where it stayed low until the development of Photoshop.

strawcomb 9y ago

I would suggest that we shouldn't ever rely on digital archiving of important information. There should always be a copy of the information in analog, that can be dated & verified with analog methods.

mwaitjmp 9y ago
This is one feature a blockchain excels at. People have stored the Bitcoin white paper in the blockchain. Anyone can then download it, and verify it is the untouched original.
- jl6 9y ago
  You don’t really need the huge overhead of a block chain for that. You just need a hash and some redundant storage.
  - SXX 9y ago
    Block chain still handy to verify that no one tried to rewrite history and pretrnd hash was different in first place. And hashes on larger documents are never safe from collisions.
  - sethgecko 9y ago
    And to trust the guy who is storing the hash?
    - drusepth 9y ago
      You could always put the hash on the blockchain. :)
- strawcomb 9y ago
  Going fully digital with info, verifying with blockchain is all well and good, until a malicious actor forbids/prevents people from using blockchain somehow. if you've gone fully digital at that stage, you're in trouble
- 9y ago

viach 9y ago

I was thinking recently, isn't any usage of Strong AI is in fact the same as slavery and, hence, immoral?

Santosh83 9y ago
According to most humans no, because while it may possess a mind, it does not have a soul and hence can be used and exploited. This is the argument we have given ourselves over millennia to use highly intelligent animals and of course human slavery was based on the myth that Africans were sub-human and slavery was good for them. Why do you think this time will be any different?
- pacala 9y ago
  String AI will have a soul.
- viach 9y ago
  Because it's 2018 and everyone is looking so cool and we've got our history lessons and we fight inequality and all this stuff? Or, do you mean, this is just another circle of the spiral of human biases?
- nradov 9y ago
  In the Roman empire they didn't even bother making a myth that their slaves were sub-human. It was simply a matter of might makes right: we conquered your nation so now we own you.
  - bloak 9y ago
    I think perhaps you're confusing Roman history with some other period of human history, perhaps on a different continent. In the Roman empire there were slaves who taught philosophy, slaves who managed large estates, etcetera, and people could both sell themselves into slavery and be freed from slavery. In fact, selling oneself into slavery was a popular route to becoming a Roman citizen. If you don't want to read a history book, for which I wouldn't blame you, you might nevertheless enjoy Robert Harris's Cicero trilogy, which gives a fairly accurate impression of Roman society (or so claim many reviewers more competent than me). It's a truly amazing period of human history, when life was so modern in some ways, and yet so different from today in other ways, and the world was small enough for an individual to change the course of history.

nukeop 9y ago

We're already seeing lots of malicious uses of AI, for example: - User profiling - De-anonymization - Mining and correllating data from purchased databases of user info - "Pre-crime" prediction that influence real decisions - Changing insurance rates, credit scores, and so on based on decisions of completely opaque AI systems that use data from unknown sources

announcerman 9y ago

I really enjoy the example set by ClarifAI, the ability to search terrabytes of video with the help of tags is going to be a very nice boon for any totalitarian regime in the future.

Santosh83 9y ago

When AI becomes practically unrecognisable from human, it will get really interesting in finding ways and means to stop it from being used for conning.

Mizza 9y ago

The disturbing thing about this paper to me - flashy though it may be - is what they left out rather than what they kept in. OpenAI appears to only be thinking of the crimes-against-individuals segment of malicious AI, rather then the crimes-against-humanity type of malicious AI that the surveillance advertising corporations who are supporting OpenAI are building. I am far, far less worried about an assassin's drones using AI to find a politician in a crowd than I am about Facebook using pictures of me that other people have posted and tagged me in, so that my face is used to track my movements, and the movements of every other human on the planet, everywhere we go, and selling that information to everybody who wants a copy, and giving it away at the request of the local police. I'm more concerned about Google using AI to mine every conversation I've ever had or my browsing history to classify me as a dissident before I apply for a visa to travel to China or the United States, or as a deadbeat before I apply for a bank loan, or sick before I apply for insurance, or as unrehabilitatable before I apply for parole. The hackers-on-steroids narrative is a smokescreen for fully automated corporate fascism.

macawfish 9y ago
I took a class Sociology class in college called "Killing". Not a single time did we once ever talk about "authoritative" or "legitimate" killings of any sort. It turned out that this class was specifically meant for 'Criminal Justice' majors, people who were going on to be police officers and prosecutors. We didn't ever talk about war, the death penalty, genocide, eugenics, abortion, differential reproductive success, famine, none of it. The whole class was about homicide. I was disturbed and appalled.
- dabbledash 9y ago
  One of these things is not like the others...
- cryoshon 9y ago
  there are many experiences like this. the worst part: those people will leave the class thinking they have the whole story. the part you mention won't ever enter their mind-- it isn't under the heading of "killing" because "killing" only happens to the groups covered in class.

jcadam 9y ago

Evil startup idea gleaned from paper: Use AI/ML to scour a sales prospect's online persona (social media) and build a 'vulnerability profile' and generate targeted, personalized cold-emails (or even phone calls eventually). Also identifies 'levers' for a particular person that can be used to influence a buying decision. May even pre-qualify leads for you and tell you when not to waste your time :) I mean, a good sales person already does a lot of this, but it's time-consuming. Imagine if you could automate this process.

meterplech 9y ago
https://bombora.com/ https://bombora.com/
no1youknowz 9y ago
It's not an idea. You are already 5 years too late. You don't need to scour online personas. Companies such as Acxiom and FullContact are just two of many which house identities of 10s of millions of individuals. Then all that's needed to do is plug into twitters firehose and facebooks stream and voila. Real-time data based on what they are thinking/doing online. Oh and connect to an RTB stream and get real-time where they are clicking. With enough data, you could forecast where they are going next. Finally, have AI compose the content in the sales funnel to secure that conversion. I can guarantee you, this is actively being worked on as I type. Within 5 years, the entire marketing aspect will be mostly automated.
- jcadam 9y ago
  > It's not an idea. You are already 5 years too late. Story of my life ;)
notabee 9y ago
Cambridge Analytica already did this for the past U.S. election. It had an app that told canvassers which doors were worth knocking on and what sort of dialogue to use depending on things like the person's estimated neuroticism and other traits. It'll only get more sophisticated.

paulintrognon 9y ago

If you are interested in this area of research, checkout Rob Miles' channel (the guy from Computerphile) => https://www.youtube.com/channel/UCLB7AzTwc6VFZrBsO2ucBMg https://www.youtube.com/channel/UCLB7AzTwc6VFZrBsO2ucBMg

deleted 9y ago

[deleted]

keymone 9y ago

what's up with whitepapers that look like powerpoint marketing material? first couple pages look like a shady ICO "whitepaper".

sudouser 9y ago
ico the video game, uh?

zitterbewegung 9y ago

I am wondering does anyone have a survey or list of AI exploits or malicious actions done on production services or systems ? For example like if a misclassified image that would target a image recognition system (such as Clarfai)? I have only seen papers that have theoretical attacks so far.

deleted 9y ago

[deleted]

3pt14159 9y ago

Reading the comments on this article make me realize how unserious most engineers and software developers are taking this. These threats are real. The weaponization of everything is actually happening. Since I wrote about it a month ago (Self-Crashing Cars) a number of people have reached out including people with actual insight into the military aspect of it. Militaries around the world are getting ready for true AI enabled weapon systems and there building deterrence strategies for mass casualty cyber attack (including nuclear weapons response), whether its from hacked industrial plants or cars it doesn't matter. They're actually talking about the weaponization of cars at the Munich security conference. We need to stop burying our head in the sand and write to our politicians about this threat. I know it sounds crazy but it's real. As an aside, my main complaint about the people that truly understand this the inability / unwillingness to accept that the act of subverting systems capable of mass destruction via cyber attack amounts to cyber weapons of mass destruction. We need to assemble all the work / treaties / regulations / research we put into securing nukes into securing AI / robotics and for the identical reasons. We know how this ends otherwise. We need wide-spread government funding and we need to communicate what these things are in language that our governments understand. Not saying something that is true just because it sounds weird is counterproductive.

evoloution 9y ago
I think you are absolutely correct. I think that people who are more entrenched in the field are even subconsciously unwilling to accept what is to come knowing that the unavoidable regulation will limit their creativity and just make their lives more miserable. But they know...
AndrewKemendo 9y ago
You can trust that people in the US Military are taking this very seriously. I know because I, alongside LtGen Shanahan and Eric Schmidt briefed Secretary Mattis personally about these issues and is part of my ongoing work within the IC/DoD. You're correct though that most line DL engineers don't have these issues at top of mind. I don't know any ML researcher worth their salt that hasn't thought about them, but the tendency is to brush the concerns off until we're closer to more generalized systems. That is not a wholly unreasonable position for many reasons, especially given the history of hype around AI, but I'd like to see more discussion happening between junior and mid-level engineers about these things - and especially more work being done about human-machine interfaces in the AI context, of which very little design thought put in.

jeffreyrogers 9y ago

It's nice to see a discussion of AI risk that addresses concrete scenarios. A lot of the forecasted doom (in other reports) resorts to handwavy arguments, but rarely goes into specifics. The examples they've given[0] seem plausible enough to me (except the persuasive ads one). [0]: Persuasive ads, vulnerability discovery and exploitation, hacking robots (this one is only tangentially AI related), and AI-augmented surveillance.

Bhilai 9y ago

I found this to be very fascinating read. I have heard of use of ML to detect, for instance spam or phishing emails, but I've never heard of attackers using ML models to generate phishing emails. How do you differentiate such a message from any other phishing attempt ? Thinking out loud, in the US, we have seen breaches of OPM, travel, healthcare and insurance companies where seemingly the only motive was to exfil data. Many of these attempts are attributed to state sponsored APT groups. Now that someone has all this data, the next potential move seems to be to train models over this data to understand habits and patterns, frequent locations and friends, and predict social and political leanings... I only have limited knowledge on this subject, but all this sounds plausible right ?

d0lph 9y ago

https://www.eff.org/files/2018/02/20/malicious_ai_report_final.pdf https://www.eff.org/files/2018/02/20/malicious_ai_report_fin... Still reading the paper and forming an opinion. But my initial thoughts are what exactly is new here that couldn't be done through some other means? I'm sure there will be interesting implications, but right now nothing seems particularly novel.

platz 9y ago
it is an injunction to policy makers, not a discovery of new methods.
- d0lph 9y ago
  But is there really anything to report really? What's new that isn't doable with low level automation, I've been able to drive a browser with python for a while. > Human-like denial-of-service. Imitating human-like behavior (e.g. through human-speed click patterns and website navigation) It's already easy to simulate activity, clicking a random link on a page would be basically as good. Are there even DoS > Prioritising targets for cyber attacks using machine learning. Large datasets are used to identify victims more efficiently, e.g. by estimating personal wealth and willingness to pay based on online behavior. So, like, sum and sort all the spending data? But then again, maybe policy makers already didn't know what was possible? [edit] - formatting
  - platz 9y ago
    Ding ding ding

platz 9y ago

enjoying the brutalist-inspired design of the pdf

TuringNYC 9y ago

And then there is obscene "research" like this: https://arxiv.org/pdf/1611.04135v1.pdf https://arxiv.org/pdf/1611.04135v1.pdf "Automated Inference on Criminality using Face Images." How does stuff like this get past IRB?

hagbardceline 9y ago

I believe much of this can be dealt with, if we start RIGHT NOW to address a serious issue in our systems - the lack of a way to represent Morals and Ethics (the When and the Why) in the systems we are building. This needs to provide important input to, and thus shape the DIKW(D) pyramid. I've been doing some work with the IEEE on this - and I'm looking here on ycombinator to get some real-world feeedback on what people are thinking and concerned about. I have some (personal) ideas that might work to address the concerns I'm seeing. {NOTE Some of this is taken from a previous post I wrote (but kinda missed the thread developing, I was late I don't think anyone read it). It is useful for this thread, so a modification of that post follows.} First, I think you need a way to define 'ethics' and 'morals', with a 'ontology' and a 'epistemology' to derive a metaphysic for the system (and for my $.02, aesthetics arises from here). Until we can have a bit of rigor surrounding this, it's a challenge to discuss ethics in the context of an AI, and AI in the context of the metaphysical stance it takes towards the world. This is vital, as we need to define what 'malicious use' IS. This is still an area (as the thread demonstrates) of serious contention. Take sesame credit (a great primer, and even if you know all about it, it is still great to watch: https://www.youtube.com/watch?v=lHcTKWiZ8sI https://www.youtube.com/watch?v=lHcTKWiZ8sI ). Now here is a question for you: Is it wrong for the government to create a system that uses social pressure, rather than retributive justice or the reactive use of force, to promote social order and a 'better way of life'? Now, I'm not arguing for this (nor against for the purposes of this missive), but using it as a way to illustrate that different cultures, governmental systems, and societies, may have vastly different perspectives on the idea of a persons relationship viz a viz the state when it comes to something like privacy. I would suggest that transperancy in these decisions is a good idea. But right now we have no way to do that. I think the current way the industry is working - seemingly hell-bent on developing better, faster, more eficient, et al ways to engineer Epistemic engines and Ontologic frameworks in isolation is the root cause of the problem of malicious use. Even the analysis of potential threats (from the article referenced 'The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation' - I just skimmed it so I can keep up with this thread, please enlighten me if I'm missing something important) only pays lip service to this idea. In the executive Summary, it says: 'Promoting a culture of responsibility. AI researchers and the organisations that employ them are in a unique position to shape the security landscape of the AI-enabled world. We highlight the importance of education, ethical statements and standards, framings, norms, and expectations.' However in the 'Areas for Further research' section, I would point out that the questions are at a higher level of abstraction than the other areas, or discuss the narrative and not the problem. This might be due to the authors not having exposure to this area of research and development (such as the IEEE) - but I will concede that the fact that the note about the narrative shows that very few are aware of the work we are doing... This isn't pie-in-the-sky stuff, it has real-world use in areas other than life or death scenarios. To quickly illustrate - let's take race or gender bias (for example the Google '3 white kids' vs. '3 black kids' issue a while back in 2016). I think this is a metaphysical problem (application of Wisdom to determine correct action) that we mistake for an epistemic issue (it came from 'bad' encoding). This is another spin on kypro's concern about the consequences of AI deployment to enable the construction of a panopticon. This is about WISDOM - making wise choices - not about coding a faster epistemic engine or ontologic classifier. Next, after we get some rigor surrounding the ethical stances you consider 'good' vs. 'bad' (a vital piece that just isn't being discussed or defined) in the context of a metaphysic - you have to consider 'who' is using the system unethically. If it is the AI itself, then we have a different, but important issue - I'm going with 'you can use the AI to do wrong' as opposed to 'the AI is doing wrong' (for whatever reason, its own motivations, or it agrees with the evil or immoral users goals, perhaps, and acts in concert). Unless you have clarity here, it becomes extremely easy to befuddle, confuse, or mislead (innocently or not) questions regarding 'who'. - Who can answer for the 'Scope' or strategic context (CEO, Board of Directors, General Staff, Politburo, etc.) - Who in 'Organizational Concepts' or 'Planning' (Division Management, Program Management, Field commanders, etc) - Who in 'Architecture' or 'Schematic' (Project Management, Solution Architecture, Company commanders, etc) - Who in 'Engineering' or 'Blueprints' (Team Leaders, Chief Engineers, NCO's, etc.) - Who in 'Tools' or 'Config' (Individual contributors, Programmers, Soldiers, etc.) that constructed the AI. Then you need to ask which person, group, or combination (none dare call it conspiracy!) of these actors used the system in an unethical manner? Might 'enabled for use' be culpable as well - and is that a programmer, or an executive, or both? What I'm getting at here, is that there is both a lack of rigor in such questions (in general in this entire area), a challenge in defining ethical stances in context (which I argue requires a metaphysic), and a lack of clarity in understanding how such systems come to creation ('who' is only one interrogative that needs to be answered, after all). I would say that until and unless we have some sort of structure and standard to answer these questions, it might be beside the point to even ask... And not being able to ask leads us to some uncomfortable near-term consequences. If someone does use such a system unethically - can our system of retributive justice determine the particulars of: - where the crimes were committed (jurisdiction) - what went wrong - who to hold accountable - how it was accomplished (in a manner hopefully understandable by lawmakers, government/corporate/organizational leadership, other implementers, and one would think - the victims) - why it could be used this way - when could it happen again just for starters. The sum total of ignorance surrounding such a question points to a serious problem in how society overall - and then down to the individuals creating and using such tech - is dealing (or rather, not dealing) with this vital issue. We need to start talking along these lines in order to stake out the playing field for everyone NOW, so we actually might have time to address these things, before the alien pops right up and runs across the kitchen table.

vicpara 9y ago

I don't think these issues have anything to do with AI but 100% with humans. AI is used everywhere where money is to be made. Let's take ads. Why do we allow ads to happen all over the place (TV, radio, internet, magazines)? Because they feel stupid, harmless, how else can we sell products, etc? Well, then when a ML system is serving you ads, it's doing it so at the right time and will target you with the bang on content so that you buy exactly what it's needed. For eg. modern ML algos can pick up in which mood and mental state you are (out of 27 different states) by looking at how you read content online, what you read, how many tabs you open or how related the pages you visit are <yes, all cookie(-pool) based>. I know this because I build these things. It works incredibly well. It finds your weak spot, your passion, your habits, your indulgences and makes sure you're always tempted. It can tell more about yourself in a split second than you can think in an hour about who you really are. That bullshit you tell yourself about you doesn't matter because you are the rat the ML algo is baiting you until you bite. The problem is not that the AI is selling to the diabetic more cupcakes, the alcoholic booze and hipsters new gadgets like no other salesman. The idea of advertising is perverted itself from skin till bone marrow. How can you allow that to be done to the people? And we think this is how things are? OMFG.. AI is just a tool. WMD are another dead stupid idea only humans could come up with. Me country builds insane big weapons that can whack entire cities in split second just because it can & has more money & and we were first & we won the war & no reason. It also tries to make sure nobody can build it (like Iran) so they all kneel and kiss the ring. AI kind of levels the field. It's just a bigger nuclear bomb. My is bigger than yours taken to the next level. And that's just the beginning. AI makes it game on for more players than we like it to be. And it's not just AI (which is just math run by computers), it's mathematics, human genome research, physics, nuclear physics...all STEM subjects. You can come up with a weapon from any mix of these. There are only two ways out of it. The humanity dies or we learn how to all work together, chill our entitlement and find our common values regardless of religion, money, skin colour, gender or part of the world. A fart made in any part of the world can kill people everywhere. If humanity is to turn against itself because countries or corporations play a retarded game in AI/ pharma/ nuclear/fill_in_the_blank .. this only only accelerates a bit the process but the result will be the same. Let's ditch advertising, pharmaceuticals, OTC, bogus pills created without any fundamentals, weapon industries and all other nonsense industries or acts focused on massive profit at the cost of humanity. Let's ditch all kinds of waste and put people first for once. Let's ditch olympics where we try to prove that my country has it bigger than yours. Let's invest in education and health, inclusive societies where everyone works hard to solve humanity's tough problems. We keep building bigger guns but we as a society didn't grow much in empathy, inclusiveness or respect for the planet or other countries. People are killing people not AI. A machine gun is the safest thing in the world. No single machine gun on this universe loaded itself and started firing precisely to people or animals. Probably the best thing that can happen to this planet is to have humans vanish while there are still trees and animals around.