10 ms·

AI is going to supercharge surveillance

191 points by devy 9y ago

reitanqild 9y ago
Two scary things about this: - When you can't trust those who have access to it (including those who have access to it in the future.) IMO this means everybody who learned anything from history should be scared. - When it becomes almost 100% reliable and some unlucky person gets convicted because of a glitch but no-one cares because they trust the system.
- beobab 9y ago
  Isn't this the plot of Minority Report? :)
  - throwaway628579 9y ago
    And Person of Interest.
  - sametmax 9y ago
    And way before that, the terrifying and disturbing "brazil".
    - maxerickson 9y ago
      The Minority Report short story was written in the 1950s.
      - neaden 9y ago
        But effectively has nothing to do with the film, besides the idea that there are three precognitives who predict crime. The characters, plot, twist, etc are all unrelated.
  - tabletiptop 9y ago
    You can take the "precog" metaphor used in Minority Report and replace it what's actually happening - dragnet surveillance combined with predictive modeling algorithms to assess each individual's threat level. This is happening so openly that the DHS have done a public Kaggle competition to improve their algos for body scanners at airports. Now extrapolate a very short distance from there and think about what they're doing that's classified. Even the most unimaginative of us can see what's happening here. Over the past several years I've seen the automatic reflex response to this concern go from "well if you've got nothing to hide" to "well, I mean they're not watching me or you specifically, they don't have the man power". Well, you're right, but not for the reason you think. The fact is they don't need man power any longer. The algos are watching you.
    - ataturk 9y ago
      It is amazing how on the one hand people can detest Nazis and Soviets saying things like "I can't believe they could have been so stupid to have gone along with that" and then on the other hand, aid and abet the police state in a Kaggle competition of all things! I know it is naive of me to think that if I don't assist that it won't happen because someone will step up if for no other reason than bragging rights. A long time ago I made a decision about my career and that was that I would not work for the MIC. It was a tough choice at times because so much US industry is involved with government one way or another and even private companies have governments as their major customers so it is all intertwined. Not all government is bad, though, but the shit like body scanners and all that--that's your technocratic police state writ large. I don't imagine the future is going to be very rosy. It is top down control forever.
      - dsfyu404ed 9y ago
        The Stalin, Hilter, Mao, etc. created secret police and whatnot to enforce their continued rule. They are despised mostly for what their rule entailed, not for having secret police. We're* creating secret police but in the absence of an oppressive government for them to act on the whims of nobody much cares. It'll get worse before it gets better. * We = most nations formerly on the "right" side of the iron curtain
- pksadiq 9y ago
  > ... and some unlucky person gets convicted ... It could be worse. What if the world resources gets depleted and some people (who can control) decides to get rid of a major population? What if they act as someone said, "One way or the other, the world is going to end. So let's end it on our terms."
  - hutzlibu 9y ago
    World ressources are allways going to be depleted. 8000 years ago it was good hunting ground, that was scarce and if too many wanted it, they fought. Now with advanced technology it is some special minerals or oil. And yes there is fighting going on about it and people get killed. Lots of them. But there is no natural end to "ressources" in general. There is allways shortage of something and fight about it, but also discovery and development how to get around and get access to other resources which serve the same effect or new technology which makes the old obsolete. Doesn't mean, that there could not be a major escalation, though, but getting "rid of a major population" is not something someone smart would dare, as it can get out of control too easily. And those people in control are usually smart, otherwise they would not be in control.
  - maxerickson 9y ago
    The other poster isn't trying to lay out a worst case scenario, they are talking about harm from a likely scenario, one where most people think things are "fine".
- thfuran 9y ago
  >When it becomes almost 100% reliable and some unlucky person gets convicted because of a glitch but no-one cares because they trust the system. Wouldn't that be a massive improvement over the current state of things? Convictions are currently made on the basis of very unreliable things.
  - ams6110 9y ago
    Not really. The system certainly isn't perfect, but the vast majority of people convicted are actually guilty. Is the system vulnerable to a person being targeted, evidence fabricated, and a false conviction obtained? Yes, assuming malicious intent or corruption. I'm not sure AI fixes that.
    - TeMPOraL 9y ago
      GP is asking if a system that's almost 100% right isn't better than the current, which is somewhat, sorta right, more often than not.
  - jancsika 9y ago
    That would only be a massive improvement if you have a set of almost 100% perfectly just laws to accompany your near-perfect enforcement. I've seen a lot of things on HN, but I've never seen anyone argue that the current set of state and federal laws in the U.S. are anywhere close to 100% just. So you're going to have to factor in the cost of having perfect enforcement of unjust laws. There's also the problem of trying to curb the bad effects of perfect enforcement after the fact. Show me a politician willing to scale back some of the bad law passed post-911. That's a difficult task because nobody wants the liability of being the one who was "soft on terrorism" or whatever the claim would be. If you take those costs seriously, you'll need to take steps to ensure that the system does not in practice attempt to do perfect enforcement. And your arguments will sound more and more counter-intuitive as the capability for perfect enforcement-- and the dollars it would bring to a municipality-- grows.
- arca_vorago 9y ago
  >IMO this means everybody who learned anything from history should be scared. Which is why tptb took over education and have manipulated how it teaches history. They want us to love our upcoming neofuedal state. That's why when people talk about Orwell vs Huxley I say; It's a brave new world... until you resist. Then it's 1984.
SubiculumCode 9y ago
Automated mass surveillance has the potential to lead to even more selective, discretionary enforcement of the law...with the obvious threat to life and liberty.
- LarryMade2 9y ago
  I was just thinking there should be a note that "Artificial Intelligence" doesn't mean improved morality or reduce bias unless someone consciously builds those into a system. Likely it will be used to implement/enhance whatever goals the owner wishes, likely uses will be waste reduction (or profit maximization) in areas like manufacture, transactions, and human labor.
  - SubiculumCode 9y ago
    I figure that noting something illegal would be the limit. Prosecuting would be discretionary.
    - jackvalentine 9y ago
      The thought occurs to me if you're a DA, and build a model of every successful prosecution to decide which cases you should pursue and which you should drop you'd end up with probably a fairly racially biased model...
      - SubiculumCode 9y ago
        Mostly because one could afford good lawyers or the laws they break are harder to prove.
- mamon 9y ago
  We are kind of lucky that Moore's law does not apply anymore and computers are not getting much faster these days. It might be that such mass surveillance will never become possible, or at least will remain too expensive even for US government.
etiam 9y ago
About time this gets more mainstream coverage. The risks with this use are both far more imminent and potentially more severe than "AI" developing godhood or putting everyone out of their jobs by doing it for them. A few years back there was this reportage: http://www.newyorker.com/magazine/2015/11/23/doomsday-invention-artificial-intelligence-nick-bostrom http://www.newyorker.com/magazine/2015/11/23/doomsday-invent... It mostly focused on Nick Bostrom's concerns about "Superintelligence", but to me the more interesting passage by far is this one: The keynote speaker at the Royal Society was another Google employee: Geoffrey Hinton, who for decades has been a central figure in developing deep learning. As the conference wound down, I spotted him chatting with Bostrom in the middle of a scrum of researchers. Hinton was saying that he did not expect A.I. to be achieved for decades. “No sooner than 2070,” he said. “I am in the camp that is hopeless.” “In that you think it will not be a cause for good?” Bostrom asked. “I think political systems will use it to terrorize people,” Hinton said. Already, he believed, agencies like the N.S.A. were attempting to abuse similar technology. “Then why are you doing the research?” Bostrom asked. “I could give you the usual arguments,” Hinton said. “But the truth is that the prospect of discovery is too sweet.” He smiled awkwardly, the word hanging in the air—an echo of Oppenheimer, who famously said of the bomb, “When you see something that is technically sweet, you go ahead and do it, and you argue about what to do about it only after you have had your technical success.” As the scientists retreated to tables set up for refreshments, I asked Hinton if he believed an A.I. could be controlled. “That is like asking if a child can control his parents,” he said. “It can happen with a baby and a mother—there is biological hardwiring—but there is not a good track record of less intelligent things controlling things of greater intelligence.” He looked as if he might elaborate. Then a scientist called out, “Let’s all get drinks!”
- hutzlibu 9y ago
  I think every technology is a tool in the first place and allmost everything can be abused. And about nuclear bombs, well you can argue that they prevented World War 3. Also I think Orion-Project. (nuclear bombs as a way to accelerate a rocket in deep space) And KI I believe likewise. It is just a problem of power balance. If you americans don't trust your agencies, then you have the power to control them. But as far as I know, most common people have rather a dubious NSA who fights terrorists for them, than none or a weak NSA and terrorists everywhere. Here in Germany it's the same. So I would not focus on the technology and rather on the people behind it, if you can trust them. And if not - find new people/add transparency. To some extent this is happening in germany after some secret service apparently let nazi terrorist murder non-germans (or were extremely incompetent in failing to notice) and hindered police investigation. So now something is happening, but only because people are upset. But yes, with recent terror attacks, most still tend to view them as a necessary evil. edit: and yes, terrorists are a problem and to fight them efficient, police/secret service maybe can't play by the normal rules all the time. But on the other hand, most terrorists justify their actions, because they view themselves as attacked and exploited first. To which there is some truth. So it is a complex world. But I don't think it is doomed because of KI ...
Puer 9y ago
AI is further enabling oppression in China which is already arguably the world's largest surveillance state. For example, using AI and the government's ID database, crosswalk cameras are now able to facially identify "chronic j-walkers." [1] The concerning part of this is the implication that these profiles don't go away. In the status quo, watch lists already exist, but I'm afraid that as AI technology becomes more accessible it will only further increase abuse and discrimination against "potential offenders" identified by the system. [1] https://www.wsj.com/articles/the-all-seeing-surveillance-state-feared-in-the-west-is-a-reality-in-china-1498493020 https://www.wsj.com/articles/the-all-seeing-surveillance-sta...
- est 9y ago
  Chinese here. AI makes social change in a really strange way. The story everyone knows, China is a authoritarian state, the gov't controls every aspect of citizen's life. But somethings strange starts appear. You know, future CPC leaders, no matter how powerful/rich their parents are, they will be grown up in a highly supervised social environment. They are forced to think and act more transparently because everyone is in the system. A random offensive act against the Party may put a citizen in jail. It also means a minor speech flaw made by a political figure leads to career suicide. In the paste decades, they major problem is that CPC has unlimited power, no matter what bullshit they pull off, they always get away with it. Now with cameras and big-data shit, no matter what you want to cover, you always leave a digital trace behind. Someone at sometime, no matter citizens or your enemey at opposing political group will find it and exploit it against you. In this "everyone's been watched" game, unless a dedicated, organized backdoor exists, everyone will be forced to comply with some kind "rule" or equilibrium, which is far different than today's arbitrary enforced laws in China. Gov't today use keywords to monitor citizen online activities, but I've also seen a monitor program to evaluate Party member's loyalty based on cognitive science and machine learning. Technology accelerates all of these. I foresee China will go through a radical, soviet style social paradigm shift, but nothing like we've ever seen before.
  - jacquesm 9y ago
    The rich and the powerful somehow always find ways to be exempted from oversight.
3pt14159 9y ago
People misunderstand the dangers of AI. The danger isn't (or isn't yet anyway) that AI is going to develop will and agency and oppose us. The danger is that AI is a force multiplier in whatever you do because it allows the intelligent to automate formerly rote tasks. For example, take self-driving cars. Formerly all you could do if you had a remote execution exploit was smash running cars into ditches or oncoming traffic. Bad, but not the end of the world. Now you can take 200k Teslas and very easily target every gas station or other soft target in the developed world. (I'm actually concerned about this, see https://www.zachaysan.com/cars https://www.zachaysan.com/cars for an, admittedly long, take on the matter.) The same is true with AI everywhere. Don't read individual Twitter accounts, scan them all, build a data model to target the influenceable influencers. Iterate. Don't try to get your honeypot to date the head of Goldman Sachs, use data science to figure out which 1000 25 year olds have a good shot at heading it up in 10 years. The cyber game / international relations is about to get much more extreme. Offence is so much easier than defence and AI is turning every little touch of exhaust data or device into a tool of war or espionage.
- 1024core 9y ago
  Exactly. We're not going to get sentient, self-replicating, anti-human robots with weapons anytime soon. What we will get is the equivalent of a rifle in the age of swords. Maybe a machine gun, even. I have an Android device. Google's face recognition is so good, that it was able to correctly identify people in photographs in very degraded circumstances (occluded faces, sideways, etc.). It's only a matter of time before this shit is used to track people in public places, allowing "law enforcement" to track everyone. Were you in the Women's March last weekend? Well, congratulations! You've won the "extra screening at the airport" lottery.
  - Tharkun 9y ago
    > Were you in the Women's March last weekend? Well, congratulations! You've won the "extra screening at the airport" lottery. This is already happening. In the UK 'forward intelligence teams' (FIT) have been applying these tactics for year. There used to be a blog about this, FIT-watch dot something, don't have the UR' on me.
hishnash 9y ago
What I hope they do is use many instances of the system trained with slightly different sets of training data, otherwise, you just need to learn how to trick one version (and it will be possible) and you can trick them all. When attacking human-managed systems they may be slow to respond, weak and unable to detect everything. But they are also weak in very different ways and will respond in different ways.
tboyd47 9y ago
No AI is more dangerous to society than mass ethnically motivated hatred. The cruelty of the human population is racing ahead much faster than AI.
- ravenstine 9y ago
  How?
  - tboyd47 9y ago
    The Rohingya crisis comes to mind. I can't think of another time in my life I've heard of a million people being ejected from their own country in less than a year. It's not worse than the Holocaust (although maybe it would be if you look at rate instead of sum), but this is supposed to be peacetime. The army didn't need AI for that; it just needed to hand out weapons and give the order. And let's not even discuss the actual stories the refugees are telling. What's going on in Congo is probably even worse, but we don't hear about it, yet the numbers of refugees are comparable. Stuff like that is what worries me in 2018. Not AI.
    - ralusek 9y ago
      Is this like an annual resolution where we just choose one thing to be worried about?
- NoGravitas 9y ago
  How about AI used as a force multiplier for mass ethnically motivated hatred? Say, an AR app with face recognition connected to social media and government records to identify members of $HATED_ETHNIC_GROUP?
  - tboyd47 9y ago
    It doesn't require deep learning or AR to know what race a person is. What's stopping militaries and police around the world from carrying out large-scale ethnic cleansing already? We act like we (techies) are the ones who enable states to carry out these atrocities by inventing gadgets, but they really don't need our gadgets. If they don't have drones and AI, they can do the same thing with machetes and walkie-talkies.
EGreg 9y ago
Someone should do a harmless stunt to snap the world to attention of what COULD have been possible if AI was out to do something malicious. That would set the tone differently before it's too late.
- tachyonbeam 9y ago
  I think it shouldn't be too hard to cross-reference anonymous accounts on HN and reddit based on writing style, for one thing. Could build a machine that gathers all the data you can find about people online, and e-mails said people that data.
  - sova 9y ago
    Allegedly the Dept. of State knows the identity(ies) of S. Nakamoto thanks to a similar technique.
InclinedPlane 9y ago
No duh. You think things are bad now? Imagine what happens if you could process and cross reference every available photograph or piece of video possible. Think face recognition is scary? Think about gait analysis. Computers can recognize you based on biometrics such as how you walk. Think about a database that collects up your entire inventory of clothing. Imagine every piece of evidence analyzed to a Sherlock level of detail. Imagine a picture of a part of you that doesn't show your face where you are recognized based on your smart watch, your arm hair, your moles, your clothing, who else is in the picture (the friends you associate with), the location (your friend's house where you hang out a lot, and a location near where you were known to be recently based on other information (social media checkins, etc.)), and so on. The amount of information that is out there already is enormous, but most of it is latent, not cataloged, not indexed, not cross-referenced. Once that process starts it will accelerate rapidly. And who will be in control of such data? Anyone with the resources to crunch it. If some corporation is doing the analysis then it might be up for sale to anyone as well.
- hacker_9 9y ago
  Dude have you not heard of fake news? You only have to flood the system with lies and half truths to make it unreliable. This sort of stuff has been done for 1000s of years...
ravitation 9y ago
I find this title amusing... It's like writing "computing is going to supercharge surveillance" in 1960 or "the internet is going to supercharge surveillance" in 1990. A technology that will "supercharge" everything will also supercharge this specific thing... I'm not saying it isn't valuable to point out the potential negatives of a technology like this, I'm just saying it's a little amusing...
seanavery 9y ago
I would be excited to see an unsupervised facial recognition classifier with an open API. To a certain extent facebook login with a third of the world classified already accomplishes this... but a better (and more direct api) where you send video --> returns top 5 probabilities from softmax output would be cool. Each app or api user can set their own security threshold (.9, .99, .999)
jhiska 9y ago
It's a calming, reassuring article about something that will destroy people's lives and societies in unpredictable ways and disrupt the balance of power for worse. At least there's always the soothing "solution" of "regulating it". The people who get to "regulate it" are exactly the ones who will wield its power. After some millions of us have died the other people who survive will eventually find a new equilibrium.
rasengan0 9y ago
Remember that meme, "Don't tase me, bro!" ? Now you can't even see who is targeting you. Golly Mabel, can AI be used to supercharge finding ... https://news.ycombinator.com/item?id=16255850 https://news.ycombinator.com/item?id=16255850 ? All this neutral technology open to the highest bidder. Augmenting the power of the few against the many.

191 points by devy 9y ago

Two scary things about this: - When you can't trust those who have access to it (including those who have access to it in the future.) IMO this means everybody who learned anything from history should be scared. - When it becomes almost 100% reliable and some unlucky person gets convicted because of a glitch but no-one cares because they trust the system.

beobab 9y ago
Isn't this the plot of Minority Report? :)
- throwaway628579 9y ago
  And Person of Interest.
- sametmax 9y ago
  And way before that, the terrifying and disturbing "brazil".
  - maxerickson 9y ago
    The Minority Report short story was written in the 1950s.
    - neaden 9y ago
      But effectively has nothing to do with the film, besides the idea that there are three precognitives who predict crime. The characters, plot, twist, etc are all unrelated.
- tabletiptop 9y ago
  You can take the "precog" metaphor used in Minority Report and replace it what's actually happening - dragnet surveillance combined with predictive modeling algorithms to assess each individual's threat level. This is happening so openly that the DHS have done a public Kaggle competition to improve their algos for body scanners at airports. Now extrapolate a very short distance from there and think about what they're doing that's classified. Even the most unimaginative of us can see what's happening here. Over the past several years I've seen the automatic reflex response to this concern go from "well if you've got nothing to hide" to "well, I mean they're not watching me or you specifically, they don't have the man power". Well, you're right, but not for the reason you think. The fact is they don't need man power any longer. The algos are watching you.
  - ataturk 9y ago
    It is amazing how on the one hand people can detest Nazis and Soviets saying things like "I can't believe they could have been so stupid to have gone along with that" and then on the other hand, aid and abet the police state in a Kaggle competition of all things! I know it is naive of me to think that if I don't assist that it won't happen because someone will step up if for no other reason than bragging rights. A long time ago I made a decision about my career and that was that I would not work for the MIC. It was a tough choice at times because so much US industry is involved with government one way or another and even private companies have governments as their major customers so it is all intertwined. Not all government is bad, though, but the shit like body scanners and all that--that's your technocratic police state writ large. I don't imagine the future is going to be very rosy. It is top down control forever.
    - dsfyu404ed 9y ago
      The Stalin, Hilter, Mao, etc. created secret police and whatnot to enforce their continued rule. They are despised mostly for what their rule entailed, not for having secret police. We're* creating secret police but in the absence of an oppressive government for them to act on the whims of nobody much cares. It'll get worse before it gets better. * We = most nations formerly on the "right" side of the iron curtain
pksadiq 9y ago
> ... and some unlucky person gets convicted ... It could be worse. What if the world resources gets depleted and some people (who can control) decides to get rid of a major population? What if they act as someone said, "One way or the other, the world is going to end. So let's end it on our terms."
- hutzlibu 9y ago
  World ressources are allways going to be depleted. 8000 years ago it was good hunting ground, that was scarce and if too many wanted it, they fought. Now with advanced technology it is some special minerals or oil. And yes there is fighting going on about it and people get killed. Lots of them. But there is no natural end to "ressources" in general. There is allways shortage of something and fight about it, but also discovery and development how to get around and get access to other resources which serve the same effect or new technology which makes the old obsolete. Doesn't mean, that there could not be a major escalation, though, but getting "rid of a major population" is not something someone smart would dare, as it can get out of control too easily. And those people in control are usually smart, otherwise they would not be in control.
- maxerickson 9y ago
  The other poster isn't trying to lay out a worst case scenario, they are talking about harm from a likely scenario, one where most people think things are "fine".
thfuran 9y ago
>When it becomes almost 100% reliable and some unlucky person gets convicted because of a glitch but no-one cares because they trust the system. Wouldn't that be a massive improvement over the current state of things? Convictions are currently made on the basis of very unreliable things.
- ams6110 9y ago
  Not really. The system certainly isn't perfect, but the vast majority of people convicted are actually guilty. Is the system vulnerable to a person being targeted, evidence fabricated, and a false conviction obtained? Yes, assuming malicious intent or corruption. I'm not sure AI fixes that.
  - TeMPOraL 9y ago
    GP is asking if a system that's almost 100% right isn't better than the current, which is somewhat, sorta right, more often than not.
- jancsika 9y ago
  That would only be a massive improvement if you have a set of almost 100% perfectly just laws to accompany your near-perfect enforcement. I've seen a lot of things on HN, but I've never seen anyone argue that the current set of state and federal laws in the U.S. are anywhere close to 100% just. So you're going to have to factor in the cost of having perfect enforcement of unjust laws. There's also the problem of trying to curb the bad effects of perfect enforcement after the fact. Show me a politician willing to scale back some of the bad law passed post-911. That's a difficult task because nobody wants the liability of being the one who was "soft on terrorism" or whatever the claim would be. If you take those costs seriously, you'll need to take steps to ensure that the system does not in practice attempt to do perfect enforcement. And your arguments will sound more and more counter-intuitive as the capability for perfect enforcement-- and the dollars it would bring to a municipality-- grows.
arca_vorago 9y ago
>IMO this means everybody who learned anything from history should be scared. Which is why tptb took over education and have manipulated how it teaches history. They want us to love our upcoming neofuedal state. That's why when people talk about Orwell vs Huxley I say; It's a brave new world... until you resist. Then it's 1984.

SubiculumCode 9y ago

Automated mass surveillance has the potential to lead to even more selective, discretionary enforcement of the law...with the obvious threat to life and liberty.

LarryMade2 9y ago
I was just thinking there should be a note that "Artificial Intelligence" doesn't mean improved morality or reduce bias unless someone consciously builds those into a system. Likely it will be used to implement/enhance whatever goals the owner wishes, likely uses will be waste reduction (or profit maximization) in areas like manufacture, transactions, and human labor.
- SubiculumCode 9y ago
  I figure that noting something illegal would be the limit. Prosecuting would be discretionary.
  - jackvalentine 9y ago
    The thought occurs to me if you're a DA, and build a model of every successful prosecution to decide which cases you should pursue and which you should drop you'd end up with probably a fairly racially biased model...
    - SubiculumCode 9y ago
      Mostly because one could afford good lawyers or the laws they break are harder to prove.
mamon 9y ago
We are kind of lucky that Moore's law does not apply anymore and computers are not getting much faster these days. It might be that such mass surveillance will never become possible, or at least will remain too expensive even for US government.

etiam 9y ago

About time this gets more mainstream coverage. The risks with this use are both far more imminent and potentially more severe than "AI" developing godhood or putting everyone out of their jobs by doing it for them. A few years back there was this reportage: http://www.newyorker.com/magazine/2015/11/23/doomsday-invention-artificial-intelligence-nick-bostrom http://www.newyorker.com/magazine/2015/11/23/doomsday-invent... It mostly focused on Nick Bostrom's concerns about "Superintelligence", but to me the more interesting passage by far is this one: The keynote speaker at the Royal Society was another Google employee: Geoffrey Hinton, who for decades has been a central figure in developing deep learning. As the conference wound down, I spotted him chatting with Bostrom in the middle of a scrum of researchers. Hinton was saying that he did not expect A.I. to be achieved for decades. “No sooner than 2070,” he said. “I am in the camp that is hopeless.” “In that you think it will not be a cause for good?” Bostrom asked. “I think political systems will use it to terrorize people,” Hinton said. Already, he believed, agencies like the N.S.A. were attempting to abuse similar technology. “Then why are you doing the research?” Bostrom asked. “I could give you the usual arguments,” Hinton said. “But the truth is that the prospect of discovery is too sweet.” He smiled awkwardly, the word hanging in the air—an echo of Oppenheimer, who famously said of the bomb, “When you see something that is technically sweet, you go ahead and do it, and you argue about what to do about it only after you have had your technical success.” As the scientists retreated to tables set up for refreshments, I asked Hinton if he believed an A.I. could be controlled. “That is like asking if a child can control his parents,” he said. “It can happen with a baby and a mother—there is biological hardwiring—but there is not a good track record of less intelligent things controlling things of greater intelligence.” He looked as if he might elaborate. Then a scientist called out, “Let’s all get drinks!”

hutzlibu 9y ago
I think every technology is a tool in the first place and allmost everything can be abused. And about nuclear bombs, well you can argue that they prevented World War 3. Also I think Orion-Project. (nuclear bombs as a way to accelerate a rocket in deep space) And KI I believe likewise. It is just a problem of power balance. If you americans don't trust your agencies, then you have the power to control them. But as far as I know, most common people have rather a dubious NSA who fights terrorists for them, than none or a weak NSA and terrorists everywhere. Here in Germany it's the same. So I would not focus on the technology and rather on the people behind it, if you can trust them. And if not - find new people/add transparency. To some extent this is happening in germany after some secret service apparently let nazi terrorist murder non-germans (or were extremely incompetent in failing to notice) and hindered police investigation. So now something is happening, but only because people are upset. But yes, with recent terror attacks, most still tend to view them as a necessary evil. edit: and yes, terrorists are a problem and to fight them efficient, police/secret service maybe can't play by the normal rules all the time. But on the other hand, most terrorists justify their actions, because they view themselves as attacked and exploited first. To which there is some truth. So it is a complex world. But I don't think it is doomed because of KI ...

Puer 9y ago

AI is further enabling oppression in China which is already arguably the world's largest surveillance state. For example, using AI and the government's ID database, crosswalk cameras are now able to facially identify "chronic j-walkers." [1] The concerning part of this is the implication that these profiles don't go away. In the status quo, watch lists already exist, but I'm afraid that as AI technology becomes more accessible it will only further increase abuse and discrimination against "potential offenders" identified by the system. [1] https://www.wsj.com/articles/the-all-seeing-surveillance-state-feared-in-the-west-is-a-reality-in-china-1498493020 https://www.wsj.com/articles/the-all-seeing-surveillance-sta...

est 9y ago
Chinese here. AI makes social change in a really strange way. The story everyone knows, China is a authoritarian state, the gov't controls every aspect of citizen's life. But somethings strange starts appear. You know, future CPC leaders, no matter how powerful/rich their parents are, they will be grown up in a highly supervised social environment. They are forced to think and act more transparently because everyone is in the system. A random offensive act against the Party may put a citizen in jail. It also means a minor speech flaw made by a political figure leads to career suicide. In the paste decades, they major problem is that CPC has unlimited power, no matter what bullshit they pull off, they always get away with it. Now with cameras and big-data shit, no matter what you want to cover, you always leave a digital trace behind. Someone at sometime, no matter citizens or your enemey at opposing political group will find it and exploit it against you. In this "everyone's been watched" game, unless a dedicated, organized backdoor exists, everyone will be forced to comply with some kind "rule" or equilibrium, which is far different than today's arbitrary enforced laws in China. Gov't today use keywords to monitor citizen online activities, but I've also seen a monitor program to evaluate Party member's loyalty based on cognitive science and machine learning. Technology accelerates all of these. I foresee China will go through a radical, soviet style social paradigm shift, but nothing like we've ever seen before.
- jacquesm 9y ago
  The rich and the powerful somehow always find ways to be exempted from oversight.

3pt14159 9y ago

People misunderstand the dangers of AI. The danger isn't (or isn't yet anyway) that AI is going to develop will and agency and oppose us. The danger is that AI is a force multiplier in whatever you do because it allows the intelligent to automate formerly rote tasks. For example, take self-driving cars. Formerly all you could do if you had a remote execution exploit was smash running cars into ditches or oncoming traffic. Bad, but not the end of the world. Now you can take 200k Teslas and very easily target every gas station or other soft target in the developed world. (I'm actually concerned about this, see https://www.zachaysan.com/cars https://www.zachaysan.com/cars for an, admittedly long, take on the matter.) The same is true with AI everywhere. Don't read individual Twitter accounts, scan them all, build a data model to target the influenceable influencers. Iterate. Don't try to get your honeypot to date the head of Goldman Sachs, use data science to figure out which 1000 25 year olds have a good shot at heading it up in 10 years. The cyber game / international relations is about to get much more extreme. Offence is so much easier than defence and AI is turning every little touch of exhaust data or device into a tool of war or espionage.

1024core 9y ago
Exactly. We're not going to get sentient, self-replicating, anti-human robots with weapons anytime soon. What we will get is the equivalent of a rifle in the age of swords. Maybe a machine gun, even. I have an Android device. Google's face recognition is so good, that it was able to correctly identify people in photographs in very degraded circumstances (occluded faces, sideways, etc.). It's only a matter of time before this shit is used to track people in public places, allowing "law enforcement" to track everyone. Were you in the Women's March last weekend? Well, congratulations! You've won the "extra screening at the airport" lottery.
- Tharkun 9y ago
  > Were you in the Women's March last weekend? Well, congratulations! You've won the "extra screening at the airport" lottery. This is already happening. In the UK 'forward intelligence teams' (FIT) have been applying these tactics for year. There used to be a blog about this, FIT-watch dot something, don't have the UR' on me.

hishnash 9y ago

What I hope they do is use many instances of the system trained with slightly different sets of training data, otherwise, you just need to learn how to trick one version (and it will be possible) and you can trick them all. When attacking human-managed systems they may be slow to respond, weak and unable to detect everything. But they are also weak in very different ways and will respond in different ways.

tboyd47 9y ago

No AI is more dangerous to society than mass ethnically motivated hatred. The cruelty of the human population is racing ahead much faster than AI.

ravenstine 9y ago
How?
- tboyd47 9y ago
  The Rohingya crisis comes to mind. I can't think of another time in my life I've heard of a million people being ejected from their own country in less than a year. It's not worse than the Holocaust (although maybe it would be if you look at rate instead of sum), but this is supposed to be peacetime. The army didn't need AI for that; it just needed to hand out weapons and give the order. And let's not even discuss the actual stories the refugees are telling. What's going on in Congo is probably even worse, but we don't hear about it, yet the numbers of refugees are comparable. Stuff like that is what worries me in 2018. Not AI.
  - ralusek 9y ago
    Is this like an annual resolution where we just choose one thing to be worried about?
NoGravitas 9y ago
How about AI used as a force multiplier for mass ethnically motivated hatred? Say, an AR app with face recognition connected to social media and government records to identify members of $HATED_ETHNIC_GROUP?
- tboyd47 9y ago
  It doesn't require deep learning or AR to know what race a person is. What's stopping militaries and police around the world from carrying out large-scale ethnic cleansing already? We act like we (techies) are the ones who enable states to carry out these atrocities by inventing gadgets, but they really don't need our gadgets. If they don't have drones and AI, they can do the same thing with machetes and walkie-talkies.

EGreg 9y ago

Someone should do a harmless stunt to snap the world to attention of what COULD have been possible if AI was out to do something malicious. That would set the tone differently before it's too late.

tachyonbeam 9y ago
I think it shouldn't be too hard to cross-reference anonymous accounts on HN and reddit based on writing style, for one thing. Could build a machine that gathers all the data you can find about people online, and e-mails said people that data.
- sova 9y ago
  Allegedly the Dept. of State knows the identity(ies) of S. Nakamoto thanks to a similar technique.

InclinedPlane 9y ago

No duh. You think things are bad now? Imagine what happens if you could process and cross reference every available photograph or piece of video possible. Think face recognition is scary? Think about gait analysis. Computers can recognize you based on biometrics such as how you walk. Think about a database that collects up your entire inventory of clothing. Imagine every piece of evidence analyzed to a Sherlock level of detail. Imagine a picture of a part of you that doesn't show your face where you are recognized based on your smart watch, your arm hair, your moles, your clothing, who else is in the picture (the friends you associate with), the location (your friend's house where you hang out a lot, and a location near where you were known to be recently based on other information (social media checkins, etc.)), and so on. The amount of information that is out there already is enormous, but most of it is latent, not cataloged, not indexed, not cross-referenced. Once that process starts it will accelerate rapidly. And who will be in control of such data? Anyone with the resources to crunch it. If some corporation is doing the analysis then it might be up for sale to anyone as well.

hacker_9 9y ago
Dude have you not heard of fake news? You only have to flood the system with lies and half truths to make it unreliable. This sort of stuff has been done for 1000s of years...

ravitation 9y ago

I find this title amusing... It's like writing "computing is going to supercharge surveillance" in 1960 or "the internet is going to supercharge surveillance" in 1990. A technology that will "supercharge" everything will also supercharge this specific thing... I'm not saying it isn't valuable to point out the potential negatives of a technology like this, I'm just saying it's a little amusing...

seanavery 9y ago

I would be excited to see an unsupervised facial recognition classifier with an open API. To a certain extent facebook login with a third of the world classified already accomplishes this... but a better (and more direct api) where you send video --> returns top 5 probabilities from softmax output would be cool. Each app or api user can set their own security threshold (.9, .99, .999)

jhiska 9y ago

It's a calming, reassuring article about something that will destroy people's lives and societies in unpredictable ways and disrupt the balance of power for worse. At least there's always the soothing "solution" of "regulating it". The people who get to "regulate it" are exactly the ones who will wield its power. After some millions of us have died the other people who survive will eventually find a new equilibrium.

rasengan0 9y ago

Remember that meme, "Don't tase me, bro!" ? Now you can't even see who is targeting you. Golly Mabel, can AI be used to supercharge finding ... https://news.ycombinator.com/item?id=16255850 https://news.ycombinator.com/item?id=16255850 ? All this neutral technology open to the highest bidder. Augmenting the power of the few against the many.