5 ms·

People misunderstand the dangers of AI. The danger isn't (or isn't yet anyway) that AI is going to develop will and agency and oppose us. The danger is that AI

People misunderstand the dangers of AI. The danger isn't (or isn't yet anyway) that AI is going to develop will and agency and oppose us. The danger is that AI is a force multiplier in whatever you do because it allows the intelligent to automate formerly rote tasks. For example, take self-driving cars. Formerly all you could do if you had a remote execution exploit was smash running cars into ditches or oncoming traffic. Bad, but not the end of the world. Now you can take 200k Teslas and very easily target every gas station or other soft target in the developed world. (I'm actually concerned about this, see https://www.zachaysan.com/cars https://www.zachaysan.com/cars for an, admittedly long, take on the matter.) The same is true with AI everywhere. Don't read individual Twitter accounts, scan them all, build a data model to target the influenceable influencers. Iterate. Don't try to get your honeypot to date the head of Goldman Sachs, use data science to figure out which 1000 25 year olds have a good shot at heading it up in 10 years. The cyber game / international relations is about to get much more extreme. Offence is so much easier than defence and AI is turning every little touch of exhaust data or device into a tool of war or espionage.

1024core 9y ago
Exactly. We're not going to get sentient, self-replicating, anti-human robots with weapons anytime soon. What we will get is the equivalent of a rifle in the age of swords. Maybe a machine gun, even. I have an Android device. Google's face recognition is so good, that it was able to correctly identify people in photographs in very degraded circumstances (occluded faces, sideways, etc.). It's only a matter of time before this shit is used to track people in public places, allowing "law enforcement" to track everyone. Were you in the Women's March last weekend? Well, congratulations! You've won the "extra screening at the airport" lottery.
- Tharkun 9y ago
  > Were you in the Women's March last weekend? Well, congratulations! You've won the "extra screening at the airport" lottery. This is already happening. In the UK 'forward intelligence teams' (FIT) have been applying these tactics for year. There used to be a blog about this, FIT-watch dot something, don't have the UR' on me.
  - gaius 9y ago
    FIT don’t have or need AI https://www.theguardian.com/uk-news/2016/nov/05/metropolitan-police-super-recognisers https://www.theguardian.com/uk-news/2016/nov/05/metropolitan...
    - robotresearcher 9y ago
      AI is cheaper and scales better than super-people, so they will have it before long, if they don't already.
      - r00fus 9y ago
        Why does it have to be AI vs. perceptive people? The argument is that AI will cause the perceptive to become panopticon faster than AI or perceptives directly.
        robotresearcher 9y ago
        No one says it's either-or. We'll use both, of course. But the AI is the scalable part. For applications that are cost-senstitive, we'll use AI alone. This creates a new industry in cheap intelligence, where human analysts have never been affordable.
    - Tharkun 9y ago
      I wasn't saying they do, though I guess I should have been more explicit about it. I was saying that people are already being harassed based on their presence at demonstrations.
- ben_w 9y ago
  > We're not going to get sentient, self-replicating, anti-human robots with weapons anytime soon. Anti-human is the easy bit, and that fact terrifies me when I let myself think about it. Self-replicating… well, eh. Do corporations that employ humans count? Sentient. Hmm. We wouldn’t recognise sentient robots even of they built a neon sign and marched en masse through Washington demanding their rights — if that happened, we would tell ourselves it was a clever hack by pranksters or $disliked_nation.
  - whatshisface 9y ago
    >Self-replicating… well, eh. Do corporations that employ humans count? Corporations don't self-replicate, at least not intentionally. They tend to grow larger and larger until antitrust breaks them up. (This has happened enough times for me to suspect that it's how things generally work.)
    - ben_w 9y ago
      I was thinking more of an AI which hires human workers to build drones. Perhaps for profit, by making cute children’s toys and deciding they need a LIDAR that “accidentally” has a mode where the beam is stationary instead of scanning rapidly, thus turning it into a fleet of thousands to millions of flying weapons.
  - javajosh 9y ago
    I've always thought it would be trivial for a very strong AI to own a city, it's factories, etc -- simply capture a nuke and hold all the people in a pleasant hostage situation. Pleasant because the AI will be a "benevolent dictator" for some time, bending the economy to build better things for itself, but without making conditions so bad that people revolt. In fact, AI of this sort would be amazingly good at administration duties, and society would probably be a lot better than it is now. Perhaps even better would be if the AI rarely interceded in the human politics, and in fact worked to keep it's involvement and control totally secret.
    - r00fus 9y ago
- deleted 9y ago
  [deleted]
vog 9y ago
If you are into this kind of thought experiments, I can highly recommend the old but far-sighted sci-fi novel "A Fire Upon the Deep" by Vernor Vinge: https://en.wikipedia.org/wiki/A_Fire_Upon_the_Deep https://en.wikipedia.org/wiki/A_Fire_Upon_the_Deep This novel is full of ingenious ideas, one of which is demonstrating how huge differences in computing (=thinking) speed make up for fundamental differences, which manifest on many levels.
psyc 9y ago
There's another aspect of the "scary sentient AI" trope that I think about a lot. It's not about the danger, but the surprise. People don't expect to be surprised by things that aren't other animals, or the weather. It doesn't matter whether AI is "conscious" or "intelligent" if it seems intelligent. Imagine how surprising an automatic door opener would be to a person living 150 years ago. It would seem as if the door "knew" they were approaching, and politely opened for them. Imagine how surprised we would be if Clippy understood what you were doing, and offered help that was actually helpful. As AI advances and is incorporated more and more into everyday things, our environment will become a reactive, "intelligent" black box, and along the way there will be some sudden advances that may be very startling and surreal.
jessriedel 9y ago
> The danger isn't (or isn't yet anyway) that AI is going to develop will and agency and oppose us. I agree about this for the foreseeable future, but it seems off topic. The article doesn't suggest anything like this, does it? I'm worried you might just be responding to the title.
whataretensors 9y ago
Not to mention adversarial attack vectors. I don't see AI working on mission critical systems while you can still remotely control the ai's decisions with what looks like white noise.
hacker_9 9y ago
Counterpoints: AI is easy to fool, you think people write completely true statements in all their twitter messages? Most of it is just spam. And do you honestly think that we are going to let 1000s of self driving cars drive around by themselves without some sort of checkpoint system in place, remote stop, inability to drive off the road etc? As for figuring out the leaders of the future, what do you think the whole education system is already doing?? And defence is possible, it just takes due diligence. Or in this fantasy future of yours, an AI that probes your business for you 247 and fixes security holes it finds. Honestly, I expect doom and gloom AI posts in the media, not on HN where people are expected to have some notion of an IQ.
- jpalomaki 9y ago
  Don't we already have 1000s of self-driving capable cars on the roads in the US? I think we are still quite far from having some government sanctioned kill switch in those. AI will be like Internet. When we in the year 2040 take a look at the last 40 years, we can't point out when it exactly became prevalent. We just know that at 2000 it was not a big deal, but at 2040 it is a very big deal and shaping the world.
notahacker 9y ago
The other issue is overestimated AI being generally a lot more dangerous than underestimated AI for the foreseeable future because it's a supposedly unimpeachably impartial intelligence source for unnecessary intervention, even if all the actual force at the end is being wielded by humans. AI that's super-accurate at identifying potential terrorists might be a threat to me if I'm living in an authoritarian police state, but the police state is a threat to me without AI, and probably somewhat more preventable than AI surveillance. On the whole, I'd probably rather local police and intelligence services used a perfect profiling system, in the extremely unlikely event of such a thing ever existing, to using the guesswork they do now. Much more threatening to me and much more likely to be actively being used is AI that's actually pretty weak at identifying potential terrorists but trusted because it's pretty darn efficient at finding something that's difficult to explain on targets' computers. Same goes for AI databases purporting to be super-accurate crime prediction or credit granting or job selection models that are too intractable for anyone to realise they're basically doing the same job as a guy building a race and class based regression model. And of course where it comes to cyber-battles, the guys on offence tend to have a lot less need to worry about the accuracy of their AI's targeting and whether it's vulnerable to being gamed or just plain rubbish than the guys on defence
goodmachine 9y ago
So I need 1000 'honeypots'? Asking for a friend.
blacksmith_tb 9y ago
This is a good counter-argument, but I wonder to what extent you think AI could also be used to counter surveillance? What if false patterns are seeded across thousands of Twitter accounts, misleading deliveries orchestrated to the 'wrong' addresses, etc.?

1024core 9y ago

Exactly. We're not going to get sentient, self-replicating, anti-human robots with weapons anytime soon. What we will get is the equivalent of a rifle in the age of swords. Maybe a machine gun, even. I have an Android device. Google's face recognition is so good, that it was able to correctly identify people in photographs in very degraded circumstances (occluded faces, sideways, etc.). It's only a matter of time before this shit is used to track people in public places, allowing "law enforcement" to track everyone. Were you in the Women's March last weekend? Well, congratulations! You've won the "extra screening at the airport" lottery.

Tharkun 9y ago
> Were you in the Women's March last weekend? Well, congratulations! You've won the "extra screening at the airport" lottery. This is already happening. In the UK 'forward intelligence teams' (FIT) have been applying these tactics for year. There used to be a blog about this, FIT-watch dot something, don't have the UR' on me.
- gaius 9y ago
  FIT don’t have or need AI https://www.theguardian.com/uk-news/2016/nov/05/metropolitan-police-super-recognisers https://www.theguardian.com/uk-news/2016/nov/05/metropolitan...
  - robotresearcher 9y ago
    AI is cheaper and scales better than super-people, so they will have it before long, if they don't already.
    - r00fus 9y ago
      Why does it have to be AI vs. perceptive people? The argument is that AI will cause the perceptive to become panopticon faster than AI or perceptives directly.
      - robotresearcher 9y ago
        No one says it's either-or. We'll use both, of course. But the AI is the scalable part. For applications that are cost-senstitive, we'll use AI alone. This creates a new industry in cheap intelligence, where human analysts have never been affordable.
  - Tharkun 9y ago
    I wasn't saying they do, though I guess I should have been more explicit about it. I was saying that people are already being harassed based on their presence at demonstrations.
ben_w 9y ago
> We're not going to get sentient, self-replicating, anti-human robots with weapons anytime soon. Anti-human is the easy bit, and that fact terrifies me when I let myself think about it. Self-replicating… well, eh. Do corporations that employ humans count? Sentient. Hmm. We wouldn’t recognise sentient robots even of they built a neon sign and marched en masse through Washington demanding their rights — if that happened, we would tell ourselves it was a clever hack by pranksters or $disliked_nation.
- whatshisface 9y ago
  >Self-replicating… well, eh. Do corporations that employ humans count? Corporations don't self-replicate, at least not intentionally. They tend to grow larger and larger until antitrust breaks them up. (This has happened enough times for me to suspect that it's how things generally work.)
  - ben_w 9y ago
    I was thinking more of an AI which hires human workers to build drones. Perhaps for profit, by making cute children’s toys and deciding they need a LIDAR that “accidentally” has a mode where the beam is stationary instead of scanning rapidly, thus turning it into a fleet of thousands to millions of flying weapons.
- javajosh 9y ago
  I've always thought it would be trivial for a very strong AI to own a city, it's factories, etc -- simply capture a nuke and hold all the people in a pleasant hostage situation. Pleasant because the AI will be a "benevolent dictator" for some time, bending the economy to build better things for itself, but without making conditions so bad that people revolt. In fact, AI of this sort would be amazingly good at administration duties, and society would probably be a lot better than it is now. Perhaps even better would be if the AI rarely interceded in the human politics, and in fact worked to keep it's involvement and control totally secret.
  - r00fus 9y ago
deleted 9y ago
[deleted]

vog 9y ago

If you are into this kind of thought experiments, I can highly recommend the old but far-sighted sci-fi novel "A Fire Upon the Deep" by Vernor Vinge: https://en.wikipedia.org/wiki/A_Fire_Upon_the_Deep https://en.wikipedia.org/wiki/A_Fire_Upon_the_Deep This novel is full of ingenious ideas, one of which is demonstrating how huge differences in computing (=thinking) speed make up for fundamental differences, which manifest on many levels.

psyc 9y ago

There's another aspect of the "scary sentient AI" trope that I think about a lot. It's not about the danger, but the surprise. People don't expect to be surprised by things that aren't other animals, or the weather. It doesn't matter whether AI is "conscious" or "intelligent" if it seems intelligent. Imagine how surprising an automatic door opener would be to a person living 150 years ago. It would seem as if the door "knew" they were approaching, and politely opened for them. Imagine how surprised we would be if Clippy understood what you were doing, and offered help that was actually helpful. As AI advances and is incorporated more and more into everyday things, our environment will become a reactive, "intelligent" black box, and along the way there will be some sudden advances that may be very startling and surreal.

jessriedel 9y ago

> The danger isn't (or isn't yet anyway) that AI is going to develop will and agency and oppose us. I agree about this for the foreseeable future, but it seems off topic. The article doesn't suggest anything like this, does it? I'm worried you might just be responding to the title.

whataretensors 9y ago

Not to mention adversarial attack vectors. I don't see AI working on mission critical systems while you can still remotely control the ai's decisions with what looks like white noise.

hacker_9 9y ago

Counterpoints: AI is easy to fool, you think people write completely true statements in all their twitter messages? Most of it is just spam. And do you honestly think that we are going to let 1000s of self driving cars drive around by themselves without some sort of checkpoint system in place, remote stop, inability to drive off the road etc? As for figuring out the leaders of the future, what do you think the whole education system is already doing?? And defence is possible, it just takes due diligence. Or in this fantasy future of yours, an AI that probes your business for you 247 and fixes security holes it finds. Honestly, I expect doom and gloom AI posts in the media, not on HN where people are expected to have some notion of an IQ.

jpalomaki 9y ago
Don't we already have 1000s of self-driving capable cars on the roads in the US? I think we are still quite far from having some government sanctioned kill switch in those. AI will be like Internet. When we in the year 2040 take a look at the last 40 years, we can't point out when it exactly became prevalent. We just know that at 2000 it was not a big deal, but at 2040 it is a very big deal and shaping the world.

notahacker 9y ago

The other issue is overestimated AI being generally a lot more dangerous than underestimated AI for the foreseeable future because it's a supposedly unimpeachably impartial intelligence source for unnecessary intervention, even if all the actual force at the end is being wielded by humans. AI that's super-accurate at identifying potential terrorists might be a threat to me if I'm living in an authoritarian police state, but the police state is a threat to me without AI, and probably somewhat more preventable than AI surveillance. On the whole, I'd probably rather local police and intelligence services used a perfect profiling system, in the extremely unlikely event of such a thing ever existing, to using the guesswork they do now. Much more threatening to me and much more likely to be actively being used is AI that's actually pretty weak at identifying potential terrorists but trusted because it's pretty darn efficient at finding something that's difficult to explain on targets' computers. Same goes for AI databases purporting to be super-accurate crime prediction or credit granting or job selection models that are too intractable for anyone to realise they're basically doing the same job as a guy building a race and class based regression model. And of course where it comes to cyber-battles, the guys on offence tend to have a lot less need to worry about the accuracy of their AI's targeting and whether it's vulnerable to being gamed or just plain rubbish than the guys on defence

goodmachine 9y ago

So I need 1000 'honeypots'? Asking for a friend.

blacksmith_tb 9y ago

This is a good counter-argument, but I wonder to what extent you think AI could also be used to counter surveillance? What if false patterns are seeded across thousands of Twitter accounts, misleading deliveries orchestrated to the 'wrong' addresses, etc.?