9 ms·

The near term danger of AI isn't in hyperintelligent SkyNet like systems, it's in human controlled autonomous weapon systems and "stupid AI" What you should be

The near term danger of AI isn't in hyperintelligent SkyNet like systems, it's in human controlled autonomous weapon systems and "stupid AI" What you should be fearing is military drones being given the ability to make decisions on targets or to fire, even with human assistance, and these systems won't just reside in the hands of large governments either. Already police and militaries around the world are using abstracted forms of force, wherein target are identified with algorithms, and then force trained on those targets What's do you think is going to happen first, SkyNet, or a predator imaging drone telling a human operator falsely that the current image is a terrorist? What's going to happen first? SkyNet, or self driving cars putting millions of people out of jobs because of a lack of demand for drivers in transportation, or manufacture of cars? (I'm not saying it's a bad thing, but it will be very very disruptive) If SkyNet is a threat, it's 50 or 100 years off I think. "AI" as it is now, is no where near the capability people are talking about. It's sheer hyperbole.

dao- 11y ago
Another more realistic potential threat is police relying too much on automated crime prediction. They are already embracing it: http://www.firstcoastnews.com/story/news/crime/2015/10/19/police-software-claims-to-predict-crime-before-it-happens/74234042/ http://www.firstcoastnews.com/story/news/crime/2015/10/19/po... An old thread on the topic: https://news.ycombinator.com/item?id=4185684 https://news.ycombinator.com/item?id=4185684
- pdkl95 11y ago
  > relying too much on automated crime prediction Including many variations of that same problem. "Sorry, you don't qualify for ____ because our 'clever' assessment algorithm doesn't account for your particular situation." Humans with decision making power correct these kind of problems all the time. Computers with clever algorithms can be incredibly useful, but they are still just tools. Unfortunately, given how bad we humans are with statistics, it's easy to overestimate[1] their accuracy. In addition to the raw accuracy problems, there is a risk of prejudices being baked into algorithms. We already see this with "redlining" and other housing practices, where loans availability ends up being coded racism. With the added complexity of machine learning and other prediction and analysis methods, it is probably a lot easier to hide improper discrimination. [1] reason #57 why statistics should be a mandatory HS math class, right after algebra
  - TeMPOraL 11y ago
    It's a hairy problem. Not understanding statistics goes both ways - and from what I've seen, it usually leads to overestimating the value of human input. People are quick to distrust or reject machine's judgment when it disagrees with their biases and generally, it's much easier to debias a machine (or code it properly in the first place) than to debias humans. While I agree that humans may enshrine their prejudices in code which will later will turn hard to adjust, again what I fear more is a reverse scenario. A machine can be perfectly fair and people won't like it, because it fails to apply the biases they want. People often apply counter-discrimination to compensate for what they believe is unfair treatment. A mandatory class in statistics and applied probability theory could maybe help the next generation to accept that what is fair may not look so at a first glance.
    - johncolanduoni 11y ago
      > A mandatory class in statistics and applied probability theory could maybe help the next generation to accept that what is fair may not look so at a first glance. To be honest, I don't think this will help much since most people won't interested in it and will forget what they learn as soon as the class is over. Also, being an expert in statistics can help you better understand reality, or it can help you present data in a way that confirms your biases (even unintentionally). It's sort of like sophistry, except not quite to that degree or moral ambivalence. edit: If you want a good example of this, see Scott Alexander show how parapsychology can actually come up with some pretty good statistics while appearing to check off all the right boxes with regard to experimental design: http://slatestarcodex.com/2014/04/28/the-control-group-is-out-of-control/ http://slatestarcodex.com/2014/04/28/the-control-group-is-ou...
- jonah 11y ago
  There was a good piece about this on the radio today. http://www.kcrw.com/news-culture/shows/to-the-point/can-big-data-prevent-crime http://www.kcrw.com/news-culture/shows/to-the-point/can-big-...
drjesusphd 11y ago
Not to mention what is happening with the NSA, which is essentially judicial-hearing-by-algorithm.
_up 11y ago
I think self driving cars and carts will also kill almost all middlemen. Why go shopping for food if you can get everything shipped to your fridge for 20-30% less, direct from the producer. Google will probably also offer to drive you for free to a medical specialist and start doing tests after you entered the car. So we probably need a lot less physicians too.
- genericresponse 11y ago
  I'd argue that we'd probably need a lot more. Diagnoses would be more accurate so people would be inclined to use the health system more. Health work would be cheaper because the expensive doctor component would be used less. Time to go to the doctor could be more efficient (still working through the car ride) so people are even more likely to go. Overall that could likely mean higher demand for healthcare, including the human component. Your food shopping point still needs middlemen. You have "chain of trust" issues with the producers. How do you know that you'll get good oranges from the orange grove. Who capitalizes the distribution system and designs the business model. Considering historical examples, we'll see more use for people in the economic problems that are still machine hard. Machine easy economic problems will converge to resource costs + any premium for monopoly. This will free up more demand for other products. People still want to buy an easier, better life.
  - Retric 11y ago
    There are often 3+ intermediaries between many farm products and the consumer. Dropping that to 1 chould be a huge change. Cow, wholesale, bottler, distributer, store, you. Granted, there is a lot of variety in this as some farms do there own botteling etc, but some products pass through even more hoops.
    - jacquesm 11y ago
      There is also the pasteurizing step to consider, and plenty of countries have 'milk boards' (cartels by any other name).
angdis 11y ago
I think the actual threat from AI is far more pedestrian than "Skynet" or killer robots. The real threat comes NOT from AI itself but from people who will be able to afford to exploit AI for profit at the expense of increasingly large numbers of people who just become redundant. The rich will get fabulously rich while everyone else just becomes marginalized into serfs. Another scenario that is less likely but still more likely than killer robots would be AI that simply loses interest in human affairs, stops interacting with us, and does its own thing-- like depicted in the movie "Her" !
- narrator 11y ago
  The other problem is if AI can be programmed to commit physical crimes then only a select priesthood of vetted developers will be allowed to program them at the service of a concentrated elite.
- bitcuration 11y ago
  Bravo to point out the obvious. Politics, moral only exist because there a large quantity of human.
tn13 11y ago
All AI researchers I spoke to laughed at this "SkyNet" like fantasies. According to them the reason why some people peddle these sort of horror stories is because they seek to get more funding by exaggerating the achievements of their field. Snowden like episodes have shown us that we should not trust our government. The government does blatantly illegal things, lies when confronted and in many cases kills or jails our own citizens just because they are exposing the government. I am more scared of government using this AI to oppress its own people. Imagine this, a person who is 18 years today becomes a potential presidential candidate or major activist in future. Some clerk at DC would simply run a computer program that will bring up the fact that this guy sent a sext to his 17 year old girlfriend in past, smoked weed when 19, bought beer when he was 20 etc. Such government would them implicate him in various court cases and jail him or even worse simply blackmail him to shut up.
- navait 11y ago
  Let's not overblow the threat. W was known to have done cocaine in his youth, and he was still elected, and I would expect statute of limitations to apply.
  - tn13 11y ago
    I am not over-blowing the threat. We would not know if government simply blackmailed people using such evidence. Many times you don't have to convict a person for the crime merely spreading a credible lie in media is enough. Obama has already used IRS to harass conservatives and we know what watergate was all about.
  - knughit 11y ago
    W was the supported choice of the power elite. It would go different Kshama Sawant was polling high numbers and such photos came out.
mtgx 11y ago
So you think that when we finally have smarter than human AI, it's just going to make completely objective decisions? First off, we'll never let such an AI to be completely objective, because, well, the paperclip scenario. And second, the AI will be about as "objective" as Google search is - in other words, not really. At the end of the day it's still humans that decide the algorithms for Google search, and it will be humans that decide the algorithms for the "smarter than human" AI.
- deleted 11y ago
  [deleted]
vlehto 11y ago
Luckily there are physical limitations. There are practically two ways to arm a drone. Bombs and guns. A small flying drone operating an 9mm pistol has extremely poor accuracy. Especially with multiple shots. If you increase the accuracy, you have to increase the mass. Which makes it bigger target. Currently regular guy with little training and a rifle can probably shoot down a gun wielding drone most of the time before the opposite can happen. https://www.youtube.com/watch?v=xqHrTtvFFIs https://www.youtube.com/watch?v=xqHrTtvFFIs Then there are drones with bombs attached to them. One of the most sophisticated is called Hellfire missile. But it's needlessly big and expensive. The quad copter in next video costs 100$ and has payload to carry regular hand grenade. https://www.youtube.com/watch?v=Lb2Tpp3CIoY https://www.youtube.com/watch?v=Lb2Tpp3CIoY You could go RC car route. But you again need a bit of size to conquer stairs. Also it's easier to track where it comes & goes. https://www.youtube.com/watch?v=kUXRMDK3r7s https://www.youtube.com/watch?v=kUXRMDK3r7s I'm less worried about government doing bad shit and more worried about private citizens getting nasty. Autonomous flight, GPS navigation, address from google maps and bit of facial recognition. First civilian drone murder is just matter of time. People kill with more ease the further they can be when it's done. I'm guessing this internet privacy thing will seem like child's play when all of sudden everybody wants to hide their physical address. Once that is handled, it's only bit like having rabid dogs with homing device.
- erikpukinskis 11y ago
  Are you suggesting there is something about drones that makes them fundamentally inaccurate? It would seem to be that advancements in computer vision will solve that problem in short order. Particularly with the VR industry introducing strong incentives for marginal improvements on image-based tracking.
  - ugexe 11y ago
    Newton's third law; Hence the comment about drone size/weight and mentioning the small caliber weapon it would have to use.
    - cma 11y ago
skybrian 11y ago
Even without the hardware, I'm a bit surprised we haven't seen more AI-based approaches to finding and exploiting security vulnerabilities. I guess it's still easy enough that they don't need fancier tools; a well-designed fuzzer can do a lot.
- dave_sullivan 11y ago
  Although I hear about machine learning-based approaches to fuzzers more and more lately. So I think you will soon!
tim333 11y ago
>predator imaging drone telling a human operator falsely that the current image is a terrorist Though that stuff isn't nice it's probably an improvement on previous technologies - looking out of the bomber and thinking that looks like some good buildings to bomb and similar.
- sanoli 11y ago
  Yeah, or looking at the monitor with the drone camera feed and thinking "hmmm, looks terrorist to me".
  - tim333 11y ago
    Well with AI they could do face recognition against their Twitter account and see if they'd posted any jihadi stuff recently
adventured 11y ago
Stupid AI in healthcare is a drastically greater threat than human controlled autonomous weapons will ever be, unless you're talking about WMDs. It's not a close risk comparison. The world is far less dangerous today than it has ever been, that includes both terrorism and war. AI and autonomous weapons systems are not going to suddenly make the developed countries that deploy those weapons want to slaughter each other. How many people die from medical mistakes per year in the US? More than died in all wars and all acts of terrorism combined globally in 2015. The AI that is no doubt going to show up throughout healthcare over the coming 30 or 40 years, will probably improve on the rate of human mistakes - while it still kills tens of thousands per year through mistakes in just the US.
- knughit 11y ago
  Not that many people die from medicla mistakes. The hysterical oft quoted paper blamed every suboptimal outcome from a "medical mistake", ignoring the contribution of whatever cause put them in medical care in the first place.
mlonkibjuyhv 11y ago
I wonder whether "a human in the loop" is some kind of illusion; for longer than we have had recorded history, we have killed our neighbours because an authority have told us to. In the latter half of the 20th century we nearly perfected the conditioning needed to make soldiers shoot to kill when told to. I wonder whether it will make a difference if the order is given by a human a or an AI. And if the soldier is going to shoot to kill regardless, why not just have an AI do that as well.
- kyllo 11y ago
  Yeah that is an interesting paradox. It feels wrong for an algorithm to decide whether to kill a human, but what is so much better about that decision being made by a human? If that seems extreme, consider the self-driving car problem. 30,000 people in the US alone are killed per year in car accidents involving human drivers. What is the acceptable number of motor vehicle related deaths per year when all cars are self-driving?
  - coldtea 11y ago
    >Yeah that is an interesting paradox. It feels wrong for an algorithm to decide whether to kill a human, but what is so much better about that decision being made by a human? Obviously because those humans then share and bear the moral dilemma of doing it or not. Which also affects the outcome, as opposed to just being some inconsequential detail. Humans can demand out of an injust war (as they've done time and again). Killing machines can not. It's the difference between blindly following orders (which not even all Nazi's did) and being able to have a change of heart. It's also the sheer (number of magnitudes) more killing efficiency of the killing algorithms, when a machine could kill 10000 people in the time it takes for a single human to make a decision about 1 of them -- until we find there's a != when we needed == in the algorithm.
    - spdionis 11y ago
      as another commenter noted there are currently physical limitations that would not allow e.g. drones to kill 10000 people in an instant. I don't think for example nukes will ever be under the control of AI algorithms ever...
avar 11y ago
> a predator imaging drone telling a human > operator falsely that the current image > is a terrorist? I think you're missing how this can be a really good thing. Right now this is happening but the "algorithm" is some unpredictable twenty year old in uniform. Are we going to have hearings auditing what the algorithm is like? is there going to be a scandal because isBrownPerson() is discovered in there somewhere? Is someone going to run it against a video collection of regular daily life and discover the drones would be indiscriminately bombing innocent situations if given the chance? The end result is that we're going to have to effectively have to come up with rules of engagement for every scenario. I think this will lead to war being fought more justly and with less collateral damage than ever before.
- Avshalom 11y ago
  The fear is that instead of an algorithm and a human double checking each other and refining each other that they will simply add. So we end up with the worst decisions of both human and computer.
- zensavona 11y ago
  I hadn't considered that angle before, there needs to be a lot of oversight (in general when we're killing people, not just computers) but I think you're right.
- bryang 11y ago
  It's adorable how you think it will improve how war is fought. These autonomous systems will always have a manual override and I will never believe that the review process will be given proper oversight. The armed forces will always err on the side of destruction in any potentially vaguely violent scenario. Fear and War is too prosperous to be peaceful.
  - peteretep 11y ago
    If this is true, one wonders why they waste all that time writing rules of engagement, and prosecuting those who don't follow them.
Shivetya 11y ago
with regards to spotting and identification I can see a system like what was shown in the last Dredd to come about, drones in the air and cameras feeding a large centralized room staffed by people directing others. While the military can get away with remote action it won't happen with police in Democratic countries except possibly with non lethal force. I could see remote shut down of cars, even a flash bang or sonic system. The real danger from AI isn't it taking over, it is from too many people checking out and not participating in life anymore
astrofinch 11y ago
"If SkyNet is a threat, it's 50 or 100 years off I think." Well we work on global warming even though the worst effects are likely decades off. I think one computer science professor put it something like this: "If experts told us aliens were likely arriving on Earth in 50 or 100 years, it'd make sense to start preparing now." You don't want to wait until it's too late to do whatever needs to be done.

dao- 11y ago

Another more realistic potential threat is police relying too much on automated crime prediction. They are already embracing it: http://www.firstcoastnews.com/story/news/crime/2015/10/19/police-software-claims-to-predict-crime-before-it-happens/74234042/ http://www.firstcoastnews.com/story/news/crime/2015/10/19/po... An old thread on the topic: https://news.ycombinator.com/item?id=4185684 https://news.ycombinator.com/item?id=4185684

pdkl95 11y ago
> relying too much on automated crime prediction Including many variations of that same problem. "Sorry, you don't qualify for ____ because our 'clever' assessment algorithm doesn't account for your particular situation." Humans with decision making power correct these kind of problems all the time. Computers with clever algorithms can be incredibly useful, but they are still just tools. Unfortunately, given how bad we humans are with statistics, it's easy to overestimate[1] their accuracy. In addition to the raw accuracy problems, there is a risk of prejudices being baked into algorithms. We already see this with "redlining" and other housing practices, where loans availability ends up being coded racism. With the added complexity of machine learning and other prediction and analysis methods, it is probably a lot easier to hide improper discrimination. [1] reason #57 why statistics should be a mandatory HS math class, right after algebra
- TeMPOraL 11y ago
  It's a hairy problem. Not understanding statistics goes both ways - and from what I've seen, it usually leads to overestimating the value of human input. People are quick to distrust or reject machine's judgment when it disagrees with their biases and generally, it's much easier to debias a machine (or code it properly in the first place) than to debias humans. While I agree that humans may enshrine their prejudices in code which will later will turn hard to adjust, again what I fear more is a reverse scenario. A machine can be perfectly fair and people won't like it, because it fails to apply the biases they want. People often apply counter-discrimination to compensate for what they believe is unfair treatment. A mandatory class in statistics and applied probability theory could maybe help the next generation to accept that what is fair may not look so at a first glance.
  - johncolanduoni 11y ago
    > A mandatory class in statistics and applied probability theory could maybe help the next generation to accept that what is fair may not look so at a first glance. To be honest, I don't think this will help much since most people won't interested in it and will forget what they learn as soon as the class is over. Also, being an expert in statistics can help you better understand reality, or it can help you present data in a way that confirms your biases (even unintentionally). It's sort of like sophistry, except not quite to that degree or moral ambivalence. edit: If you want a good example of this, see Scott Alexander show how parapsychology can actually come up with some pretty good statistics while appearing to check off all the right boxes with regard to experimental design: http://slatestarcodex.com/2014/04/28/the-control-group-is-out-of-control/ http://slatestarcodex.com/2014/04/28/the-control-group-is-ou...
jonah 11y ago
There was a good piece about this on the radio today. http://www.kcrw.com/news-culture/shows/to-the-point/can-big-data-prevent-crime http://www.kcrw.com/news-culture/shows/to-the-point/can-big-...

drjesusphd 11y ago

Not to mention what is happening with the NSA, which is essentially judicial-hearing-by-algorithm.

_up 11y ago

I think self driving cars and carts will also kill almost all middlemen. Why go shopping for food if you can get everything shipped to your fridge for 20-30% less, direct from the producer. Google will probably also offer to drive you for free to a medical specialist and start doing tests after you entered the car. So we probably need a lot less physicians too.

genericresponse 11y ago
I'd argue that we'd probably need a lot more. Diagnoses would be more accurate so people would be inclined to use the health system more. Health work would be cheaper because the expensive doctor component would be used less. Time to go to the doctor could be more efficient (still working through the car ride) so people are even more likely to go. Overall that could likely mean higher demand for healthcare, including the human component. Your food shopping point still needs middlemen. You have "chain of trust" issues with the producers. How do you know that you'll get good oranges from the orange grove. Who capitalizes the distribution system and designs the business model. Considering historical examples, we'll see more use for people in the economic problems that are still machine hard. Machine easy economic problems will converge to resource costs + any premium for monopoly. This will free up more demand for other products. People still want to buy an easier, better life.
- Retric 11y ago
  There are often 3+ intermediaries between many farm products and the consumer. Dropping that to 1 chould be a huge change. Cow, wholesale, bottler, distributer, store, you. Granted, there is a lot of variety in this as some farms do there own botteling etc, but some products pass through even more hoops.
  - jacquesm 11y ago
    There is also the pasteurizing step to consider, and plenty of countries have 'milk boards' (cartels by any other name).

angdis 11y ago

I think the actual threat from AI is far more pedestrian than "Skynet" or killer robots. The real threat comes NOT from AI itself but from people who will be able to afford to exploit AI for profit at the expense of increasingly large numbers of people who just become redundant. The rich will get fabulously rich while everyone else just becomes marginalized into serfs. Another scenario that is less likely but still more likely than killer robots would be AI that simply loses interest in human affairs, stops interacting with us, and does its own thing-- like depicted in the movie "Her" !

narrator 11y ago
The other problem is if AI can be programmed to commit physical crimes then only a select priesthood of vetted developers will be allowed to program them at the service of a concentrated elite.
bitcuration 11y ago
Bravo to point out the obvious. Politics, moral only exist because there a large quantity of human.

tn13 11y ago

All AI researchers I spoke to laughed at this "SkyNet" like fantasies. According to them the reason why some people peddle these sort of horror stories is because they seek to get more funding by exaggerating the achievements of their field. Snowden like episodes have shown us that we should not trust our government. The government does blatantly illegal things, lies when confronted and in many cases kills or jails our own citizens just because they are exposing the government. I am more scared of government using this AI to oppress its own people. Imagine this, a person who is 18 years today becomes a potential presidential candidate or major activist in future. Some clerk at DC would simply run a computer program that will bring up the fact that this guy sent a sext to his 17 year old girlfriend in past, smoked weed when 19, bought beer when he was 20 etc. Such government would them implicate him in various court cases and jail him or even worse simply blackmail him to shut up.

navait 11y ago
Let's not overblow the threat. W was known to have done cocaine in his youth, and he was still elected, and I would expect statute of limitations to apply.
- tn13 11y ago
  I am not over-blowing the threat. We would not know if government simply blackmailed people using such evidence. Many times you don't have to convict a person for the crime merely spreading a credible lie in media is enough. Obama has already used IRS to harass conservatives and we know what watergate was all about.
- knughit 11y ago
  W was the supported choice of the power elite. It would go different Kshama Sawant was polling high numbers and such photos came out.

mtgx 11y ago

So you think that when we finally have smarter than human AI, it's just going to make completely objective decisions? First off, we'll never let such an AI to be completely objective, because, well, the paperclip scenario. And second, the AI will be about as "objective" as Google search is - in other words, not really. At the end of the day it's still humans that decide the algorithms for Google search, and it will be humans that decide the algorithms for the "smarter than human" AI.

deleted 11y ago
[deleted]

vlehto 11y ago

Luckily there are physical limitations. There are practically two ways to arm a drone. Bombs and guns. A small flying drone operating an 9mm pistol has extremely poor accuracy. Especially with multiple shots. If you increase the accuracy, you have to increase the mass. Which makes it bigger target. Currently regular guy with little training and a rifle can probably shoot down a gun wielding drone most of the time before the opposite can happen. https://www.youtube.com/watch?v=xqHrTtvFFIs https://www.youtube.com/watch?v=xqHrTtvFFIs Then there are drones with bombs attached to them. One of the most sophisticated is called Hellfire missile. But it's needlessly big and expensive. The quad copter in next video costs 100$ and has payload to carry regular hand grenade. https://www.youtube.com/watch?v=Lb2Tpp3CIoY https://www.youtube.com/watch?v=Lb2Tpp3CIoY You could go RC car route. But you again need a bit of size to conquer stairs. Also it's easier to track where it comes & goes. https://www.youtube.com/watch?v=kUXRMDK3r7s https://www.youtube.com/watch?v=kUXRMDK3r7s I'm less worried about government doing bad shit and more worried about private citizens getting nasty. Autonomous flight, GPS navigation, address from google maps and bit of facial recognition. First civilian drone murder is just matter of time. People kill with more ease the further they can be when it's done. I'm guessing this internet privacy thing will seem like child's play when all of sudden everybody wants to hide their physical address. Once that is handled, it's only bit like having rabid dogs with homing device.

erikpukinskis 11y ago
Are you suggesting there is something about drones that makes them fundamentally inaccurate? It would seem to be that advancements in computer vision will solve that problem in short order. Particularly with the VR industry introducing strong incentives for marginal improvements on image-based tracking.
- ugexe 11y ago
  Newton's third law; Hence the comment about drone size/weight and mentioning the small caliber weapon it would have to use.
  - cma 11y ago

skybrian 11y ago

Even without the hardware, I'm a bit surprised we haven't seen more AI-based approaches to finding and exploiting security vulnerabilities. I guess it's still easy enough that they don't need fancier tools; a well-designed fuzzer can do a lot.

dave_sullivan 11y ago
Although I hear about machine learning-based approaches to fuzzers more and more lately. So I think you will soon!

tim333 11y ago

>predator imaging drone telling a human operator falsely that the current image is a terrorist Though that stuff isn't nice it's probably an improvement on previous technologies - looking out of the bomber and thinking that looks like some good buildings to bomb and similar.

sanoli 11y ago
Yeah, or looking at the monitor with the drone camera feed and thinking "hmmm, looks terrorist to me".
- tim333 11y ago
  Well with AI they could do face recognition against their Twitter account and see if they'd posted any jihadi stuff recently

adventured 11y ago

Stupid AI in healthcare is a drastically greater threat than human controlled autonomous weapons will ever be, unless you're talking about WMDs. It's not a close risk comparison. The world is far less dangerous today than it has ever been, that includes both terrorism and war. AI and autonomous weapons systems are not going to suddenly make the developed countries that deploy those weapons want to slaughter each other. How many people die from medical mistakes per year in the US? More than died in all wars and all acts of terrorism combined globally in 2015. The AI that is no doubt going to show up throughout healthcare over the coming 30 or 40 years, will probably improve on the rate of human mistakes - while it still kills tens of thousands per year through mistakes in just the US.

knughit 11y ago
Not that many people die from medicla mistakes. The hysterical oft quoted paper blamed every suboptimal outcome from a "medical mistake", ignoring the contribution of whatever cause put them in medical care in the first place.

mlonkibjuyhv 11y ago

I wonder whether "a human in the loop" is some kind of illusion; for longer than we have had recorded history, we have killed our neighbours because an authority have told us to. In the latter half of the 20th century we nearly perfected the conditioning needed to make soldiers shoot to kill when told to. I wonder whether it will make a difference if the order is given by a human a or an AI. And if the soldier is going to shoot to kill regardless, why not just have an AI do that as well.

kyllo 11y ago
Yeah that is an interesting paradox. It feels wrong for an algorithm to decide whether to kill a human, but what is so much better about that decision being made by a human? If that seems extreme, consider the self-driving car problem. 30,000 people in the US alone are killed per year in car accidents involving human drivers. What is the acceptable number of motor vehicle related deaths per year when all cars are self-driving?
- coldtea 11y ago
  >Yeah that is an interesting paradox. It feels wrong for an algorithm to decide whether to kill a human, but what is so much better about that decision being made by a human? Obviously because those humans then share and bear the moral dilemma of doing it or not. Which also affects the outcome, as opposed to just being some inconsequential detail. Humans can demand out of an injust war (as they've done time and again). Killing machines can not. It's the difference between blindly following orders (which not even all Nazi's did) and being able to have a change of heart. It's also the sheer (number of magnitudes) more killing efficiency of the killing algorithms, when a machine could kill 10000 people in the time it takes for a single human to make a decision about 1 of them -- until we find there's a != when we needed == in the algorithm.
  - spdionis 11y ago
    as another commenter noted there are currently physical limitations that would not allow e.g. drones to kill 10000 people in an instant. I don't think for example nukes will ever be under the control of AI algorithms ever...

avar 11y ago

> a predator imaging drone telling a human > operator falsely that the current image > is a terrorist? I think you're missing how this can be a really good thing. Right now this is happening but the "algorithm" is some unpredictable twenty year old in uniform. Are we going to have hearings auditing what the algorithm is like? is there going to be a scandal because isBrownPerson() is discovered in there somewhere? Is someone going to run it against a video collection of regular daily life and discover the drones would be indiscriminately bombing innocent situations if given the chance? The end result is that we're going to have to effectively have to come up with rules of engagement for every scenario. I think this will lead to war being fought more justly and with less collateral damage than ever before.

Avshalom 11y ago
The fear is that instead of an algorithm and a human double checking each other and refining each other that they will simply add. So we end up with the worst decisions of both human and computer.
zensavona 11y ago
I hadn't considered that angle before, there needs to be a lot of oversight (in general when we're killing people, not just computers) but I think you're right.
bryang 11y ago
It's adorable how you think it will improve how war is fought. These autonomous systems will always have a manual override and I will never believe that the review process will be given proper oversight. The armed forces will always err on the side of destruction in any potentially vaguely violent scenario. Fear and War is too prosperous to be peaceful.
- peteretep 11y ago
  If this is true, one wonders why they waste all that time writing rules of engagement, and prosecuting those who don't follow them.

Shivetya 11y ago

with regards to spotting and identification I can see a system like what was shown in the last Dredd to come about, drones in the air and cameras feeding a large centralized room staffed by people directing others. While the military can get away with remote action it won't happen with police in Democratic countries except possibly with non lethal force. I could see remote shut down of cars, even a flash bang or sonic system. The real danger from AI isn't it taking over, it is from too many people checking out and not participating in life anymore

astrofinch 11y ago

"If SkyNet is a threat, it's 50 or 100 years off I think." Well we work on global warming even though the worst effects are likely decades off. I think one computer science professor put it something like this: "If experts told us aliens were likely arriving on Earth in 50 or 100 years, it'd make sense to start preparing now." You don't want to wait until it's too late to do whatever needs to be done.