9 ms·

I heard a story on the radio the other day about the debate of whether or not we should allow robots to kill people. I wonder if ever in history there was a ti

I heard a story on the radio the other day about the debate of whether or not we should allow robots to kill people. I wonder if ever in history there was a time when we needed to kill people faster and set our engineering minds to solve that problem. Would there be any lessons to learn we could apply to our current problem of automated human life destruction. I just can't believe there are human beings who would actually write that code! Edit: Clarification, Yes, robots that kill automatically with no human intervention. Programmed to seek and kill the enemy.

inreplytothat 7y ago
How about the machine gun?
- anon1m0us 7y ago
  Humans aim and pull the trigger -- and they aren't self-motive. Robots are different.
  - PhilWright 7y ago
    There is no difference between a machine gun and a robot. A person designs a machine gun but does not pull the actual trigger. There is an army officer that orders the weapon be deployed and the rules of engagement for when it can be used. A person designs a killer robot but does not activate it. There is an army officer that orders the weapon be deployed and the rules of engagement for when it can be used. There is always a person that is responsible for the use of a munition. Maybe that person is higher up the chain with a killer robot than a machine gun. If you want to launch a nuclear bomb then it goes all the way to the top of the chain. But there is always a person that is responsible for the death caused by the munition.
QuercusMax 7y ago
I mean, you have people who design weapons. They're explicitly intended to kill people. Is programing a robot to kill people any different from programming a weapon targeting system? I'm not endorsing this, mind you, but it's not really any different that what we've been doing to each other for millenia.
- evancox100 7y ago
  I wonder if OP meant “deciding whether or not to kill people”.
  - anon1m0us 7y ago
    Yes, thank you, I did. To clarify, the debate was about allowing robots to kill without a human in the loop. Basically, drop the robots in and let them start killing by whatever algorithm has been programmed into them.
- anon1m0us 7y ago
  That's my point. There's been a steady trend toward global annihilation and as a species we are running toward it. I was out with a bunch of friends who worked for Northrup Grumman and I was like, "Why do you guys work there building things designed to kill people?" Synchronized response: The money is good.
  - tonyedgecombe 7y ago
    >Synchronized response: The money is good. It has to be because many people wouldn't work there.
- onemoresoop 7y ago
  But the responsibility of “pulling the trigger” makes these engineers sheltered from any guilt. However, engineering the cases for triggering the robots to kill and the resulting bugs can be directly attributed to the team that designs it. Whether they care or not, or ever have a hard time falling asleep at night is a different story..
  - gizmo686 7y ago
    There are plenty of engineers who build systems where bugs can lead directly to death. In the case of killer robots, the engineers are not (outside of a test gone wrong) going to be realeasing the robots themselves. They are going to be giving them to the military, where officers create and load particular configurations then release the robots with that. These are the same people who current create orders release frontline soldiers to execute said orders.
pessimizer 7y ago
Oppenheimer and Einstein were very nice people who in general weren't in agreement with most of the policies of the governments that they helped develop killing machines for. Those governments knew this, and distrusted them for it. Only some of us are sociopaths (a lot of people in tech, sadly), but most of us just compartmentalize and rationalize, with a special pitfall for the math-oriented is the truism that technology doesn't have moral implications in and of itself, just the uses to which it is put. The last is one of those things that people never bother to justify, they just think of it as an obvious implication of a impenetrable division between abstraction and the material world that is nevertheless marked by humans shuttling information between the two. To me, thinking of people this way is just warmed-over dualism. If you introduce a technology into the world, you have to consider the possibilities every type of person with every possible aim having control of that technology. Not doing this is simply not scientific.
- PavlovsCat 7y ago
  > People have a series of rationalizations. People say for example that science and technology have their own logic, that they are in fact autonomous. This particular rationalization is profoundly false. It is not true that science marches on in defiance of human will, independent of human will, that just is not the case. But it is comfortable, as I said: it leads to the position that "if I don't do it, someone else will." > Of course if one takes that as an ethical principle then obviously it can serve as a license to do anything at all. "People will be murdered; if I don't do it, someone else will." "Women will be raped; if I don't do it, someone else will." That is just a license for violence. > Other people say, and I think this is a widely used rationalization, that fundamentally the tools we work on are "mere" tools; This means that whether they get use for good or evil depends on the person who ultimately buys them and so on. > There's nothing bad about working in computer vision, for example. Computer vision may very well some day be used to heal people who would otherwise die. Of course, it could also be used to guide missiles, cruise missiles for example, to their destination, and all that. You see, the technology itself is neutral and value-free and it just depends how one uses it. And besides -- consistent with that -- we can't know, we scientists cannot know how it is going to be used. So therefore we have no responsibility. > Well, that is false. It is true that a computer, for example, can be used for good or evil. It is true that a helicopter can be used as a gunship and it can also be used to rescue people from a mountain pass. And if the question arises of how a specific device is going to be used, in what I call an abstract ideal society, then one might very well say one cannot know. > But we live in a concrete society, [and] with concrete social and historical circumstances and political realities in this society, it is perfectly obvious that when something like a computer is invented, then it is going to be adopted will be for military purposes. It follows from the concrete realities in which we live, it does not follow from pure logic. But we're not living in an abstract society, we're living in the society in which we in fact live. > If you look at the enormous fruits of human genius that mankind has developed in the last 50 years, atomic energy and rocketry and flying to the moon and coherent light, and it goes on and on and on -- and then it turns out that every one of these triumphs is used primarily in military terms. So it is not reasonable for a scientist or technologist to insist that he or she does not know -- or cannot know -- how it is going to be used. -- Joseph Weizenbaum, http://tech.mit.edu/V105/N16/weisen.16n.html http://tech.mit.edu/V105/N16/weisen.16n.html
PavlovsCat 7y ago
[beaming smile] "Zat could be eezily done with a compjutah!" One person writes calculate_attitudes(person) Another writes calculate_abilities(person) Another writes calculate_desirability(person, calculated_attitudes, calculated_abilities) Another writes determine_spatial_location(person) Another writes parse_list_of_protected_persons(whitelist) Another writes move_to_safe_vantage_point(drone, picked_target, whitelist) Another writes aim_cannon(cannon, spatial_location) Another writes fire_cannon(cannon) Another writes determine_witnesses(spatial_location, timestamp) Then you just feed witness_person, witness_attitudes, witness_abilities into calculate_desirability, and loop it forever, killing the least obedient n%. No involved person really did much, or even anything, as far as they are aware. Especially since you can, as Joseph Weizenbaum -- the one person HN really should know, but really doesn't -- pointed out, design/build/train those algorithms and machines with all sorts of rather benign stuff. E.g. you train for finding missing persons, but then you can also just feed it names and photos of dissidents instead of missing persons. The people "refitting" may know about the people who build them in the first place, but not the other way around.
- anon1m0us 7y ago
  Yes, that's correct. That's the idea. That is the goal. People are working on this right now. Writing basically, exactly those functions. The survival of humanity has escaped ALL tribal boundaries. The human beings with the most power are the ones contributing the most to our assured destruction. It's basically inevitable. I am not a religious person, but ... has this happened before? Have we destroyed ourselves before? On another planet? Millions of years ago, before the dinosaurs? How could revelations have predicted it so well thousands of years ago, hundreds maybe, before there was any evidence all of this would be happening right now. Lots of religions have these stories about ourselves destroying ourselves and our planet. Indian religions describe it too I know for sure. Battles of the gods in ships with laser like weapons. It's like we are uncontrollably, knowingly, ... desiring of self-destruction as a species. Why are we like this and how do we change us? Maybe I'm insane.
  - PavlovsCat 7y ago
    To give credit where it's due, I kind of took this and ran with it: > It should be noted that no ethically-trained software engineer would ever consent to write a DestroyBaghdad procedure. Basic professional ethics would instead require him to write a DestroyCity procedure, to which Baghdad could be given as a parameter. -- Nathaniel Borenstein > Why are we like this and how do we change us? Don't take this the wrong way, because I totally "feel you", to put it briefly, but my first thought is, there is no "us": > When all are guilty, no one is; confessions of collective guilt are the best possible safeguard against the discovery of culprits, and the very magnitude of the crime the best excuse for doing nothing. -- Hannah Arendt For me it's important to not forget that sociopaths are weak rather than strong, and bluffing rather than laughing. Outside of forms of communications they can dictate and manipulate, that is, without some form of violence or dishonesty, on their own and as themselves, they have nothing, and can be made to implode rather easily. A murdering SS officer is not what sums up this phenomenon, Hitler, in his bunker, blowing his brains out, after blaming everyone with words that are obviously garbage once the claqueurs are gone, is. No, I'm not saying I'm innocent and others are the sole source of evil. But we're still not all the same, it's a spectrum and we're not all in the same spot, and I reject that it's some "average human" doing everything. That's an abstraction, only the actually existing individuals are real, and the worst thing that can happen is not resisting and realizing it was "in vain" (and it never is when it comes to what it makes you as a person), the worst thing that can happen is that we realize we fell for rather mediocre people who faked it until they made it, and that we could have resisted easily (in comparison to the titanium steel gravity well humanity would be in then).
deleted 7y ago
[deleted]
quotemstr 7y ago
I'd write that code. I don't believe that violence is wrong. I don't believe that defense work is wrong. I don't have to lean on my ported ignorance of a technology's applications because I don't buy into the moral framework that says that this use or that use is wrong. Every technology not forbidden by physics is inevitable. If somebody else is going to get a powerful tool to use against me, I'll damn well make sure I have that tool too. And if I can, I'll get it first. You want safety from killer robots? Make more and better killer robots than the other guy.
Noos 7y ago
> Yes, robots that kill automatically with no human intervention. Programmed to seek and kill the enemy. You know what those robots can't do? They can't rape the enemy. They can't kill their superior officers, traffic in or get addicted to drugs, and ideally, they won't be killing anyone but their target. They can't be made prisoners of war and tortured, and they can't really be hostages, either captured or because of proximity to the combat theater. If the robot is destroyed, you don't need to tell someone's mother their son isn't coming home. It may even stop wars because the other side is only destroying materiel that can be replaced instead of lives. A deterrent. I know the negative arguments against them very well, and I also agree with them; you can't do this because it can and will spread and it makes killing much "cheaper" for the same reasons. But why is it so hard to imagine why people would write that code? It's really the same reaasons why people here love self-driving cars; to remove human error that causes waste and death.
- timoth3y 7y ago
  > They can't be made prisoners of war and tortured, and they can't really be hostages, either captured or because of proximity to the combat theater. That sounds like a feature, but it has a very important downside. One of the biggest reasons that politicians hesitate to send in the troops is for the very reason you mention. Killing from afar with no accountability or political downside is a recipe for permanent warfare. In fact, there is a significant upside to them from the defense contractors. You could argue (and many have) that we have already seen the start of this with the US drone program.
hkai 7y ago
Would you program robots to kill Nazis?
wademealing 7y ago
There is almost 0% reason why this can't be done already.
deleted 7y ago
[deleted]

inreplytothat 7y ago

How about the machine gun?

anon1m0us 7y ago
Humans aim and pull the trigger -- and they aren't self-motive. Robots are different.
- PhilWright 7y ago
  There is no difference between a machine gun and a robot. A person designs a machine gun but does not pull the actual trigger. There is an army officer that orders the weapon be deployed and the rules of engagement for when it can be used. A person designs a killer robot but does not activate it. There is an army officer that orders the weapon be deployed and the rules of engagement for when it can be used. There is always a person that is responsible for the use of a munition. Maybe that person is higher up the chain with a killer robot than a machine gun. If you want to launch a nuclear bomb then it goes all the way to the top of the chain. But there is always a person that is responsible for the death caused by the munition.

QuercusMax 7y ago

I mean, you have people who design weapons. They're explicitly intended to kill people. Is programing a robot to kill people any different from programming a weapon targeting system? I'm not endorsing this, mind you, but it's not really any different that what we've been doing to each other for millenia.

evancox100 7y ago
I wonder if OP meant “deciding whether or not to kill people”.
- anon1m0us 7y ago
  Yes, thank you, I did. To clarify, the debate was about allowing robots to kill without a human in the loop. Basically, drop the robots in and let them start killing by whatever algorithm has been programmed into them.
anon1m0us 7y ago
That's my point. There's been a steady trend toward global annihilation and as a species we are running toward it. I was out with a bunch of friends who worked for Northrup Grumman and I was like, "Why do you guys work there building things designed to kill people?" Synchronized response: The money is good.
- tonyedgecombe 7y ago
  >Synchronized response: The money is good. It has to be because many people wouldn't work there.
onemoresoop 7y ago
But the responsibility of “pulling the trigger” makes these engineers sheltered from any guilt. However, engineering the cases for triggering the robots to kill and the resulting bugs can be directly attributed to the team that designs it. Whether they care or not, or ever have a hard time falling asleep at night is a different story..
- gizmo686 7y ago
  There are plenty of engineers who build systems where bugs can lead directly to death. In the case of killer robots, the engineers are not (outside of a test gone wrong) going to be realeasing the robots themselves. They are going to be giving them to the military, where officers create and load particular configurations then release the robots with that. These are the same people who current create orders release frontline soldiers to execute said orders.

pessimizer 7y ago

Oppenheimer and Einstein were very nice people who in general weren't in agreement with most of the policies of the governments that they helped develop killing machines for. Those governments knew this, and distrusted them for it. Only some of us are sociopaths (a lot of people in tech, sadly), but most of us just compartmentalize and rationalize, with a special pitfall for the math-oriented is the truism that technology doesn't have moral implications in and of itself, just the uses to which it is put. The last is one of those things that people never bother to justify, they just think of it as an obvious implication of a impenetrable division between abstraction and the material world that is nevertheless marked by humans shuttling information between the two. To me, thinking of people this way is just warmed-over dualism. If you introduce a technology into the world, you have to consider the possibilities every type of person with every possible aim having control of that technology. Not doing this is simply not scientific.

PavlovsCat 7y ago
> People have a series of rationalizations. People say for example that science and technology have their own logic, that they are in fact autonomous. This particular rationalization is profoundly false. It is not true that science marches on in defiance of human will, independent of human will, that just is not the case. But it is comfortable, as I said: it leads to the position that "if I don't do it, someone else will." > Of course if one takes that as an ethical principle then obviously it can serve as a license to do anything at all. "People will be murdered; if I don't do it, someone else will." "Women will be raped; if I don't do it, someone else will." That is just a license for violence. > Other people say, and I think this is a widely used rationalization, that fundamentally the tools we work on are "mere" tools; This means that whether they get use for good or evil depends on the person who ultimately buys them and so on. > There's nothing bad about working in computer vision, for example. Computer vision may very well some day be used to heal people who would otherwise die. Of course, it could also be used to guide missiles, cruise missiles for example, to their destination, and all that. You see, the technology itself is neutral and value-free and it just depends how one uses it. And besides -- consistent with that -- we can't know, we scientists cannot know how it is going to be used. So therefore we have no responsibility. > Well, that is false. It is true that a computer, for example, can be used for good or evil. It is true that a helicopter can be used as a gunship and it can also be used to rescue people from a mountain pass. And if the question arises of how a specific device is going to be used, in what I call an abstract ideal society, then one might very well say one cannot know. > But we live in a concrete society, [and] with concrete social and historical circumstances and political realities in this society, it is perfectly obvious that when something like a computer is invented, then it is going to be adopted will be for military purposes. It follows from the concrete realities in which we live, it does not follow from pure logic. But we're not living in an abstract society, we're living in the society in which we in fact live. > If you look at the enormous fruits of human genius that mankind has developed in the last 50 years, atomic energy and rocketry and flying to the moon and coherent light, and it goes on and on and on -- and then it turns out that every one of these triumphs is used primarily in military terms. So it is not reasonable for a scientist or technologist to insist that he or she does not know -- or cannot know -- how it is going to be used. -- Joseph Weizenbaum, http://tech.mit.edu/V105/N16/weisen.16n.html http://tech.mit.edu/V105/N16/weisen.16n.html

PavlovsCat 7y ago

[beaming smile] "Zat could be eezily done with a compjutah!" One person writes calculate_attitudes(person) Another writes calculate_abilities(person) Another writes calculate_desirability(person, calculated_attitudes, calculated_abilities) Another writes determine_spatial_location(person) Another writes parse_list_of_protected_persons(whitelist) Another writes move_to_safe_vantage_point(drone, picked_target, whitelist) Another writes aim_cannon(cannon, spatial_location) Another writes fire_cannon(cannon) Another writes determine_witnesses(spatial_location, timestamp) Then you just feed witness_person, witness_attitudes, witness_abilities into calculate_desirability, and loop it forever, killing the least obedient n%. No involved person really did much, or even anything, as far as they are aware. Especially since you can, as Joseph Weizenbaum -- the one person HN really should know, but really doesn't -- pointed out, design/build/train those algorithms and machines with all sorts of rather benign stuff. E.g. you train for finding missing persons, but then you can also just feed it names and photos of dissidents instead of missing persons. The people "refitting" may know about the people who build them in the first place, but not the other way around.

anon1m0us 7y ago
Yes, that's correct. That's the idea. That is the goal. People are working on this right now. Writing basically, exactly those functions. The survival of humanity has escaped ALL tribal boundaries. The human beings with the most power are the ones contributing the most to our assured destruction. It's basically inevitable. I am not a religious person, but ... has this happened before? Have we destroyed ourselves before? On another planet? Millions of years ago, before the dinosaurs? How could revelations have predicted it so well thousands of years ago, hundreds maybe, before there was any evidence all of this would be happening right now. Lots of religions have these stories about ourselves destroying ourselves and our planet. Indian religions describe it too I know for sure. Battles of the gods in ships with laser like weapons. It's like we are uncontrollably, knowingly, ... desiring of self-destruction as a species. Why are we like this and how do we change us? Maybe I'm insane.
- PavlovsCat 7y ago
  To give credit where it's due, I kind of took this and ran with it: > It should be noted that no ethically-trained software engineer would ever consent to write a DestroyBaghdad procedure. Basic professional ethics would instead require him to write a DestroyCity procedure, to which Baghdad could be given as a parameter. -- Nathaniel Borenstein > Why are we like this and how do we change us? Don't take this the wrong way, because I totally "feel you", to put it briefly, but my first thought is, there is no "us": > When all are guilty, no one is; confessions of collective guilt are the best possible safeguard against the discovery of culprits, and the very magnitude of the crime the best excuse for doing nothing. -- Hannah Arendt For me it's important to not forget that sociopaths are weak rather than strong, and bluffing rather than laughing. Outside of forms of communications they can dictate and manipulate, that is, without some form of violence or dishonesty, on their own and as themselves, they have nothing, and can be made to implode rather easily. A murdering SS officer is not what sums up this phenomenon, Hitler, in his bunker, blowing his brains out, after blaming everyone with words that are obviously garbage once the claqueurs are gone, is. No, I'm not saying I'm innocent and others are the sole source of evil. But we're still not all the same, it's a spectrum and we're not all in the same spot, and I reject that it's some "average human" doing everything. That's an abstraction, only the actually existing individuals are real, and the worst thing that can happen is not resisting and realizing it was "in vain" (and it never is when it comes to what it makes you as a person), the worst thing that can happen is that we realize we fell for rather mediocre people who faked it until they made it, and that we could have resisted easily (in comparison to the titanium steel gravity well humanity would be in then).

deleted 7y ago

[deleted]

quotemstr 7y ago

I'd write that code. I don't believe that violence is wrong. I don't believe that defense work is wrong. I don't have to lean on my ported ignorance of a technology's applications because I don't buy into the moral framework that says that this use or that use is wrong. Every technology not forbidden by physics is inevitable. If somebody else is going to get a powerful tool to use against me, I'll damn well make sure I have that tool too. And if I can, I'll get it first. You want safety from killer robots? Make more and better killer robots than the other guy.

Noos 7y ago

> Yes, robots that kill automatically with no human intervention. Programmed to seek and kill the enemy. You know what those robots can't do? They can't rape the enemy. They can't kill their superior officers, traffic in or get addicted to drugs, and ideally, they won't be killing anyone but their target. They can't be made prisoners of war and tortured, and they can't really be hostages, either captured or because of proximity to the combat theater. If the robot is destroyed, you don't need to tell someone's mother their son isn't coming home. It may even stop wars because the other side is only destroying materiel that can be replaced instead of lives. A deterrent. I know the negative arguments against them very well, and I also agree with them; you can't do this because it can and will spread and it makes killing much "cheaper" for the same reasons. But why is it so hard to imagine why people would write that code? It's really the same reaasons why people here love self-driving cars; to remove human error that causes waste and death.

timoth3y 7y ago
> They can't be made prisoners of war and tortured, and they can't really be hostages, either captured or because of proximity to the combat theater. That sounds like a feature, but it has a very important downside. One of the biggest reasons that politicians hesitate to send in the troops is for the very reason you mention. Killing from afar with no accountability or political downside is a recipe for permanent warfare. In fact, there is a significant upside to them from the defense contractors. You could argue (and many have) that we have already seen the start of this with the US drone program.

hkai 7y ago

Would you program robots to kill Nazis?

wademealing 7y ago

There is almost 0% reason why this can't be done already.

deleted 7y ago

[deleted]