4 ms·

Yeah, this argument is interesting. There are good moral rebuttals. Someone on HN said recently, “your actions remain immoral even if others are ready to be jus

by tyleo 3y ago

Yeah, this argument is interesting. There are good moral rebuttals. Someone on HN said recently, “your actions remain immoral even if others are ready to be just as immoral as you are.” I haven’t heard a good practical rebuttal though :/ W.r.t. nukes, for example, there is a good chance that mutually assured destruction prevented war despite how terrifying the policy is.

bluefirebrand 3y ago
It's also worth considering if the act of inventing something horrifying is actually immoral, or if using it for immoral purposes is. For instance, dynamite. We would not have the society we do now without dynamite. The fact that it can also be used to destroy property and kill people makes it a deadly invention, but was it immoral to invent? Of course dynamite is nothing compared to the Atomic Bomb. Or maybe inventing it for the purposes of killing people makes it immoral regardless of other possible applications. How much does intent matter, versus outcome? And then, does it absolve the inventor if they had pure intentions but others used their inventions to do harm? Is it all moot because trying to live a moral life when others are willing to be immoral will basically never work out in your favor?
- innagadadavida 3y ago
  We could also consider failure to prevent bad actors from doing immoral things as immoral itself. The problem with this line of thinking is there is no tight definition of immorality.
- uwagar 3y ago
  >Or maybe inventing it for the purposes of killing people makes it immoral regardless of other possible applications. this is the key. the atom bomb was made to kill people. i also felt a mandate driven ie arm twisting people until they vaccinated themselves against covid was also immoral.
  - uwagar 3y ago
    talk about covid vaccine and get downvoted. doesnt matter what.
- 12907835202 3y ago
  (1) if one day a nuke was used to deflect an asteroid and save the entire planet, would that then outweigh any deaths caused by nukes and ottenheimer would be a hero rather than a villain? That's a tricky road to go down. (2) as is trying to focus on intent. Do you want to risk absolving Hitler or Pol Pot because they thought they were making the world a better place?
  - oh_sigh 3y ago
    Oppenheimer is a villain? First I'm hearing about it.
    - toyg 3y ago
      Probably not popular in Japan.
      - localplume 3y ago
        [dead]
  - bluefirebrand 3y ago
    > (2) as is trying to focus on intent. Do you want to risk absolving Hitler or Pol Pot because they thought they were making the world a better place? I would argue that "trying to make the world better" is too broad to be an intent. It's an ideal, or something. Maybe that's just language semantics. As soon as you dig into that ideal at all, the malicious intent becomes extremely clear. Ideal: "I am going to make the world a better place" How? Intent: "By purging lesser humans so my people can become the master race as we should" Versus Intent: "By volunteering my time to set up soup kitchens in the inner city" Ideal leads to intent leads to action leads to outcome. My opinion, anyways.
- johnnyworker 3y ago
  > inventing it for the purposes of Regardless of how the sentence continues, there is no such thing. There is making inventions in the world as it currently is, and what follows from that. edit: since the page http://tech.mit.edu/V105/N16/weisen.16n.html http://tech.mit.edu/V105/N16/weisen.16n.html is down atm, I have to quote it instead of linking it: > People have a series of rationalizations. People say for example that science and technology have their own logic, that they are in fact autonomous. This particular rationalization is profoundly false. It is not true that science marches on in defiance of human will, independent of human will, that just is not the case. But it is comfortable, as I said: it leads to the position that "if I don't do it, someone else will." > Of course if one takes that as an ethical principle then obviously it can serve as a license to do anything at all. "People will be murdered; if I don't do it, someone else will." "Women will be raped; if I don't do it, someone else will." That is just a license for violence. > Other people say, and I think this is a widely used rationalization, that fundamentally the tools we work on are "mere" tools; This means that whether they get use for good or evil depends on the person who ultimately buys them and so on. > There's nothing bad about working in computer vision, for example. Computer vision may very well some day be used to heal people who would otherwise die. Of course, it could also be used to guide missiles, cruise missiles for example, to their destination, and all that. You see, the technology itself is neutral and value-free and it just depends how one uses it. And besides -- consistent with that -- we can't know, we scientists cannot know how it is going to be used. So therefore we have no responsibility. > Well, that is false. It is true that a computer, for example, can be used for good or evil. It is true that a helicopter can be used as a gunship and it can also be used to rescue people from a mountain pass. And if the question arises of how a specific device is going to be used, in what I call an abstract ideal society, then one might very well say one cannot know. > But we live in a concrete society, [and] with concrete social and historical circumstances and political realities in this society, it is perfectly obvious that when something like a computer is invented, then it is going to be adopted will be for military purposes. It follows from the concrete realities in which we live, it does not follow from pure logic. But we're not living in an abstract society, we're living in the society in which we in fact live. > If you look at the enormous fruits of human genius that mankind has developed in the last 50 years, atomic energy and rocketry and flying to the moon and coherent light, and it goes on and on and on -- and then it turns out that every one of these triumphs is used primarily in military terms. So it is not reasonable for a scientist or technologist to insist that he or she does not know -- or cannot know -- how it is going to be used. -- Joseph Weizenbaum
- _aavaa_ 3y ago
  I think a good starting point for this discussion is clarifying the implicit false dichotomy: it's either moral or moral, technology is either good or bad. To quote Melvin Kranzberg "Technology is neither good nor bad; nor is it neutral". Just because technology requires people to create it and to use it, does not mean that it has no impact on us (an by extension it also means that it's creation has impacts). To take an obvious example, guns are not inherently good or bad. But they unlock the possibility of shooting people.
  - bluefirebrand 3y ago
    If I invent something that can be used for good, but others use it for bad, does it reflect on my character? Does my good intentions absolve me of the harm my invention did? Or, the opposite? If I invent something fully intending to do harm with it, but others find non-harmful uses for it, does that absolve me of my original intentions to cause harm? A gun may not be good or bad, but the intention of the person who invented guns likely were good or bad.
    - haswell 3y ago
      This takes us back down the road of binary outcomes though. The intentions of the inventor are one thing to examine. The outcomes of the inventions another. And the notion that something is “moral”, may not have answers in practice, even if there might be an answer in principle. Even if those answers could be found, they could only be found after enough time had passed to judge the outcome fairly. But it is unpalatable to retroactively assign moral judgement for actions the originator could not have possibly predicted (there’s a different version of this thought experiment that imagines we could perfectly predict the future, but that seems less useful here). I do think that the moral peril of certain actions/decisions is proportional to the potential risk. There are few discoveries as dangerous as that of splitting atoms, and the more obvious the danger, the more reasonable it seems to judge the original act of invention. When questions like “will we explode the universe?” seem reasonable to ask, that seems like a good sign an invention is not something that should exist in an ideal world. Most inventions don’t have these kinds of stakes, and the effects of their use are local to the actors using them, at which point it seems more reasonable to judge the actions of individual users than the invention/inventor.
johnnyworker 3y ago
> I haven’t heard a good practical rebuttal though :/ The practical rebuttal to people justifying their deeds with sophistry a teenager can see through is to stop them, regardless of whether they like it or not, and not getting too tangled up in the chaff they came up with to rationalize what they are doing.
- startupsfail 3y ago
  A teenager probably would tell that ChatGPT helps her do homework and acts as a Pokémon. And she doesn’t understand why it’s compared to a bomb that was dropped onto Hiroshima (by the military) to stop the aggressor that had attacked the United States.
faeriechangling 3y ago
>there is a good chance that mutually assured destruction prevented war despite how terrifying the policy is. If I bet at the Casino using a martingale system I'll usually leave the Casino with more money than I came with. Doesn't mean that the Martingale system is a winning strategy or that I'm actually beating the house. MAD certainly prevents war, but that doesn't prove it's worth it.
deleted 3y ago
[deleted]
zardo 3y ago
> W.r.t. nukes, for example, there is a good chance that mutually assured destruction prevented war despite how terrifying the policy is. But Oppenheimer didn't spy for the Soviet Union. He worked to make the US the sole nuclear power, not to create a nuclear stand-off between opposing powers.
- solumunus 3y ago
  I’m certainly no expert on this, but even then surely this was seen as an inevitability? The US scientists knew Russians and Germans were approaching this, did they think their opponents would stop developing the capability just because they got there first? Seems unlikely that they would think that.
  - zardo 3y ago
    > I’m certainly no expert on this, but even then surely this was seen as an inevitability? There was one sure way to prevent the stand off, shoot before they draw.

by tyleo 3y ago

bluefirebrand 3y ago

It's also worth considering if the act of inventing something horrifying is actually immoral, or if using it for immoral purposes is. For instance, dynamite. We would not have the society we do now without dynamite. The fact that it can also be used to destroy property and kill people makes it a deadly invention, but was it immoral to invent? Of course dynamite is nothing compared to the Atomic Bomb. Or maybe inventing it for the purposes of killing people makes it immoral regardless of other possible applications. How much does intent matter, versus outcome? And then, does it absolve the inventor if they had pure intentions but others used their inventions to do harm? Is it all moot because trying to live a moral life when others are willing to be immoral will basically never work out in your favor?

innagadadavida 3y ago
We could also consider failure to prevent bad actors from doing immoral things as immoral itself. The problem with this line of thinking is there is no tight definition of immorality.
uwagar 3y ago
>Or maybe inventing it for the purposes of killing people makes it immoral regardless of other possible applications. this is the key. the atom bomb was made to kill people. i also felt a mandate driven ie arm twisting people until they vaccinated themselves against covid was also immoral.
- uwagar 3y ago
  talk about covid vaccine and get downvoted. doesnt matter what.
12907835202 3y ago
(1) if one day a nuke was used to deflect an asteroid and save the entire planet, would that then outweigh any deaths caused by nukes and ottenheimer would be a hero rather than a villain? That's a tricky road to go down. (2) as is trying to focus on intent. Do you want to risk absolving Hitler or Pol Pot because they thought they were making the world a better place?
- oh_sigh 3y ago
  Oppenheimer is a villain? First I'm hearing about it.
  - toyg 3y ago
    Probably not popular in Japan.
    - localplume 3y ago
      [dead]
- bluefirebrand 3y ago
  > (2) as is trying to focus on intent. Do you want to risk absolving Hitler or Pol Pot because they thought they were making the world a better place? I would argue that "trying to make the world better" is too broad to be an intent. It's an ideal, or something. Maybe that's just language semantics. As soon as you dig into that ideal at all, the malicious intent becomes extremely clear. Ideal: "I am going to make the world a better place" How? Intent: "By purging lesser humans so my people can become the master race as we should" Versus Intent: "By volunteering my time to set up soup kitchens in the inner city" Ideal leads to intent leads to action leads to outcome. My opinion, anyways.
johnnyworker 3y ago
> inventing it for the purposes of Regardless of how the sentence continues, there is no such thing. There is making inventions in the world as it currently is, and what follows from that. edit: since the page http://tech.mit.edu/V105/N16/weisen.16n.html http://tech.mit.edu/V105/N16/weisen.16n.html is down atm, I have to quote it instead of linking it: > People have a series of rationalizations. People say for example that science and technology have their own logic, that they are in fact autonomous. This particular rationalization is profoundly false. It is not true that science marches on in defiance of human will, independent of human will, that just is not the case. But it is comfortable, as I said: it leads to the position that "if I don't do it, someone else will." > Of course if one takes that as an ethical principle then obviously it can serve as a license to do anything at all. "People will be murdered; if I don't do it, someone else will." "Women will be raped; if I don't do it, someone else will." That is just a license for violence. > Other people say, and I think this is a widely used rationalization, that fundamentally the tools we work on are "mere" tools; This means that whether they get use for good or evil depends on the person who ultimately buys them and so on. > There's nothing bad about working in computer vision, for example. Computer vision may very well some day be used to heal people who would otherwise die. Of course, it could also be used to guide missiles, cruise missiles for example, to their destination, and all that. You see, the technology itself is neutral and value-free and it just depends how one uses it. And besides -- consistent with that -- we can't know, we scientists cannot know how it is going to be used. So therefore we have no responsibility. > Well, that is false. It is true that a computer, for example, can be used for good or evil. It is true that a helicopter can be used as a gunship and it can also be used to rescue people from a mountain pass. And if the question arises of how a specific device is going to be used, in what I call an abstract ideal society, then one might very well say one cannot know. > But we live in a concrete society, [and] with concrete social and historical circumstances and political realities in this society, it is perfectly obvious that when something like a computer is invented, then it is going to be adopted will be for military purposes. It follows from the concrete realities in which we live, it does not follow from pure logic. But we're not living in an abstract society, we're living in the society in which we in fact live. > If you look at the enormous fruits of human genius that mankind has developed in the last 50 years, atomic energy and rocketry and flying to the moon and coherent light, and it goes on and on and on -- and then it turns out that every one of these triumphs is used primarily in military terms. So it is not reasonable for a scientist or technologist to insist that he or she does not know -- or cannot know -- how it is going to be used. -- Joseph Weizenbaum
_aavaa_ 3y ago
I think a good starting point for this discussion is clarifying the implicit false dichotomy: it's either moral or moral, technology is either good or bad. To quote Melvin Kranzberg "Technology is neither good nor bad; nor is it neutral". Just because technology requires people to create it and to use it, does not mean that it has no impact on us (an by extension it also means that it's creation has impacts). To take an obvious example, guns are not inherently good or bad. But they unlock the possibility of shooting people.
- bluefirebrand 3y ago
  If I invent something that can be used for good, but others use it for bad, does it reflect on my character? Does my good intentions absolve me of the harm my invention did? Or, the opposite? If I invent something fully intending to do harm with it, but others find non-harmful uses for it, does that absolve me of my original intentions to cause harm? A gun may not be good or bad, but the intention of the person who invented guns likely were good or bad.
  - haswell 3y ago
    This takes us back down the road of binary outcomes though. The intentions of the inventor are one thing to examine. The outcomes of the inventions another. And the notion that something is “moral”, may not have answers in practice, even if there might be an answer in principle. Even if those answers could be found, they could only be found after enough time had passed to judge the outcome fairly. But it is unpalatable to retroactively assign moral judgement for actions the originator could not have possibly predicted (there’s a different version of this thought experiment that imagines we could perfectly predict the future, but that seems less useful here). I do think that the moral peril of certain actions/decisions is proportional to the potential risk. There are few discoveries as dangerous as that of splitting atoms, and the more obvious the danger, the more reasonable it seems to judge the original act of invention. When questions like “will we explode the universe?” seem reasonable to ask, that seems like a good sign an invention is not something that should exist in an ideal world. Most inventions don’t have these kinds of stakes, and the effects of their use are local to the actors using them, at which point it seems more reasonable to judge the actions of individual users than the invention/inventor.

johnnyworker 3y ago

> I haven’t heard a good practical rebuttal though :/ The practical rebuttal to people justifying their deeds with sophistry a teenager can see through is to stop them, regardless of whether they like it or not, and not getting too tangled up in the chaff they came up with to rationalize what they are doing.

startupsfail 3y ago
A teenager probably would tell that ChatGPT helps her do homework and acts as a Pokémon. And she doesn’t understand why it’s compared to a bomb that was dropped onto Hiroshima (by the military) to stop the aggressor that had attacked the United States.

faeriechangling 3y ago

>there is a good chance that mutually assured destruction prevented war despite how terrifying the policy is. If I bet at the Casino using a martingale system I'll usually leave the Casino with more money than I came with. Doesn't mean that the Martingale system is a winning strategy or that I'm actually beating the house. MAD certainly prevents war, but that doesn't prove it's worth it.

deleted 3y ago

[deleted]

zardo 3y ago

> W.r.t. nukes, for example, there is a good chance that mutually assured destruction prevented war despite how terrifying the policy is. But Oppenheimer didn't spy for the Soviet Union. He worked to make the US the sole nuclear power, not to create a nuclear stand-off between opposing powers.

solumunus 3y ago
I’m certainly no expert on this, but even then surely this was seen as an inevitability? The US scientists knew Russians and Germans were approaching this, did they think their opponents would stop developing the capability just because they got there first? Seems unlikely that they would think that.
- zardo 3y ago
  > I’m certainly no expert on this, but even then surely this was seen as an inevitability? There was one sure way to prevent the stand off, shoot before they draw.