5 ms·

Feynman, on working on the bomb: "With regard to moral questions, I do have something I would like to say about it. The original reason to start the project, w

by GuiA 11y ago

Feynman, on working on the bomb: "With regard to moral questions, I do have something I would like to say about it. The original reason to start the project, which was that the Germans were a danger, started me off on a process of action which was to try to develop this first system at Princeton and then at Los Alamos, to try to make the bomb work. All kinds of attempts were made to redesign it to make it a worse bomb and so on. It was a project on which we all worked very, very hard, all co-operating together. And with any project like that you continue to work trying to get success, having decided to do it. But what I did—immorally I would say—was to not remember the reason that I said I was doing it, so that when the reason changed, because Germany was defeated, not the singlest thought came to my mind at all about that, that that meant now that I have to reconsider why I am continuing to do this. I simply didn't think, okay?" (from "The Pleasure of Finding Things Out", transcript here: http://www.worldcat.org/wcpa/servlet/DCARead?standardNo=0738201081&standardNoType=1&excerpt=true http://www.worldcat.org/wcpa/servlet/DCARead?standardNo=0738...) This is extremely idealistic, but we need a way for engineers and scientists to feel accountable for the outcomes of their work, and to straight out refuse working on such projects. And the people who do work on such systems should be held accountable in some deep way. We have reached a developmental stage where building tools and techniques in the active goal of harming human lives has become morally unacceptable. Engaging in civil disobedience if you are working on such projects is the only acceptable outcome; Snowden should be remembered as the first of many, not as an exception. (yes, there are many counterpoints to my argument, but starting debates is more interesting than spewing out platitudes. I'm interested in reading the replies)

clarkmoody 11y ago
> straight out refuse working on such projects People do. Then they quit their jobs and are replaced by other smart people willing to do the work (and needing the job). > people who do work on such systems should be held accountable in some deep way Never going to happen. The political and military leaders are the ones who choose to develop and deploy such weapons. They should be held accountable, and sometimes they are. Should we go out and prosecute all the engineers and scientists who worked on nuclear bombs that have been sitting in bunkers and silos for the past 60 years? > We have reached a developmental stage where building tools and techniques in the active goal of harming human lives has become morally unacceptable. Who is "we"? A gun is specifically designed to kill things, but the wielder of the gun decides whether it will be used for good or for evil. Likewise, there are plenty of other objects not designed to kill people that are used for that purpose (stones, rope, buckets of water, etc). Would you consider working on AI countermeasures? Would you want to have a strong defense that can fend off AI invaders, even if it means that defensive force could be re-purposed for offense? > This is extremely idealistic... Ideally, you want to rid the world of conflict and war. But this is impossible while there remain limited resources and different ideas. You would need to find an infinite source of food/water/land as well as force everyone to conform to one ideology to avoid war. So aside from being impossible (as far as resources go), you would need a totalitarian world government imposing thought control on all of humanity to bring about such a "peace."
return0 11y ago
Have we reached that point? Most would disagree. It is also dishonest to attribute the blame to scientists, the real burden is always with the ones giving the orders. Scientists working in military projects create a threat at best. Is threat morally unacceptable? The answer is again no, or else diplomacy could not work. Humans are not an angel society. The key is to keep the threats on the hands of responsible leaders/societies.
julianpye 11y ago
I once worked in a German field engineering department of a large US semiconductor company as a student. In the department there was a noticeable barrier between one manager and the engineers. The following had happened there a few years ago: a client required a DSP to calculate the weight on a landmine switch. The departments engineers refused to work for the client bar one manager. They were threatened to be fired and they stayed on course and ended up keeping their jobs. The way it worked was by one guy rallying, taking apart the specifications and explaining the actual moral implications to the engineers.
- kstop 11y ago
  They were probably legally in the right too, considering that Germany is a signatory to the Mine Ban Treaty: https://en.wikipedia.org/wiki/Ottawa_Treaty https://en.wikipedia.org/wiki/Ottawa_Treaty
  - deleted 11y ago
    [deleted]
- chinathrow 11y ago
  Thank you. A million trillion times this. If people developing software, hardware and support systems for use in war - stand up and leave your job including by telling everyone who listens why you did. Telling you have a morale is never wrong. And it greatly empowers others to follow. I used to work for a supplier to an aerospace/defense company and did it. It makes you sleep better at night, I promise.
jgome 11y ago
Einstein and many nuclear scientists and engineers that participated in the project felt betrayed by the U.S. dropping the bomb in Japan, they worked on the making of the bomb, expecting that it would serve as a deterrent, not as a weapon. Because of this, some fled to the USSR and China. But even if the scientist/engineers wanted to stop the use of the bomb after working on it, they couldn't have, because it's a political decision. That's why this has to be stopped even before the weapons race starts, IMO.
noahlt 11y ago
> we need a way for engineers and scientists to feel accountable for the outcomes of their work, and to straight out refuse working on such projects. One difficulty, not just here but with pacifism in general, is the asymmetry of violence. For your scheme to work, we would need close to 100% cooperation from engineers. For war hawk politicians to get their weapons, they need just a handful of engineers. "Today we were unlucky, but remember we only have to be lucky once. You will have to be lucky always." — https://en.wikipedia.org/wiki/Brighton_hotel_bombing https://en.wikipedia.org/wiki/Brighton_hotel_bombing
- ZenoArrow 11y ago
  Two things. 1. Not all engineers are equal. The level of engineering talent you need for this type of work is relatively high. Let's keep it that way. 2. This isn't directed at you in particular, but this line of thinking pisses me off. People should act based on their principals, even if you aren't guaranteed success. If it's something you believe in, you can resist, and resist peacefully. Education and awareness is key to this resistance. Culture is ours to shape together, and even if the prevailing culture makes derailing autonomous weaponry hard, let's not give up just yet.
Sven7 11y ago
Take a look at the history of the Manhattan project. It's a fascinating case study of how the most brilliant minds on the planet, can end up on opposite sides of a moral question. For every Robert Oppenheimer who has a doubt and raises the moral question, there is an Edward Teller who is charging ahead doubt free. At the end of the story the strength of the personalities mattered much more than the soundness of their arguments.
- johngalt 11y ago
  Not simply charging ahead scientifically, but actively hawkish. "If you say why not bomb them tomorrow, I say why not today? If you say today at five o' clock, I say why not one o' clock?" - John von Neumann
lotharbot 11y ago
> "straight out refuse working on such projects" One of my family members turned down an offer of double his salary because it would entail working on military systems, and he's a conscientious objector. > "the people who do work on such systems should be held accountable in some deep way" ... another of my family members has worked on autonomous military systems, and believes herself to be a viable military target because of it. > "building tools and techniques in the active goal of harming human lives has become morally unacceptable. Engaging in civil disobedience if you are working on such projects is the only acceptable outcome" The two people I referenced above have a deep, thoughtful, respectful disagreement. Your version is incredibly oversimplified. (For a taste, see the responses to https://news.ycombinator.com/item?id=1823802 https://news.ycombinator.com/item?id=1823802 .)
- omginternets 11y ago
  >One of my family members turned down an offer of double his salary because it would entail working on military systems, and he's a conscientious objector. This happens a lot in my field (cognitive neuroscience). The Army Research Lab is a huge recruiter at cognitive neuroscience conventions, but there are many who refuse to work with them on principle, citing gitmo/drones/abu grahib/mk ultra/extraordinary rendition/etc... At the end of the day, Army research tends to be shittier than public research, and I attribute this to two equally-weighted factors: 1. Army research is done in relative isolation. Collaboration of ideas is difficult because of OPSEC rules. 2. The best researchers, by and large, tend to be wary of Army laboratories, in no small part because the majority of them are foreign.
glennsl 11y ago
> we need a way for engineers and scientists to feel accountable for the outcomes of their work, and to straight out refuse working on such projects This is one of the major reasons why I'm in favor of an unconditional basic income. We can't hold people responsible for doing their job if they're not realistically allowed to refuse it. Only by allowing them to say no, by providing an alternative, in this case an unconditional income to cover their basic needs, can we allow ourselves to hold them responsible for not saying no.
- Zarathustra30 11y ago
  I had never even thought of this argument before. Ensuring Hobson's choice is an actual choice. (Though I take issue with the term "income". That implies it can be used for luxury goods. I'll pay for people's toilet paper. I won't pay for double-ply.)

by GuiA 11y ago

clarkmoody 11y ago

> straight out refuse working on such projects People do. Then they quit their jobs and are replaced by other smart people willing to do the work (and needing the job). > people who do work on such systems should be held accountable in some deep way Never going to happen. The political and military leaders are the ones who choose to develop and deploy such weapons. They should be held accountable, and sometimes they are. Should we go out and prosecute all the engineers and scientists who worked on nuclear bombs that have been sitting in bunkers and silos for the past 60 years? > We have reached a developmental stage where building tools and techniques in the active goal of harming human lives has become morally unacceptable. Who is "we"? A gun is specifically designed to kill things, but the wielder of the gun decides whether it will be used for good or for evil. Likewise, there are plenty of other objects not designed to kill people that are used for that purpose (stones, rope, buckets of water, etc). Would you consider working on AI countermeasures? Would you want to have a strong defense that can fend off AI invaders, even if it means that defensive force could be re-purposed for offense? > This is extremely idealistic... Ideally, you want to rid the world of conflict and war. But this is impossible while there remain limited resources and different ideas. You would need to find an infinite source of food/water/land as well as force everyone to conform to one ideology to avoid war. So aside from being impossible (as far as resources go), you would need a totalitarian world government imposing thought control on all of humanity to bring about such a "peace."

return0 11y ago

Have we reached that point? Most would disagree. It is also dishonest to attribute the blame to scientists, the real burden is always with the ones giving the orders. Scientists working in military projects create a threat at best. Is threat morally unacceptable? The answer is again no, or else diplomacy could not work. Humans are not an angel society. The key is to keep the threats on the hands of responsible leaders/societies.

julianpye 11y ago

I once worked in a German field engineering department of a large US semiconductor company as a student. In the department there was a noticeable barrier between one manager and the engineers. The following had happened there a few years ago: a client required a DSP to calculate the weight on a landmine switch. The departments engineers refused to work for the client bar one manager. They were threatened to be fired and they stayed on course and ended up keeping their jobs. The way it worked was by one guy rallying, taking apart the specifications and explaining the actual moral implications to the engineers.

kstop 11y ago
They were probably legally in the right too, considering that Germany is a signatory to the Mine Ban Treaty: https://en.wikipedia.org/wiki/Ottawa_Treaty https://en.wikipedia.org/wiki/Ottawa_Treaty
- deleted 11y ago
  [deleted]
chinathrow 11y ago
Thank you. A million trillion times this. If people developing software, hardware and support systems for use in war - stand up and leave your job including by telling everyone who listens why you did. Telling you have a morale is never wrong. And it greatly empowers others to follow. I used to work for a supplier to an aerospace/defense company and did it. It makes you sleep better at night, I promise.

jgome 11y ago

Einstein and many nuclear scientists and engineers that participated in the project felt betrayed by the U.S. dropping the bomb in Japan, they worked on the making of the bomb, expecting that it would serve as a deterrent, not as a weapon. Because of this, some fled to the USSR and China. But even if the scientist/engineers wanted to stop the use of the bomb after working on it, they couldn't have, because it's a political decision. That's why this has to be stopped even before the weapons race starts, IMO.

noahlt 11y ago

> we need a way for engineers and scientists to feel accountable for the outcomes of their work, and to straight out refuse working on such projects. One difficulty, not just here but with pacifism in general, is the asymmetry of violence. For your scheme to work, we would need close to 100% cooperation from engineers. For war hawk politicians to get their weapons, they need just a handful of engineers. "Today we were unlucky, but remember we only have to be lucky once. You will have to be lucky always." — https://en.wikipedia.org/wiki/Brighton_hotel_bombing https://en.wikipedia.org/wiki/Brighton_hotel_bombing

ZenoArrow 11y ago
Two things. 1. Not all engineers are equal. The level of engineering talent you need for this type of work is relatively high. Let's keep it that way. 2. This isn't directed at you in particular, but this line of thinking pisses me off. People should act based on their principals, even if you aren't guaranteed success. If it's something you believe in, you can resist, and resist peacefully. Education and awareness is key to this resistance. Culture is ours to shape together, and even if the prevailing culture makes derailing autonomous weaponry hard, let's not give up just yet.

Sven7 11y ago

Take a look at the history of the Manhattan project. It's a fascinating case study of how the most brilliant minds on the planet, can end up on opposite sides of a moral question. For every Robert Oppenheimer who has a doubt and raises the moral question, there is an Edward Teller who is charging ahead doubt free. At the end of the story the strength of the personalities mattered much more than the soundness of their arguments.

johngalt 11y ago
Not simply charging ahead scientifically, but actively hawkish. "If you say why not bomb them tomorrow, I say why not today? If you say today at five o' clock, I say why not one o' clock?" - John von Neumann

lotharbot 11y ago

> "straight out refuse working on such projects" One of my family members turned down an offer of double his salary because it would entail working on military systems, and he's a conscientious objector. > "the people who do work on such systems should be held accountable in some deep way" ... another of my family members has worked on autonomous military systems, and believes herself to be a viable military target because of it. > "building tools and techniques in the active goal of harming human lives has become morally unacceptable. Engaging in civil disobedience if you are working on such projects is the only acceptable outcome" The two people I referenced above have a deep, thoughtful, respectful disagreement. Your version is incredibly oversimplified. (For a taste, see the responses to https://news.ycombinator.com/item?id=1823802 https://news.ycombinator.com/item?id=1823802 .)

omginternets 11y ago
>One of my family members turned down an offer of double his salary because it would entail working on military systems, and he's a conscientious objector. This happens a lot in my field (cognitive neuroscience). The Army Research Lab is a huge recruiter at cognitive neuroscience conventions, but there are many who refuse to work with them on principle, citing gitmo/drones/abu grahib/mk ultra/extraordinary rendition/etc... At the end of the day, Army research tends to be shittier than public research, and I attribute this to two equally-weighted factors: 1. Army research is done in relative isolation. Collaboration of ideas is difficult because of OPSEC rules. 2. The best researchers, by and large, tend to be wary of Army laboratories, in no small part because the majority of them are foreign.

glennsl 11y ago

> we need a way for engineers and scientists to feel accountable for the outcomes of their work, and to straight out refuse working on such projects This is one of the major reasons why I'm in favor of an unconditional basic income. We can't hold people responsible for doing their job if they're not realistically allowed to refuse it. Only by allowing them to say no, by providing an alternative, in this case an unconditional income to cover their basic needs, can we allow ourselves to hold them responsible for not saying no.

Zarathustra30 11y ago
I had never even thought of this argument before. Ensuring Hobson's choice is an actual choice. (Though I take issue with the term "income". That implies it can be used for luxury goods. I'll pay for people's toilet paper. I won't pay for double-ply.)