12 ms·

From the movie "Flash of Genius" about Robert Kearns, the inventor of the intermittent windshield wiper: "I can't think of a job or a career where the understa

From the movie "Flash of Genius" about Robert Kearns, the inventor of the intermittent windshield wiper: "I can't think of a job or a career where the understanding of ethics is more important than engineering," Dr. Kearns continues. "Who designed the artificial aortic heart valve? An engineer did that. Who designed the gas chambers at Auschwitz? An engineer did that, too. One man was responsible for helping save tens of thousands of lives. Another man helped kill millions." "Now, I don't know what any of you are going to end up doing in your lives," Dr. Kearns says, "but I can guarantee you that there will come a day when you have a decision to make. And it won't be as easy as deciding between a heart valve and a gas chamber." To me this is incredibly valid for Silicon Valley engineers these days.

hyperdunc 9y ago
Whether military AI endsup being closer to a heart valve or a gas chamber, it's going to be built first by some nation. But there is space for ethical contemplation during an arms race.
- killjoywashere 9y ago
  Absolutely. As consequentialist first move, let me suggest that a first order principle for the US military is "Sweat now or bleed later". Better to work your ass off to develop superior tactics, discipline, and technology now than die later. No one hates war more than soldiers.
  - jstanley 9y ago
    If soldiers hate war so much why do they join up?
    - hueving 9y ago
      The best way out of a terrible life for many poor people. If you didn't do great in school and don't have rich parents to pay for college, the military in the US will look great with its education perks and chance to "see the world".
    - pteredactyl 9y ago
      They believe in their country, what it represents, and are willing to sacrifice their life for it. In the US, it's the Constitution, to which they make an oath to upon joining. Or they're broke, have their back against the wall, and can get good training and education through the military.
    - gaius 9y ago
      Possibly because the one thing worse than war is being invaded, history has shown that
    - geodel 9y ago
      Same reason most people work on jobs they hate. Gotta work for money to live.
    - ionised 9y ago
      Predatory sign-up practices in low-income neighbourhoods, and a systemic and relentless propaganda campaign starting in nursery school that instils jingoist rhetoric and military worship.
      - supertrope 9y ago
        To play devil's advocate, we (USA) have a volunteer military. Even if you think it's a "poverty draft" driven by structural unemployment and dearth of opportunities in left behind cities and rural areas it's still not conscription. How should the military fill its ranks? Would you rather have a lottery?
microcolonel 9y ago
> Who designed the gas chambers at Auschwitz? An engineer did that, too. Well, given how unimaginative a leap the national socialists took from the national fumigation program to the gas chambers, I doubt there was much more to it than "take bug poison room tech, use it on humans". There's no such thing as an "engineer" in the real world, and there's almost always somebody more unscrupulous, devout, or patriotic than you who knows or is willing to learn what it takes to finish the job, and school is not going to get in the way. Added: Engineering is a practice, not an immutable identity. Even if you publicly shame all the "engineers" away from contributing to an effort you disagree with, replacing them is a matter of learning enough of the practice to complete the task. Even if we were just talking about what is in paper books today, it's hard to say that anyone could have control over who is or isn't capable of engineering efforts.
- drtz 9y ago
  > ... there's almost always somebody more unscrupulous, devout, or patriotic than you who knows or is willing to learn what it takes to finish the job ... If you believe that doing something is wrong, the idea that someone more unscrupulous will do it if you do not must not be used as an excuse for doing that thing yourself. Giving in to this notion will prove yourself to be the one who is unscrupulous.
  - microcolonel 9y ago
    > If you believe that doing something is wrong, the idea that someone more unscrupulous will do it if you do not must not be used as an excuse for doing that thing yourself. By involving yourself, you gain the opportunity to do the right thing (for the given the circumstances). If quitting accomplishes the same thing as sabotage or internal lobbying, then by all means.
  - bb88 9y ago
    In 2018, the following statements are true: 1. If you're on an active battlefield, both sides have already agreed that killing people is morally justified. 2. If the enemy is shooting at you, he is not interested in the ethics of killing someone on a battlefield. 3. If ethics helped wars end, there would be a US Army 27th Ethics Brigade to parachute in, but there's not and probably never will be. If on a battlefield, given a choice between holding scrupulous ideals and living, most people would choose living.
sillysaurus3 9y ago
The reason Rome was so influential was its military. Ditto for the US. Are you sure it's a good idea to become less influential on the world stage?
- microcolonel 9y ago
  > The reason Rome was so influential was its military. Ditto for the US. I think people often have that twisted. A lot of the U.S.'s expansion is cultural, not military. That is to say: people liked what they were doing, and became a fan. Think: music, movies, and to some extent, literature, and now you might add software and consumer electronics to that list. And even when looking at Rome, can you imagine how much harder those repeated annexations would have been if the Roman culture wasn't respected? I'd say that being capable at least of defending oneself, independently, from most individual powers and believable coalitions, creates a certain amount of stability.
  - maroonblazer 9y ago
    Who can project force like the U.S. military? No one. For the most part it's used to enable people to have more say in how they're governed, modulo certain constraints. That's not to say we do it perfectly but we shouldn't underestimate how positively influential that military power is.
    - pteredactyl 9y ago
      Agreed. And it's a position that I don't think many people realize.
    - smsm42 9y ago
      In USSR, before it collapsed, young people dreamt of having jeans (not everybody could get or afford them), listened to Western music and watched Western movies. Not because US army has secretly occupied USSR, but because Western culture was more attractive to them. Not everything is done with the military.
      - 9y ago
Abishek_Muthian 9y ago
>"One man was responsible for helping save tens of thousands of lives. Another man helped kill millions." In the case of Fritz Haber, it's the same man who saved billions from hunger and killed millions by prolonging World War-I. Excellent article on the same - https://medium.com/the-mission/the-tragedy-of-fritz-haber-the-monster-who-fed-the-world-ec19a9834f74 https://medium.com/the-mission/the-tragedy-of-fritz-haber-th... Sometimes, the decision isn't clear as black & white; especially when you mix nationalism & patriotism into it.
- sundvor 9y ago
  That was a thoroughly depressing read, but a most excellent one as such. Thanks.
- deft 9y ago
  That's totally different. He chose to kill the people and did it intentionally out of hate / pride for his country. This thread and the posted link are discussing whether something is immoral or not. He knew it would kill a ton of people and designed it to on purpose. That situation is totally black and white. Mass intentional killing is wrong, pretty much everyone agrees with that.
  - peteretep 9y ago
    > Mass intentional killing is wrong, pretty much everyone agrees with that Nope. Bomber Command in WWII and the use of nuclear weapons in WWII are both examples where you will find nuanced, intelligent debate on both sides of the argument of whether "mass, intentional killing" being wrong or not. What you describe as "hate / pride for his country" ... but isn't that the reason most soldiers kill? That you even bother to call this out shows that you know the reasons for killing people need to be taken into account. I'm not making a statement on whether or not mass killing is wrong here, simply pointing out what seems obvious: it's far from black and white.
    - SeriousM 9y ago
      "intelligent debate on both sides of the argument of whether "mass, intentional killing" being wrong or not" I'm sorry, but this sentence contains already the problem.
cmontella 9y ago
As a roboticist at the beginning of my career working on drones, I decided then and there that I would never make "bombs", a metaphor I used to mean anything that could be weaponized. I realized a lot of the work I was doing was funded by DARPA, and I was very cognizant about my research being used in this way. And like Dr. Kearns suggests, it's not entirely black and white. Would my path planning algorithm be used to more efficiently deliver scientific payloads to the atmosphere, or would it be used to route missiles to maximize casualties? Hard to really say, but I've avoided overtly military applications (even things like BigDog, designed to carry equipment for troops). Sometimes the distinction is even more insidious. I did work on perpetual flight for drones, and Facebook had a perpetual flight project that had the goal of bringing internet access to remote locations in Africa. Sounds humanitarian, but I also didn't want to be responsible for subjecting poor Africans to what I consider Facebook's panopticon. Or maybe it would have been a net boon for the region? It's really hard to tell a priori, so the best I have managed for myself is just to try and stay in theory, where developments are further removed from direct consequence.
- dweekly 9y ago
  As an engineer, founder, and investor I'll note that there's an ethical mandate for investors as well; when I started drone.vc I committed as a Quaker to not investing in "offensive" technologies. Separately, RE: bringing internet to new places, there's reasonable evidence to show that introducing broadband lifts GDP[1], so whatever you may think of Facebook, giving people high-speed connectivity to the whole Internet is probably a net positive. Disclaimer: I helped start Facebook's Connectivity Lab but no longer work there. [1] http://pubdocs.worldbank.org/en/391452529895999/WDR16-BP-Exploring-the-Relationship-between-Broadband-and-Economic-Growth-Minges.pdf http://pubdocs.worldbank.org/en/391452529895999/WDR16-BP-Exp...
  - cbHXBY1D 9y ago
    Most people on this site have not trusted FB's intentions with the drone project for good reason. Ever since we collectively realized a three years ago that FB's internet.org was a wolf in sheep's clothing, most people on here have been distrusting of FB.
unclebucknasty 9y ago
>incredibly valid for Silicon Valley engineers these days If one could engineer a solution to save lives, but instead uses his/her talents to build the nth iteration of, say, a food delivery app, does he/she bear any ethical culpability? Because it seems the bigger challenge for engineers is not avoiding overt evil, but in the opportunity cost of spending their time on work of significantly lesser consequence than their talents might allow.
- matuszeg 9y ago
  So youre saying we need another messaging app?
- lsc 9y ago
  I think you're taking the idea too far. look at what the civil engineers have: https://www.asce.org/code-of-ethics/ https://www.asce.org/code-of-ethics/ Yes, yes, we have a responsibility to not defraud people, and to not work on obviously evil projects, and even to speak out if your employer is, say, building a bridge that is not up to the clearly defined, established standards. (which is super different from when you think a bridge will collapse, but can't explain why in a way that other people can understand. This parenthetical part is particularly important, I think.) But we are free humans, and if we want to waste our lives on trivial things, so long as those trivial things don't hurt others, that's our choice. We are humans, and humans need entertainment, we need play.
  - unclebucknasty 9y ago
    >look at what the civil engineers have Well, that's a narrow view of ethical guidelines in conducting one's work. We all generally agree that people shouldn't be evil. My question sits above that. Is it enough for a moral society to simply decline to do evil? Or do we have a responsibility beyond that? And if it's the latter, then whose responsibility is it?
    - lsc 9y ago
      Declining to do evil, with real standards for what that means is, at the very least, a good first step, and a possible one.
didibus 9y ago
Why would the engineers be more burdened with the ethics of their creation over the investors, founders, managements, lawyers, execs, CEOs and government policy makers who all contributed to, promoted and made happen its fabrication by engineers?
- cmyr 9y ago
  They aren't more burdened, but they are the last line of defence.
- confounded 9y ago
  > investors, founders, managements, lawyers, execs, CEOs and government policy makers Power, corruption and lies. Those people often stand to personally (financially) benefit from unethical business decisions. Engineers have to think through all the dirty consequences, and still draw pretty much the same salary.
- aangjie 9y ago
  Ah.. the causality.. the argument can be made that without engineers the creation would never happen, (even if others are still trying to make money without ethics..).
- solutron 9y ago
  For the scope of this audience, I don't think it's necessarily a question of who's more responsible than the other. You're right, all the parties you've mentioned are just as culpable for their role in wrong-doing, even if it's abstracted or removed from implementation. As engineers though, we're in the unique position of being the gate-keeper between ideas, "visions" if you will (in the parlance of our times), and fruition. We're a gatekeeper between reality and the delusional musings afforded to people who aren't the engineers. We can say, "Fuck no, that should literally not exist or be built in any way", and do it loudly to stop the other parties from realizing their abusive vision. Or under dire circumstances, we can sabotage systems that need to be broken if it's too late to stop them from being created.
MichaelMoser123 9y ago
just a reminder: you are writing this over the internet - that one was funded by the pentagon too (darpanet) http://ccr.sigcomm.org/archive/1995/jan95/ccr-9501-clark.pdf http://ccr.sigcomm.org/archive/1995/jan95/ccr-9501-clark.pdf Here they say that the design of the internet was very much determined by the military priorities of the project (Telcos had different priorities so they came up with OSI where the link layer is supposed to be reliable, as opposed to DARPANET/IP, where the link layer/IP routing is dumb and the endpoint is smart; this allowed link layer to be pluggable and IP routers to be cheap)
- betolink 9y ago
  There is a huge difference between creating a network protocol and designing autonomous killer drones.
  - mrlala 9y ago
    Really? The killer drones couldn't communicate without said network protocols.
    - ByThyGrace 9y ago
      The point being that "killer drones" are by sole purpose immoral. You can hardly find a humanitarian use for them, while network protocols are pretty much amoral. You can use them for bad, but also for good. Please excuse my manichaeism.
      - TangoTrotFox 9y ago
        One could say the same of atomic weapons, yet it's likely that nuclear weapons have saved more lives that we can even imagine. The impact of war was accelerating in raw numbers, and increasing in relative numbers, all the way up to WW2. 3-4% of the world's population was killed in WW2. In today's terms that would be ~270 million people. Think about that number for a minute. Let's put it in scales of 9/11s. During 9/11 2,996 people were killed. Our scaled WW2 had 90,120 times more deaths. That number is so absurdly large it's difficult to grasp. So put another way that's a 9/11 scale event every single day for 246 years. The United States, as a nation, was founded 242 years to give that number some scale. Just think about that! If each generation has children at 30 years old then another way of seeing that number would be a 9/11 event each and every day from today until 2 years after your great great great great great great great grandchild is born. But then came the nuke. Nuclear weapons make traditional war between developed and nuclear capable nations basically impossible. Think about what the Cold War really was, or even what the geopolitical 'disagreements' of today are. Those are World Wars 3 and 4, averted because these wars would be unwinnable by any side. Like Einstein said, 'I'm not sure what World War 3 will be fought with, but the 4th will be fought with sticks and stones!' Did the people developing nukes understand that they would finally create a weapon that immensely powerful that it would deter open unrestrained war for decades to come? It's possible, but I think that their motivation was something more straight forward - gain military superiority in the present, probably also mixed with a bit of scientific curiosity about the challenge of creating a nuclear bomb. It's difficult to predict the future. The most awful and aweful weapon created in the history of mankind ended up creating the most unprecedented period of peace for mankind as well. This is also why I vehemently oppose nuclear disarmament. That's how you get WW3 if people actually disarm, though in reality it'd simply likely result in nations obfuscating their nuclear weapon programs and facilities. It's just a molten salt thorium reactor guys, come on - breeders are awesome!
Demiurge 9y ago
The engineers in both cases are not essential to the outcomes. Engineers or craftsmen, or any other workers execute tasks put forth to them by society that gave them birth and education. Any person in the society has some influence, but clearly more influence is the hands of the leaders and organizers. But, there is power in organizing, or unionizing, for collective bargaining.
- bobthepanda 9y ago
  That defense didn’t work at Nuremberg.
  - Demiurge 9y ago
    It's not a defense. It doesn't absolve anyone of personal responsibility.
- cmontella 9y ago
  This reminds me of the movie "Cube", which is about a sadistic torture chamber built by accident through a network of government sub-contractors who each built some innocuous part of the thing, which was then assembled through layers of opaque bureaucracy. It's a terrible film, but I always thought that premise was hilarious. https://en.wikipedia.org/wiki/Cube_(film) https://en.wikipedia.org/wiki/Cube_(film)
fsloth 9y ago
I don't think you can ever blame the engineer who designs the weapon, unless it is obviously ethically problematic like, a better iron maiden (as an example, I know it's probably a hoax device). Merely offering a force projecting capability is not evil. If you look at world history, the only concrete definition of who has most political power is this: it's the party with the most capability for projecting violence in a given area. Small close knit groups that live outside of large populations can live independently and do without military force. Once you have a large population with more or less fixed hearth and home the party with most violence projection capability owns them politically. This dynamic is played in almost ridiculously minuscule scale in the peloponnesian wars of the greek city states of antiquity. The pathological manifestation of this principle can be viewed, say, in the rise of ISIS and the Somali warlords. While a modern state is seldom a benevolent actor, at least in the western countries it's the best of known alternatives. In this framework violence is a key tool of the state, just like good governance, tax collection, etc. Given that violence is a necessity, in my opinion designing military instruments in itself is not evil. It's not as good as designing new vaccines, so there is some ethical scale in the matter, but I'm damned if I can put it in concrete terms.
numismatex 9y ago
You think it's a matter for engineers, I think it's a matter for executives and controlling interests. If you think that if the engineers refuse to do it that it won't get done, or that if Google doesn't do it, someone else won't, you're nuts.
grangerize 9y ago
I do not think that this is a simple black and white separation,but rather a dialectical one. I used to find myself most of the times confused while using technologies whose CEOs, mission I do not agree with. It felt as if I was contradicting myself. But then I realized the dialectical nature of this relationship which is if you want to create something better, improve something it is only natural that you will do so using the existing tools available in your world. A great example is Facebook. There are things you might not agree with, you might hate it but if you look closer you will see volunteer groups to political campaigns to marches and protests which are organized only via Facebook. One of such political activities one day could as well change Facebook or the world we live in drastically. You can apply this to other cases. You might have an anti-capitalist philosophy podcast hosted on Amazon or Spotify or Apple which in essence contradicts the very existence of such companies. But in the end you are using them to deliver your message, cause change etc. I am curious about what others think about this and how they feel about looking at it from a dialectical point of view as I am continuously thinking about this and still forming my thoughts.
RcouF1uZ4gsC 9y ago
From the Wikipedia article on Robert Kearns https://en.m.wikipedia.org/wiki/Robert_Kearns https://en.m.wikipedia.org/wiki/Robert_Kearns >Kearns was a member of the Office of Strategic Services, the forerunner of the U.S. CIA, during World War II.[1][2][3] Somehow I don't think that Kearns would have had an issue with wriing software to analyze video footage for the military.
philwelch 9y ago
To me, the salient point of that analogy is that the engineer designed execution chambers for Nazis. The engineers who designed the Sherman tank, the Garand rifle, or the Spitfire fighter plane also built weapons, weapons that killed people, but the judgment of history is that those people needed to be killed and thank God for the engineers who made that possible. So, there are two aspects of that: first, do you trust the people you're designing a weapon for to use it for justifiable purposes, and second, how does the weapon itself influence whether it can be used for just or unjust reasons? Gas chambers, for example, can only be used to kill someone whom you are already able to coerce into entering it; its only use is to murder a helpless prisoner. A Spitfire or Hurricane can shoot down bombers but not necessarily do that much damage on the ground. Orwell had an interesting perspective on this question: "Thus, for example, tanks, battleships and bombing planes are inherently tyrannical weapons, while rifles, muskets, long-bows, and hand-grenades are inherently democratic weapons. A complex weapon makes the strong stronger, while a simple weapon — so long as there is no answer to it — gives claws to the weak." I really don't have answers. The stance of, "weapons equals bad, therefore it's wrong to ever help design weapons" is a pacifist notion, and maybe that appeals to you, but if you're not a pacifist, it becomes a complicated question with certain political ramifications.
- qkls 9y ago
  The judgement of history is always that those people needed to be killed, because winners write the history.
  - philwelch 9y ago
    Not always, but I figured WWII would be a relatively uncontroversial example.
paulddraper 9y ago
> And it won't be as easy as deciding between a heart valve and a gas chamber That's the truth. Did the engineers of the atomic bomb save millions of lives and usher in an era of unprecedented peace? Or did they slaughter millions of innocents and sow the seeds of humanity's destruction? It's been 70 years; we've had plenty of time to make up our minds. Many inventions are not so easily classified as innately good or evil.

hyperdunc 9y ago

Whether military AI endsup being closer to a heart valve or a gas chamber, it's going to be built first by some nation. But there is space for ethical contemplation during an arms race.

killjoywashere 9y ago
Absolutely. As consequentialist first move, let me suggest that a first order principle for the US military is "Sweat now or bleed later". Better to work your ass off to develop superior tactics, discipline, and technology now than die later. No one hates war more than soldiers.
- jstanley 9y ago
  If soldiers hate war so much why do they join up?
  - hueving 9y ago
    The best way out of a terrible life for many poor people. If you didn't do great in school and don't have rich parents to pay for college, the military in the US will look great with its education perks and chance to "see the world".
  - pteredactyl 9y ago
    They believe in their country, what it represents, and are willing to sacrifice their life for it. In the US, it's the Constitution, to which they make an oath to upon joining. Or they're broke, have their back against the wall, and can get good training and education through the military.
  - gaius 9y ago
    Possibly because the one thing worse than war is being invaded, history has shown that
  - geodel 9y ago
    Same reason most people work on jobs they hate. Gotta work for money to live.
  - ionised 9y ago
    Predatory sign-up practices in low-income neighbourhoods, and a systemic and relentless propaganda campaign starting in nursery school that instils jingoist rhetoric and military worship.
    - supertrope 9y ago
      To play devil's advocate, we (USA) have a volunteer military. Even if you think it's a "poverty draft" driven by structural unemployment and dearth of opportunities in left behind cities and rural areas it's still not conscription. How should the military fill its ranks? Would you rather have a lottery?

microcolonel 9y ago

> Who designed the gas chambers at Auschwitz? An engineer did that, too. Well, given how unimaginative a leap the national socialists took from the national fumigation program to the gas chambers, I doubt there was much more to it than "take bug poison room tech, use it on humans". There's no such thing as an "engineer" in the real world, and there's almost always somebody more unscrupulous, devout, or patriotic than you who knows or is willing to learn what it takes to finish the job, and school is not going to get in the way. Added: Engineering is a practice, not an immutable identity. Even if you publicly shame all the "engineers" away from contributing to an effort you disagree with, replacing them is a matter of learning enough of the practice to complete the task. Even if we were just talking about what is in paper books today, it's hard to say that anyone could have control over who is or isn't capable of engineering efforts.

drtz 9y ago
> ... there's almost always somebody more unscrupulous, devout, or patriotic than you who knows or is willing to learn what it takes to finish the job ... If you believe that doing something is wrong, the idea that someone more unscrupulous will do it if you do not must not be used as an excuse for doing that thing yourself. Giving in to this notion will prove yourself to be the one who is unscrupulous.
- microcolonel 9y ago
  > If you believe that doing something is wrong, the idea that someone more unscrupulous will do it if you do not must not be used as an excuse for doing that thing yourself. By involving yourself, you gain the opportunity to do the right thing (for the given the circumstances). If quitting accomplishes the same thing as sabotage or internal lobbying, then by all means.
- bb88 9y ago
  In 2018, the following statements are true: 1. If you're on an active battlefield, both sides have already agreed that killing people is morally justified. 2. If the enemy is shooting at you, he is not interested in the ethics of killing someone on a battlefield. 3. If ethics helped wars end, there would be a US Army 27th Ethics Brigade to parachute in, but there's not and probably never will be. If on a battlefield, given a choice between holding scrupulous ideals and living, most people would choose living.

sillysaurus3 9y ago

The reason Rome was so influential was its military. Ditto for the US. Are you sure it's a good idea to become less influential on the world stage?

microcolonel 9y ago
> The reason Rome was so influential was its military. Ditto for the US. I think people often have that twisted. A lot of the U.S.'s expansion is cultural, not military. That is to say: people liked what they were doing, and became a fan. Think: music, movies, and to some extent, literature, and now you might add software and consumer electronics to that list. And even when looking at Rome, can you imagine how much harder those repeated annexations would have been if the Roman culture wasn't respected? I'd say that being capable at least of defending oneself, independently, from most individual powers and believable coalitions, creates a certain amount of stability.
- maroonblazer 9y ago
  Who can project force like the U.S. military? No one. For the most part it's used to enable people to have more say in how they're governed, modulo certain constraints. That's not to say we do it perfectly but we shouldn't underestimate how positively influential that military power is.
  - pteredactyl 9y ago
    Agreed. And it's a position that I don't think many people realize.
  - smsm42 9y ago
    In USSR, before it collapsed, young people dreamt of having jeans (not everybody could get or afford them), listened to Western music and watched Western movies. Not because US army has secretly occupied USSR, but because Western culture was more attractive to them. Not everything is done with the military.
    - 9y ago

Abishek_Muthian 9y ago

>"One man was responsible for helping save tens of thousands of lives. Another man helped kill millions." In the case of Fritz Haber, it's the same man who saved billions from hunger and killed millions by prolonging World War-I. Excellent article on the same - https://medium.com/the-mission/the-tragedy-of-fritz-haber-the-monster-who-fed-the-world-ec19a9834f74 https://medium.com/the-mission/the-tragedy-of-fritz-haber-th... Sometimes, the decision isn't clear as black & white; especially when you mix nationalism & patriotism into it.

sundvor 9y ago
That was a thoroughly depressing read, but a most excellent one as such. Thanks.
deft 9y ago
That's totally different. He chose to kill the people and did it intentionally out of hate / pride for his country. This thread and the posted link are discussing whether something is immoral or not. He knew it would kill a ton of people and designed it to on purpose. That situation is totally black and white. Mass intentional killing is wrong, pretty much everyone agrees with that.
- peteretep 9y ago
  > Mass intentional killing is wrong, pretty much everyone agrees with that Nope. Bomber Command in WWII and the use of nuclear weapons in WWII are both examples where you will find nuanced, intelligent debate on both sides of the argument of whether "mass, intentional killing" being wrong or not. What you describe as "hate / pride for his country" ... but isn't that the reason most soldiers kill? That you even bother to call this out shows that you know the reasons for killing people need to be taken into account. I'm not making a statement on whether or not mass killing is wrong here, simply pointing out what seems obvious: it's far from black and white.
  - SeriousM 9y ago
    "intelligent debate on both sides of the argument of whether "mass, intentional killing" being wrong or not" I'm sorry, but this sentence contains already the problem.

cmontella 9y ago

As a roboticist at the beginning of my career working on drones, I decided then and there that I would never make "bombs", a metaphor I used to mean anything that could be weaponized. I realized a lot of the work I was doing was funded by DARPA, and I was very cognizant about my research being used in this way. And like Dr. Kearns suggests, it's not entirely black and white. Would my path planning algorithm be used to more efficiently deliver scientific payloads to the atmosphere, or would it be used to route missiles to maximize casualties? Hard to really say, but I've avoided overtly military applications (even things like BigDog, designed to carry equipment for troops). Sometimes the distinction is even more insidious. I did work on perpetual flight for drones, and Facebook had a perpetual flight project that had the goal of bringing internet access to remote locations in Africa. Sounds humanitarian, but I also didn't want to be responsible for subjecting poor Africans to what I consider Facebook's panopticon. Or maybe it would have been a net boon for the region? It's really hard to tell a priori, so the best I have managed for myself is just to try and stay in theory, where developments are further removed from direct consequence.

dweekly 9y ago
As an engineer, founder, and investor I'll note that there's an ethical mandate for investors as well; when I started drone.vc I committed as a Quaker to not investing in "offensive" technologies. Separately, RE: bringing internet to new places, there's reasonable evidence to show that introducing broadband lifts GDP[1], so whatever you may think of Facebook, giving people high-speed connectivity to the whole Internet is probably a net positive. Disclaimer: I helped start Facebook's Connectivity Lab but no longer work there. [1] http://pubdocs.worldbank.org/en/391452529895999/WDR16-BP-Exploring-the-Relationship-between-Broadband-and-Economic-Growth-Minges.pdf http://pubdocs.worldbank.org/en/391452529895999/WDR16-BP-Exp...
- cbHXBY1D 9y ago
  Most people on this site have not trusted FB's intentions with the drone project for good reason. Ever since we collectively realized a three years ago that FB's internet.org was a wolf in sheep's clothing, most people on here have been distrusting of FB.

unclebucknasty 9y ago

>incredibly valid for Silicon Valley engineers these days If one could engineer a solution to save lives, but instead uses his/her talents to build the nth iteration of, say, a food delivery app, does he/she bear any ethical culpability? Because it seems the bigger challenge for engineers is not avoiding overt evil, but in the opportunity cost of spending their time on work of significantly lesser consequence than their talents might allow.

matuszeg 9y ago
So youre saying we need another messaging app?
lsc 9y ago
I think you're taking the idea too far. look at what the civil engineers have: https://www.asce.org/code-of-ethics/ https://www.asce.org/code-of-ethics/ Yes, yes, we have a responsibility to not defraud people, and to not work on obviously evil projects, and even to speak out if your employer is, say, building a bridge that is not up to the clearly defined, established standards. (which is super different from when you think a bridge will collapse, but can't explain why in a way that other people can understand. This parenthetical part is particularly important, I think.) But we are free humans, and if we want to waste our lives on trivial things, so long as those trivial things don't hurt others, that's our choice. We are humans, and humans need entertainment, we need play.
- unclebucknasty 9y ago
  >look at what the civil engineers have Well, that's a narrow view of ethical guidelines in conducting one's work. We all generally agree that people shouldn't be evil. My question sits above that. Is it enough for a moral society to simply decline to do evil? Or do we have a responsibility beyond that? And if it's the latter, then whose responsibility is it?
  - lsc 9y ago
    Declining to do evil, with real standards for what that means is, at the very least, a good first step, and a possible one.

didibus 9y ago

Why would the engineers be more burdened with the ethics of their creation over the investors, founders, managements, lawyers, execs, CEOs and government policy makers who all contributed to, promoted and made happen its fabrication by engineers?

cmyr 9y ago
They aren't more burdened, but they are the last line of defence.
confounded 9y ago
> investors, founders, managements, lawyers, execs, CEOs and government policy makers Power, corruption and lies. Those people often stand to personally (financially) benefit from unethical business decisions. Engineers have to think through all the dirty consequences, and still draw pretty much the same salary.
aangjie 9y ago
Ah.. the causality.. the argument can be made that without engineers the creation would never happen, (even if others are still trying to make money without ethics..).
solutron 9y ago
For the scope of this audience, I don't think it's necessarily a question of who's more responsible than the other. You're right, all the parties you've mentioned are just as culpable for their role in wrong-doing, even if it's abstracted or removed from implementation. As engineers though, we're in the unique position of being the gate-keeper between ideas, "visions" if you will (in the parlance of our times), and fruition. We're a gatekeeper between reality and the delusional musings afforded to people who aren't the engineers. We can say, "Fuck no, that should literally not exist or be built in any way", and do it loudly to stop the other parties from realizing their abusive vision. Or under dire circumstances, we can sabotage systems that need to be broken if it's too late to stop them from being created.

MichaelMoser123 9y ago

just a reminder: you are writing this over the internet - that one was funded by the pentagon too (darpanet) http://ccr.sigcomm.org/archive/1995/jan95/ccr-9501-clark.pdf http://ccr.sigcomm.org/archive/1995/jan95/ccr-9501-clark.pdf Here they say that the design of the internet was very much determined by the military priorities of the project (Telcos had different priorities so they came up with OSI where the link layer is supposed to be reliable, as opposed to DARPANET/IP, where the link layer/IP routing is dumb and the endpoint is smart; this allowed link layer to be pluggable and IP routers to be cheap)

betolink 9y ago
There is a huge difference between creating a network protocol and designing autonomous killer drones.
- mrlala 9y ago
  Really? The killer drones couldn't communicate without said network protocols.
  - ByThyGrace 9y ago
    The point being that "killer drones" are by sole purpose immoral. You can hardly find a humanitarian use for them, while network protocols are pretty much amoral. You can use them for bad, but also for good. Please excuse my manichaeism.
    - TangoTrotFox 9y ago
      One could say the same of atomic weapons, yet it's likely that nuclear weapons have saved more lives that we can even imagine. The impact of war was accelerating in raw numbers, and increasing in relative numbers, all the way up to WW2. 3-4% of the world's population was killed in WW2. In today's terms that would be ~270 million people. Think about that number for a minute. Let's put it in scales of 9/11s. During 9/11 2,996 people were killed. Our scaled WW2 had 90,120 times more deaths. That number is so absurdly large it's difficult to grasp. So put another way that's a 9/11 scale event every single day for 246 years. The United States, as a nation, was founded 242 years to give that number some scale. Just think about that! If each generation has children at 30 years old then another way of seeing that number would be a 9/11 event each and every day from today until 2 years after your great great great great great great great grandchild is born. But then came the nuke. Nuclear weapons make traditional war between developed and nuclear capable nations basically impossible. Think about what the Cold War really was, or even what the geopolitical 'disagreements' of today are. Those are World Wars 3 and 4, averted because these wars would be unwinnable by any side. Like Einstein said, 'I'm not sure what World War 3 will be fought with, but the 4th will be fought with sticks and stones!' Did the people developing nukes understand that they would finally create a weapon that immensely powerful that it would deter open unrestrained war for decades to come? It's possible, but I think that their motivation was something more straight forward - gain military superiority in the present, probably also mixed with a bit of scientific curiosity about the challenge of creating a nuclear bomb. It's difficult to predict the future. The most awful and aweful weapon created in the history of mankind ended up creating the most unprecedented period of peace for mankind as well. This is also why I vehemently oppose nuclear disarmament. That's how you get WW3 if people actually disarm, though in reality it'd simply likely result in nations obfuscating their nuclear weapon programs and facilities. It's just a molten salt thorium reactor guys, come on - breeders are awesome!

Demiurge 9y ago

The engineers in both cases are not essential to the outcomes. Engineers or craftsmen, or any other workers execute tasks put forth to them by society that gave them birth and education. Any person in the society has some influence, but clearly more influence is the hands of the leaders and organizers. But, there is power in organizing, or unionizing, for collective bargaining.

bobthepanda 9y ago
That defense didn’t work at Nuremberg.
- Demiurge 9y ago
  It's not a defense. It doesn't absolve anyone of personal responsibility.
cmontella 9y ago
This reminds me of the movie "Cube", which is about a sadistic torture chamber built by accident through a network of government sub-contractors who each built some innocuous part of the thing, which was then assembled through layers of opaque bureaucracy. It's a terrible film, but I always thought that premise was hilarious. https://en.wikipedia.org/wiki/Cube_(film) https://en.wikipedia.org/wiki/Cube_(film)

fsloth 9y ago

I don't think you can ever blame the engineer who designs the weapon, unless it is obviously ethically problematic like, a better iron maiden (as an example, I know it's probably a hoax device). Merely offering a force projecting capability is not evil. If you look at world history, the only concrete definition of who has most political power is this: it's the party with the most capability for projecting violence in a given area. Small close knit groups that live outside of large populations can live independently and do without military force. Once you have a large population with more or less fixed hearth and home the party with most violence projection capability owns them politically. This dynamic is played in almost ridiculously minuscule scale in the peloponnesian wars of the greek city states of antiquity. The pathological manifestation of this principle can be viewed, say, in the rise of ISIS and the Somali warlords. While a modern state is seldom a benevolent actor, at least in the western countries it's the best of known alternatives. In this framework violence is a key tool of the state, just like good governance, tax collection, etc. Given that violence is a necessity, in my opinion designing military instruments in itself is not evil. It's not as good as designing new vaccines, so there is some ethical scale in the matter, but I'm damned if I can put it in concrete terms.

numismatex 9y ago

You think it's a matter for engineers, I think it's a matter for executives and controlling interests. If you think that if the engineers refuse to do it that it won't get done, or that if Google doesn't do it, someone else won't, you're nuts.

grangerize 9y ago

I do not think that this is a simple black and white separation,but rather a dialectical one. I used to find myself most of the times confused while using technologies whose CEOs, mission I do not agree with. It felt as if I was contradicting myself. But then I realized the dialectical nature of this relationship which is if you want to create something better, improve something it is only natural that you will do so using the existing tools available in your world. A great example is Facebook. There are things you might not agree with, you might hate it but if you look closer you will see volunteer groups to political campaigns to marches and protests which are organized only via Facebook. One of such political activities one day could as well change Facebook or the world we live in drastically. You can apply this to other cases. You might have an anti-capitalist philosophy podcast hosted on Amazon or Spotify or Apple which in essence contradicts the very existence of such companies. But in the end you are using them to deliver your message, cause change etc. I am curious about what others think about this and how they feel about looking at it from a dialectical point of view as I am continuously thinking about this and still forming my thoughts.

RcouF1uZ4gsC 9y ago

From the Wikipedia article on Robert Kearns https://en.m.wikipedia.org/wiki/Robert_Kearns https://en.m.wikipedia.org/wiki/Robert_Kearns >Kearns was a member of the Office of Strategic Services, the forerunner of the U.S. CIA, during World War II.[1][2][3] Somehow I don't think that Kearns would have had an issue with wriing software to analyze video footage for the military.

philwelch 9y ago

To me, the salient point of that analogy is that the engineer designed execution chambers for Nazis. The engineers who designed the Sherman tank, the Garand rifle, or the Spitfire fighter plane also built weapons, weapons that killed people, but the judgment of history is that those people needed to be killed and thank God for the engineers who made that possible. So, there are two aspects of that: first, do you trust the people you're designing a weapon for to use it for justifiable purposes, and second, how does the weapon itself influence whether it can be used for just or unjust reasons? Gas chambers, for example, can only be used to kill someone whom you are already able to coerce into entering it; its only use is to murder a helpless prisoner. A Spitfire or Hurricane can shoot down bombers but not necessarily do that much damage on the ground. Orwell had an interesting perspective on this question: "Thus, for example, tanks, battleships and bombing planes are inherently tyrannical weapons, while rifles, muskets, long-bows, and hand-grenades are inherently democratic weapons. A complex weapon makes the strong stronger, while a simple weapon — so long as there is no answer to it — gives claws to the weak." I really don't have answers. The stance of, "weapons equals bad, therefore it's wrong to ever help design weapons" is a pacifist notion, and maybe that appeals to you, but if you're not a pacifist, it becomes a complicated question with certain political ramifications.

qkls 9y ago
The judgement of history is always that those people needed to be killed, because winners write the history.
- philwelch 9y ago
  Not always, but I figured WWII would be a relatively uncontroversial example.

paulddraper 9y ago

> And it won't be as easy as deciding between a heart valve and a gas chamber That's the truth. Did the engineers of the atomic bomb save millions of lives and usher in an era of unprecedented peace? Or did they slaughter millions of innocents and sow the seeds of humanity's destruction? It's been 70 years; we've had plenty of time to make up our minds. Many inventions are not so easily classified as innately good or evil.