9 ms·

I find this new trend of trying to predict how technology will be used - and presuming its users will either be evil or too stupid to use it responsibly incredi

I find this new trend of trying to predict how technology will be used - and presuming its users will either be evil or too stupid to use it responsibly incredibly patronizing. Engineers are not philosophers and should not be placed in this role. We do not have the tools to do it. As with any human we have a duty to consider first order effects - but even the most astute philosopher will have trouble anticipating second order effects. This burden should not be placed on those least qualified to perform it. Edit: Thank you to everyone for your thoughtful replies. I find this topic fascinating to discuss.

glitchc 8y ago
Then don't call yourself an engineer. Engineers build bridges, airplanes and when those fail, engineers are investigated for culpability. If the failure is due to a design flaw or cut corners, engineers are held responsible. I say this as a computer scientist with an undergrad in eng and a PE.
- slededit 8y ago
  Those are first order effects connected directly to the immediate aim of allowing transport over an obstacle. A civil engineer does not consider "who" will cross or what their aims will be. For example a bridge may allow an enemy army to move their tanks faster during wartime. Similarly tech shouldn't have to consider whether someone with unrelated goals will use their tech for unwanted aims.
  - glitchc 8y ago
    If you know your software will increase emissions to illegal levels, and still write it, you are culpable. If you write software for autonomous cars that is not foolproof at detecting pedestrians, you are culpable. In aviation, catastrophic failure means at least one death and the system should have an overall probability of failure at 10^-9. Critical software is frequently written at one or two orders below. That means 95%, 99%, 99.9%, are not enough.
    - Milner08 8y ago
      Do you understand the difference between a first order effect and a second order effect?
      - dekhn 8y ago
        I don't (btw, I'm a software engineer and scientist with extensive training). So I looked it up. Did you mean first order or second order kind of like the way physics people mean it (direct linear relationship with strong effect on the unknown variable). Or the way that MBAs mean it (they have their own definition). Or did you mean structural engineering, same as physics: https://www.quora.com/In-structural-engineering-what-are-first-order-effects https://www.quora.com/In-structural-engineering-what-are-fir... I think what you probably mean is a colloquial interpretation, where first order means something is directly causal in an obvious to see way, where second order is more of an indirect effect that occurs through complex dynamics. Either way, I don't think those terms are well defined, and they're kind of complex enough that using them to argue for morality, and calling people out for not knowing your definitions, makes you look caddish.
        jonnycomputer 8y ago
        I agree. I doubt that they could mean causally indirect. I think it must mean something like: not used in accordance to their the engineer's intent, especially when that intent covers a a general affordance (e.g. affords communication over long distances), but is being judged for a more specific affordance (affords communication over long distances to order a terror attack). I dunno. I'd have to be a philosopher to work it out.
        learc83 8y ago
        I think that's spot on. When people say something is a first/ second/nth, order effect they are really just arbitrarily drawing discrete lines in a continuous chain of causality. The problem of where to draw those lines is so difficult that legal systems resort to constructing tests based on whether a "reasonable person" could have foreseen the specific consequence of an act.
    - 8y ago
  - onceKnowable 8y ago
    Yeah but that’s a different argument to what you’re replying to. If that civil engineer sold or allowed (for any reason from negligence, insufficient security protocols, malice, ignorance etc) blueprints for his design to end up in enemy hands, he most certainly would be held liable. The thrust of that point is the very idea that software engineers on projects involving user data, regardless of how and what they think they are building, are not simply building “bridges”. They are building systems that have the potential to be misused as mass-surveillance apparatuses. And they need to build their systems accordingly, with appropriate data-security and with acknowledgement that their work could leave them personally liable to action should their work be found to be used as a threat-vector towards their users. The current attitude of “well people know what they’re signing up for and since they signed the User Agreement, we don’t have any liability here” towards users in the industry is sick.
    - smoony 8y ago
      I literally signed up for HN, something I've been uninterested in doing, just because of this comment. Saying that engineers need to make responsible decisions in their software design is one thing. But you imply that misuse is a liability for the designer? So if someone takes an engineer's code and modifies it for malicious intent the engineer is liable? That's absurd. That's like saying the person who designed the airplanes used in 9/11 are partially liable for 9/11.
      - onceKnowable 8y ago
        Absolutely, if that data is stolen due to the designers’ neglect in the security aspects of the systems they signed off as “adequately designed”.
        smoony 8y ago
        Yup! It just concerns me since this topic has a lot of great discussion but many people are missing the distinction between different types of effects. Sorry if I misunderstood what you were saying! For others who are confused what I mean... Consider: The person who makes a car that someone uses to kill someone by running a bystander over. The engineer isn't responsible. The person who messed up the Prius with the issues where the gas pedal or whatever didn't work. The engineer IS responsible. The first example is analogous to a software engineer writing image recognition software for a safety app. Then someone maliciously taking the software and using it to discriminate via misuse. The engineer is NOT responsible. If someone takes your software (could even be open source) and applies it or modifies it in a malicious way they are responsible not the engineer. The second is example is where if an engineer makes a widget app and has a vulnerability but signs off on it, and then widget purchaser credit card info is stolen. The engineer is responsible for negligence and should be liable for the misuse.
- candiodari 8y ago
  > engineers are investigated for culpability Funny how you believe that. Remember the Miami bridge collapse ? Workers on the ground misplaced one of the supports, and then didn't stop work when loud noises started coming out of the concrete. This was not documented wrong on the design, it was those same companies first mismeasuring the location of the supports, then simply moving them around when they didn't fit. Who's culpable ? The designers, of course. Or take Dieselgate. European, German and French car manufacturers systematically and purposefully installed software in the cars to commit fraud on the testers (the government testers knew that this was happening). Who's culpable ? A single engineer, of course. Not the regulators that knew what was happening. Not the European investment bank, who had seen reports on this. Not Volkswagen management, that had built several facilities explicitly for committing this fraud. Let's stop pretending the problem with engineering will be solved by assigning blame to engineers. It won't. It can't. Large scale fraud, committed by management and government is the problem. It cannot be fixed with engineering.
goalieca 8y ago
What!? Professional engineers do consider the greater good of society in their code of ethics. Introductory courses in ethics are a mandatory part of the curriculum where I’m from.
- stlHusker 8y ago
  And yet that is not even close to the hand-wringing being discussed in the article: "In my first ever AI class, we learned about how a system had been developed to automate something that had previously been a person’s job. Everyone said, “Isn’t this amazing?” — but I was concerned about who did these jobs. It stuck with me that no one else’s ears perked up at the significant downside to this very cool invention." "Then, the gatekeepers are the press, and it’s up to them to ask what the negative impacts of the technology are." Building an entire society of victims and allowing the hysterical press, who are in the business of feeding a hysterical public sounds like a formula for paralysis.
- ghein 8y ago
  I'm appalled at the lack of ethics in civil and mechanical engineering! Automotive engineers willfully destroyed opportunity for stablehands, whip manufacturers, and train conductors out of hundreds of other occupations annihilated. Civil engineers with their oriented strand board trusses, reinforced concrete, and dimensional lumber stole the livelihoods of masons, carpenters, foresters and so many more. Engineers have centuries of unethical practice to atone for. They need to stop their work because everything that they do is harmful and unethical!
vanderZwan 8y ago
> I find this new trend of trying to predict how technology will be used - and presuming its users will either be evil or too stupid to use it responsibly incredibly patronizing. People who create something, who research something, should by definition have a better grasp on their topic than everyone else. To expect that they are more knowledgeable, and therefore better equipped to see what it means, and therefore also have a civic duty to explain this to less well-informed, not not patronizing. It's acknowledging their expertise. > Engineers are not philosophers and should not be placed in this role. This is an interview in Nature. Academics with the ambition to publish in academic journals, be they prestigious like Nature or Science, or smaller and more niche like <insert your favourite journal here>, are taking on a role of a philosopher. And sure, engineers may "just" apply technology by comparison, but honestly I find that a very dismissive attitude as well. They are the "praxis" to academia's "theory", meaning that what they do actually has consequences in our lives. So if anything, their works comes with even more of a moral obligation to "Do The Right Thing" than that of academics.
- slededit 8y ago
  > People who create something, who research something, should by definition have a better grasp on their topic than everyone else. To expect that they are more knowledgeable, and therefore better equipped to see what it means, and therefore also have a civic duty to explain this to less well-informed, not not patronizing. It's acknowledging their expertise. They are experts in how the technology works. That doesn't make them an expert in how other humans will take advantage of what they have discovered/created. I've taken the charity of assuming the scientist or engineer themselves had non-evil aims. > This is an interview in Nature. Academics with the ambition to publish in academic journals, be they prestigious like Nature or Science, or smaller and more niche like <insert your favourite journal here>, are taking on a role of a philosopher. Yes, but a philosopher on which topic? A doctorate from an engineering background is not equally as educated as someone with degrees in history or other studies of human nature in regards to the societal effects of technology.
  - s-shellfish 8y ago
    Doctorate of philosophy. The Ph in PhD actually means something. It's not some kitschy decoration.
ccccccccccccc 8y ago
Computer Science is a new field and individual developers can have reaching effects on society, direct or indirect, that extend further than any other engineering discipline. My belief is that it 100% is on the responsibility of researchers and developers to think of the ethical and social implications of their work. As an example, a podcast I listened to recently was interviewing CS researchers working on "deep fake" videos. When asked if they thought their work could have negative societal impact, their response was along the lines of "its just really cool to make this". CS as a whole has a lot of maturing to do and I hope we can get their sooner rather than later.
s-shellfish 8y ago
There is a big difference between computer science and computer engineering. Science is about understanding. Engineering is about using that understanding for building.
- slededit 8y ago
  Nobody would be discussing the moral implications of scientific discoveries if it didn't ultimately lead to machines that cause effect in the real world. Science and Engineering are inextricably linked in this discussion.
  - s-shellfish 8y ago
    > Science and Engineering are inextricably linked in this discussion. Of course they are, but this article is very specifically about the role of the scientist. Not the engineer. Will negative disclosure cause problems for engineers? I don't know. What I do know, is it isn't fair to place that burden of knowledge entirely on scientists while tying their hands behind their back with respect to open discussion in a professional setting. Academic publishing is intended to be open without fear of professional and personal repercussion for increasing factual awareness. Get rid of that and, well, sit back, relax, and I hope we all can enjoy the apocalypse of knowledge, because that's where stuff is headed without the ability to professionally discuss existing problems. > Nobody would be discussing the moral implications of scientific discoveries if it didn't ultimately lead to machines that cause effect in the real world. That's just factually wrong. Mathematics. Philosophers. Logicians. All the living and dead theorists who built the abstract foundations essential for the functioning of our machines. People who deal entirely in the world of the abstract. You don't develop a passion for abstraction because you want something out of reality. You develop a passion for abstraction because reality is sometimes terrible.
    - slededit 8y ago
      > That's just factually wrong. Mathematics. Philosophers. Logicians. All the living and dead theorists who built the abstract foundations essential for the functioning of our machines. People who deal entirely in the world of the abstract. You don't develop a passion for abstraction because you want something out of reality. You develop a passion for abstraction because reality is sometimes terrible. We don't have moral discussions about the discovery of a new law of physics or a mathematical lemma. This only becomes an issue when we can see how it can be applied to some aim. Thus if there were no engineers nobody would consider the scientists moral obligations with the exception of their experiments themselves causing harm. The argument to limiting science is that engineers might take it and create some terrible machine with it. Perhaps the most notorious example being a nuclear bomb. I believe we agree on the role of science, I don't think we agree on what their culpability should be.
jonnycomputer 8y ago
I guess a little humility is warranted. But I think your attitude is really about letting engineers shirk their responsibility as moral agents. You know, sliiiiide by, as they've always done. Take responsibility.
- slededit 8y ago
  > But I think your attitude is really about letting engineers shirk their responsibility as moral agents. I absolutely disagree. That is why I specifically mentioned culpability for first order effects. If you build a gun for someone you know to be violent you have culpability for its use. What I disagree with is the broadening of this "liability" to include second order effects. That we are now supposed to predict chaotic systems and know in advance all perverse outcomes - and that we should have moral liability should we fail in this task.
  - jonnycomputer 8y ago
    we have moral liability for second order effects (whatever those are)
    - slededit 8y ago
      On what basis?
      - jonnycomputer 8y ago
        on what basis are we culpable for anything? if you can answer that I question I'll try to answer yours. not trying to be flippant (we've had a good discussion so far), but I don't really have an answer, except to say that I don't know on what basis we might distinguish the two cases. if you can answer me what makes someone morally culpable of anything, maybe I'd have a clue.
        slededit 8y ago
        The basis for my arguments are from English law. Who is guilty of what has been fine tuned over centuries. It's not enough that something bad happened. There must be either intent to harm or gross negligence. The modern trend both in law and society of statutory offenses and ex post facto "something bad happened so let's see what we can charge him with" is something I find deeply disturbing. I find the original articles proposals as yet another step upon that road. Our forefathers accepted the universe was unpredictable. We seem to feel we have enough grasp now that if bad things happen someone must be to blame.
everdev 8y ago
That's why diverse teams are important. You're right, we can't expect a CS major to understand social impact, but if you're working on AI or weapons out some other high-risk project there should be someone in the team who does.
ashelmire 8y ago
>Engineers are not philosophers Hold your horses, some of us engineers have philosophy degrees! In all seriousness, engineers are more than capable of considering the impact of their creations. But can you predict future uses for technology with accuracy? Of course not. I'm not sure how relevant it is for most work. People will always find ways to use technology to the detriment of humanity, but the overall effect of technology is beneficial. The internet is obviously for the benefit of humanity, yet there are plenty of people using it for evil.
- dekhn 8y ago
  If a person says "Engineers are not philosophers", the common interpretation is not "All engineers are not philosophers", it's "Nearly all engineers are not philosophers". The small set of engineers with philosophy degrees does not negate the parent's statement.
  - ashelmire 8y ago
    Did you just pick a formal logic debate with a philosopher? I disagree with your interpretation of the language used. The comment was authoritative in tone generally, and this line said (in my estimation) that the argument was "if someone is an engineer, they are not a philosopher". Even worse, there's a suggestion that this philosophical work shouldn't burden those "least qualified to perform it". Leaving the philosophy to "better men" is how we justify evils committed by underlings and evokes thoughts of the Nuremberg trials. Every person should come to terms with the moral implications of their actions. In any case, one doesn't need a philosophy degree to do philosophy, and many of the great engineers and scientists of history had a great deal to say with regard to the moral implications of their work. Even today, engineers at many of the big tech companies are objecting to some work their companies are doing. Many others choose not to work for companies whose mission they disagree with.
    - dekhn 8y ago
      No formal logic debate here; I was explaining colloquial use.
dredmorbius 8y ago
The story goes that Thamus said many things to Theuth in praise or blame of the various arts, which it would take too long to repeat; but when they came to the letters, [274e] “This invention, O king,” said Theuth, “will make the Egyptians wiser and will improve their memories; for it is an elixir of memory and wisdom that I have discovered.” But Thamus replied, “Most ingenious Theuth, one man has the ability to beget arts, but the ability to judge of their usefulness or harmfulness to their users belongs to another; [275a] and now you, who are the father of letters, have been led by your affection to ascribe to them a power the opposite of that which they really possess. Socrates, From Plato's dialogue Phaedrus 14, 274c-275b http://www.historyofinformation.com/expanded.php?id=3894 http://www.historyofinformation.com/expanded.php?id=3894
filo_ 8y ago
Totally agree. And this is not new at all. All you have to do is read the history of engineers and scientists who worked on nukes or rocketry. The ones that tried failed miserably. Understanding why is crucial in understanding the limitations of domain experts.
eli_gottlieb 8y ago
I have a slightly different take: engineers shouldn't be blamed for negative outcomes that propagate down from the political and cultural levels. For instance, machine learning is now widely used for surveillance and advertising in ways that violate basic presumptions about privacy rights. Thing is, I imagine that if, say, ML researchers had gone on strike over privacy issues somehow, society would have essentially told them to shut up and get back to work. Sure, the same larger society now wants to blame ML for the constriction of privacy and for extensive state and private surveillance networks, but it was that very society which cheered the construction of these exact surveillance networks "becuz terrorizm" and for the sake of "free speech" (for billionaire multinational corporations) and for other awful reasons. The people who were skeptical or critical over this whole social trend knew damn well it was all a bad idea, and we said so. Society deserves to get what it voted for, with both its ballot box and its dollars, and on some level, it deserves to get it good and hard without scapegoating the hired help, when it was them in the first place who told us skeptical, critical engineers and scientists to shut up and produce what society wanted from us.
- stlHusker 8y ago
  I agree but have a bit different take... "For instance, machine learning is now widely used for surveillance and advertising in ways that violate basic presumptions about privacy rights." Those same mathematical techniques are also being used to more accurately detect cancer in MRI scans. Saying everyone needs to be held responsible for second order effects is unrealistic and ultimately damaging to society. Blaming the toolmaker for things they cannot control will only cause no tools to be made. To me it points to a larger issue in our society that we are on a trend of completely removing context. Context no longer matters, what matters are the effects, regardless of whether you can reasonably be held accountable for them or not. It is thought-crime induced by those who in the same breath rail against that totalitarian behavior and then skip to implementing a version of it themselves. A number of people on this thread immediately jump to the argument on surveillance and data privacy which is a useful discussion but tangential to what is being proposed in the original article. The article is moral hand-wringing with zero clarity or practicality. And the judge of who is culpable or not is left to a hysterical press, driven by a hysterical and sometimes uninformed public. Look at what this behavior has done to political (and personal) discourse. Now imagine it being applied to science. No thank you.
627467 8y ago
There are more knowledge creators/workers than ever before. I don't think that expecting more information from any given study (especially if it is applied research) is detrimental to society or the study itself. It's about raising the standards because we can. Standard publication sections: intro, material and methods, results, discussion, conclusion. It feels like you can add a "speculative impact" section somewhere between discussion and conclusion.
thedancollins 8y ago
I have to agree. I think that calling it "patronizing" is harsh but science generally does not progress on a timeline that depends on and adapts to humans - it is the other way around. And whereas history may judge the oil industry negatively it also judges theocratic regimes negatively for trying to "mitigate" progress.
thatjsguy 8y ago
> Engineers are not philosophers and should not be placed in this role. Which is how we wind up with companies hell-bent on vacuuming up as much data as possible and selling it to anyone who asks. Perhaps if some software developers (or, at this point, some legislators) stopped to think about the ethical considerations of the tech industry’s actions, we all wouldn’t have to have conversations about how other countries can buy elections on the same platform where you share pictures of your food.
- Nasrudith 8y ago
  You're kidding yourself if you think that electioneering and lobbying are a new thing. Look at Saudi role in promoting Wahhabism, religious fundimentalists worsening AIDs epidemics and promoting homophobia by pushing bullshit on the third world for one alone let alone actual CIA or KGB influence. Before people were falling for literal fake news they were falling for Satanic Ritual Panic because the truth wasn't what they were concerned about but how it made them feel. Tech was never the issue here - people were. The drive to scapegoat the upstarts should be regarded with great suspicion. While there may be questionable ethics involved with companies it looks to be a classic attempt to scapegoat the troublesome for matters that are peripheral at best and any action they could have taken would have dissatisfied people like how both the left and right are convinced Facebook is biased against them because they don't reflect the world as they wish to see it.
hodgesrm 8y ago
> Engineers are not philosophers and should not be placed in this role. We do not have the tools to do it. As with any human we have a duty to consider first order effects - but even the most astute philosopher will have trouble anticipating second order effects. This burden should not be placed on those least qualified to perform it. Here are two rebuttals to the above statement. 1. At the level of projects it's asking engineers to think a bit more broadly about solutions. For instance, when designing a user comment service for social media you can trade-off between designs that encourage informed debate vs. ranting flame wars. In many cases both designs have similar implementation costs and meet business requirements, so why not pick the one that is more socially responsible? As these choices become better understood they should just be part of user experience design. 2.) To develop the trade-offs we should address more research to social consequences of technology and take the results seriously. Great engineers are philosophers or at least more broad-minded about the interactions between technology and society than some of the current leaders in Silicon Valley. Andy Grove, David Hewlett, Bill Packard, and Bill Gates were/are not just problem solvers but examples of engineers who have thought about how to benefit society as a whole.
vegetablepotpie 8y ago
I disagree because part of the job of engineering is stake holder analysis, where engineers consider all the groups that will be affected by a system before developing requirements of the system. This is one of the things that differentiates engineers from technicians. For CS to be more than a field of coders they should consider the larger impact of their work.

glitchc 8y ago

Then don't call yourself an engineer. Engineers build bridges, airplanes and when those fail, engineers are investigated for culpability. If the failure is due to a design flaw or cut corners, engineers are held responsible. I say this as a computer scientist with an undergrad in eng and a PE.

slededit 8y ago
Those are first order effects connected directly to the immediate aim of allowing transport over an obstacle. A civil engineer does not consider "who" will cross or what their aims will be. For example a bridge may allow an enemy army to move their tanks faster during wartime. Similarly tech shouldn't have to consider whether someone with unrelated goals will use their tech for unwanted aims.
- glitchc 8y ago
  If you know your software will increase emissions to illegal levels, and still write it, you are culpable. If you write software for autonomous cars that is not foolproof at detecting pedestrians, you are culpable. In aviation, catastrophic failure means at least one death and the system should have an overall probability of failure at 10^-9. Critical software is frequently written at one or two orders below. That means 95%, 99%, 99.9%, are not enough.
  - Milner08 8y ago
    Do you understand the difference between a first order effect and a second order effect?
    - dekhn 8y ago
      I don't (btw, I'm a software engineer and scientist with extensive training). So I looked it up. Did you mean first order or second order kind of like the way physics people mean it (direct linear relationship with strong effect on the unknown variable). Or the way that MBAs mean it (they have their own definition). Or did you mean structural engineering, same as physics: https://www.quora.com/In-structural-engineering-what-are-first-order-effects https://www.quora.com/In-structural-engineering-what-are-fir... I think what you probably mean is a colloquial interpretation, where first order means something is directly causal in an obvious to see way, where second order is more of an indirect effect that occurs through complex dynamics. Either way, I don't think those terms are well defined, and they're kind of complex enough that using them to argue for morality, and calling people out for not knowing your definitions, makes you look caddish.
      - jonnycomputer 8y ago
        I agree. I doubt that they could mean causally indirect. I think it must mean something like: not used in accordance to their the engineer's intent, especially when that intent covers a a general affordance (e.g. affords communication over long distances), but is being judged for a more specific affordance (affords communication over long distances to order a terror attack). I dunno. I'd have to be a philosopher to work it out.
      - learc83 8y ago
        I think that's spot on. When people say something is a first/ second/nth, order effect they are really just arbitrarily drawing discrete lines in a continuous chain of causality. The problem of where to draw those lines is so difficult that legal systems resort to constructing tests based on whether a "reasonable person" could have foreseen the specific consequence of an act.
  - 8y ago
- onceKnowable 8y ago
  Yeah but that’s a different argument to what you’re replying to. If that civil engineer sold or allowed (for any reason from negligence, insufficient security protocols, malice, ignorance etc) blueprints for his design to end up in enemy hands, he most certainly would be held liable. The thrust of that point is the very idea that software engineers on projects involving user data, regardless of how and what they think they are building, are not simply building “bridges”. They are building systems that have the potential to be misused as mass-surveillance apparatuses. And they need to build their systems accordingly, with appropriate data-security and with acknowledgement that their work could leave them personally liable to action should their work be found to be used as a threat-vector towards their users. The current attitude of “well people know what they’re signing up for and since they signed the User Agreement, we don’t have any liability here” towards users in the industry is sick.
  - smoony 8y ago
    I literally signed up for HN, something I've been uninterested in doing, just because of this comment. Saying that engineers need to make responsible decisions in their software design is one thing. But you imply that misuse is a liability for the designer? So if someone takes an engineer's code and modifies it for malicious intent the engineer is liable? That's absurd. That's like saying the person who designed the airplanes used in 9/11 are partially liable for 9/11.
    - onceKnowable 8y ago
      Absolutely, if that data is stolen due to the designers’ neglect in the security aspects of the systems they signed off as “adequately designed”.
      - smoony 8y ago
        Yup! It just concerns me since this topic has a lot of great discussion but many people are missing the distinction between different types of effects. Sorry if I misunderstood what you were saying! For others who are confused what I mean... Consider: The person who makes a car that someone uses to kill someone by running a bystander over. The engineer isn't responsible. The person who messed up the Prius with the issues where the gas pedal or whatever didn't work. The engineer IS responsible. The first example is analogous to a software engineer writing image recognition software for a safety app. Then someone maliciously taking the software and using it to discriminate via misuse. The engineer is NOT responsible. If someone takes your software (could even be open source) and applies it or modifies it in a malicious way they are responsible not the engineer. The second is example is where if an engineer makes a widget app and has a vulnerability but signs off on it, and then widget purchaser credit card info is stolen. The engineer is responsible for negligence and should be liable for the misuse.
candiodari 8y ago
> engineers are investigated for culpability Funny how you believe that. Remember the Miami bridge collapse ? Workers on the ground misplaced one of the supports, and then didn't stop work when loud noises started coming out of the concrete. This was not documented wrong on the design, it was those same companies first mismeasuring the location of the supports, then simply moving them around when they didn't fit. Who's culpable ? The designers, of course. Or take Dieselgate. European, German and French car manufacturers systematically and purposefully installed software in the cars to commit fraud on the testers (the government testers knew that this was happening). Who's culpable ? A single engineer, of course. Not the regulators that knew what was happening. Not the European investment bank, who had seen reports on this. Not Volkswagen management, that had built several facilities explicitly for committing this fraud. Let's stop pretending the problem with engineering will be solved by assigning blame to engineers. It won't. It can't. Large scale fraud, committed by management and government is the problem. It cannot be fixed with engineering.

goalieca 8y ago

What!? Professional engineers do consider the greater good of society in their code of ethics. Introductory courses in ethics are a mandatory part of the curriculum where I’m from.

stlHusker 8y ago
And yet that is not even close to the hand-wringing being discussed in the article: "In my first ever AI class, we learned about how a system had been developed to automate something that had previously been a person’s job. Everyone said, “Isn’t this amazing?” — but I was concerned about who did these jobs. It stuck with me that no one else’s ears perked up at the significant downside to this very cool invention." "Then, the gatekeepers are the press, and it’s up to them to ask what the negative impacts of the technology are." Building an entire society of victims and allowing the hysterical press, who are in the business of feeding a hysterical public sounds like a formula for paralysis.
ghein 8y ago
I'm appalled at the lack of ethics in civil and mechanical engineering! Automotive engineers willfully destroyed opportunity for stablehands, whip manufacturers, and train conductors out of hundreds of other occupations annihilated. Civil engineers with their oriented strand board trusses, reinforced concrete, and dimensional lumber stole the livelihoods of masons, carpenters, foresters and so many more. Engineers have centuries of unethical practice to atone for. They need to stop their work because everything that they do is harmful and unethical!

vanderZwan 8y ago

> I find this new trend of trying to predict how technology will be used - and presuming its users will either be evil or too stupid to use it responsibly incredibly patronizing. People who create something, who research something, should by definition have a better grasp on their topic than everyone else. To expect that they are more knowledgeable, and therefore better equipped to see what it means, and therefore also have a civic duty to explain this to less well-informed, not not patronizing. It's acknowledging their expertise. > Engineers are not philosophers and should not be placed in this role. This is an interview in Nature. Academics with the ambition to publish in academic journals, be they prestigious like Nature or Science, or smaller and more niche like <insert your favourite journal here>, are taking on a role of a philosopher. And sure, engineers may "just" apply technology by comparison, but honestly I find that a very dismissive attitude as well. They are the "praxis" to academia's "theory", meaning that what they do actually has consequences in our lives. So if anything, their works comes with even more of a moral obligation to "Do The Right Thing" than that of academics.

slededit 8y ago
> People who create something, who research something, should by definition have a better grasp on their topic than everyone else. To expect that they are more knowledgeable, and therefore better equipped to see what it means, and therefore also have a civic duty to explain this to less well-informed, not not patronizing. It's acknowledging their expertise. They are experts in how the technology works. That doesn't make them an expert in how other humans will take advantage of what they have discovered/created. I've taken the charity of assuming the scientist or engineer themselves had non-evil aims. > This is an interview in Nature. Academics with the ambition to publish in academic journals, be they prestigious like Nature or Science, or smaller and more niche like <insert your favourite journal here>, are taking on a role of a philosopher. Yes, but a philosopher on which topic? A doctorate from an engineering background is not equally as educated as someone with degrees in history or other studies of human nature in regards to the societal effects of technology.
- s-shellfish 8y ago
  Doctorate of philosophy. The Ph in PhD actually means something. It's not some kitschy decoration.

ccccccccccccc 8y ago

Computer Science is a new field and individual developers can have reaching effects on society, direct or indirect, that extend further than any other engineering discipline. My belief is that it 100% is on the responsibility of researchers and developers to think of the ethical and social implications of their work. As an example, a podcast I listened to recently was interviewing CS researchers working on "deep fake" videos. When asked if they thought their work could have negative societal impact, their response was along the lines of "its just really cool to make this". CS as a whole has a lot of maturing to do and I hope we can get their sooner rather than later.

s-shellfish 8y ago

There is a big difference between computer science and computer engineering. Science is about understanding. Engineering is about using that understanding for building.

slededit 8y ago
Nobody would be discussing the moral implications of scientific discoveries if it didn't ultimately lead to machines that cause effect in the real world. Science and Engineering are inextricably linked in this discussion.
- s-shellfish 8y ago
  > Science and Engineering are inextricably linked in this discussion. Of course they are, but this article is very specifically about the role of the scientist. Not the engineer. Will negative disclosure cause problems for engineers? I don't know. What I do know, is it isn't fair to place that burden of knowledge entirely on scientists while tying their hands behind their back with respect to open discussion in a professional setting. Academic publishing is intended to be open without fear of professional and personal repercussion for increasing factual awareness. Get rid of that and, well, sit back, relax, and I hope we all can enjoy the apocalypse of knowledge, because that's where stuff is headed without the ability to professionally discuss existing problems. > Nobody would be discussing the moral implications of scientific discoveries if it didn't ultimately lead to machines that cause effect in the real world. That's just factually wrong. Mathematics. Philosophers. Logicians. All the living and dead theorists who built the abstract foundations essential for the functioning of our machines. People who deal entirely in the world of the abstract. You don't develop a passion for abstraction because you want something out of reality. You develop a passion for abstraction because reality is sometimes terrible.
  - slededit 8y ago
    > That's just factually wrong. Mathematics. Philosophers. Logicians. All the living and dead theorists who built the abstract foundations essential for the functioning of our machines. People who deal entirely in the world of the abstract. You don't develop a passion for abstraction because you want something out of reality. You develop a passion for abstraction because reality is sometimes terrible. We don't have moral discussions about the discovery of a new law of physics or a mathematical lemma. This only becomes an issue when we can see how it can be applied to some aim. Thus if there were no engineers nobody would consider the scientists moral obligations with the exception of their experiments themselves causing harm. The argument to limiting science is that engineers might take it and create some terrible machine with it. Perhaps the most notorious example being a nuclear bomb. I believe we agree on the role of science, I don't think we agree on what their culpability should be.

jonnycomputer 8y ago

I guess a little humility is warranted. But I think your attitude is really about letting engineers shirk their responsibility as moral agents. You know, sliiiiide by, as they've always done. Take responsibility.

slededit 8y ago
> But I think your attitude is really about letting engineers shirk their responsibility as moral agents. I absolutely disagree. That is why I specifically mentioned culpability for first order effects. If you build a gun for someone you know to be violent you have culpability for its use. What I disagree with is the broadening of this "liability" to include second order effects. That we are now supposed to predict chaotic systems and know in advance all perverse outcomes - and that we should have moral liability should we fail in this task.
- jonnycomputer 8y ago
  we have moral liability for second order effects (whatever those are)
  - slededit 8y ago
    On what basis?
    - jonnycomputer 8y ago
      on what basis are we culpable for anything? if you can answer that I question I'll try to answer yours. not trying to be flippant (we've had a good discussion so far), but I don't really have an answer, except to say that I don't know on what basis we might distinguish the two cases. if you can answer me what makes someone morally culpable of anything, maybe I'd have a clue.
      - slededit 8y ago
        The basis for my arguments are from English law. Who is guilty of what has been fine tuned over centuries. It's not enough that something bad happened. There must be either intent to harm or gross negligence. The modern trend both in law and society of statutory offenses and ex post facto "something bad happened so let's see what we can charge him with" is something I find deeply disturbing. I find the original articles proposals as yet another step upon that road. Our forefathers accepted the universe was unpredictable. We seem to feel we have enough grasp now that if bad things happen someone must be to blame.

everdev 8y ago

That's why diverse teams are important. You're right, we can't expect a CS major to understand social impact, but if you're working on AI or weapons out some other high-risk project there should be someone in the team who does.

ashelmire 8y ago

>Engineers are not philosophers Hold your horses, some of us engineers have philosophy degrees! In all seriousness, engineers are more than capable of considering the impact of their creations. But can you predict future uses for technology with accuracy? Of course not. I'm not sure how relevant it is for most work. People will always find ways to use technology to the detriment of humanity, but the overall effect of technology is beneficial. The internet is obviously for the benefit of humanity, yet there are plenty of people using it for evil.

dekhn 8y ago
If a person says "Engineers are not philosophers", the common interpretation is not "All engineers are not philosophers", it's "Nearly all engineers are not philosophers". The small set of engineers with philosophy degrees does not negate the parent's statement.
- ashelmire 8y ago
  Did you just pick a formal logic debate with a philosopher? I disagree with your interpretation of the language used. The comment was authoritative in tone generally, and this line said (in my estimation) that the argument was "if someone is an engineer, they are not a philosopher". Even worse, there's a suggestion that this philosophical work shouldn't burden those "least qualified to perform it". Leaving the philosophy to "better men" is how we justify evils committed by underlings and evokes thoughts of the Nuremberg trials. Every person should come to terms with the moral implications of their actions. In any case, one doesn't need a philosophy degree to do philosophy, and many of the great engineers and scientists of history had a great deal to say with regard to the moral implications of their work. Even today, engineers at many of the big tech companies are objecting to some work their companies are doing. Many others choose not to work for companies whose mission they disagree with.
  - dekhn 8y ago
    No formal logic debate here; I was explaining colloquial use.

dredmorbius 8y ago

The story goes that Thamus said many things to Theuth in praise or blame of the various arts, which it would take too long to repeat; but when they came to the letters, [274e] “This invention, O king,” said Theuth, “will make the Egyptians wiser and will improve their memories; for it is an elixir of memory and wisdom that I have discovered.” But Thamus replied, “Most ingenious Theuth, one man has the ability to beget arts, but the ability to judge of their usefulness or harmfulness to their users belongs to another; [275a] and now you, who are the father of letters, have been led by your affection to ascribe to them a power the opposite of that which they really possess. Socrates, From Plato's dialogue Phaedrus 14, 274c-275b http://www.historyofinformation.com/expanded.php?id=3894 http://www.historyofinformation.com/expanded.php?id=3894

filo_ 8y ago

Totally agree. And this is not new at all. All you have to do is read the history of engineers and scientists who worked on nukes or rocketry. The ones that tried failed miserably. Understanding why is crucial in understanding the limitations of domain experts.

eli_gottlieb 8y ago

I have a slightly different take: engineers shouldn't be blamed for negative outcomes that propagate down from the political and cultural levels. For instance, machine learning is now widely used for surveillance and advertising in ways that violate basic presumptions about privacy rights. Thing is, I imagine that if, say, ML researchers had gone on strike over privacy issues somehow, society would have essentially told them to shut up and get back to work. Sure, the same larger society now wants to blame ML for the constriction of privacy and for extensive state and private surveillance networks, but it was that very society which cheered the construction of these exact surveillance networks "becuz terrorizm" and for the sake of "free speech" (for billionaire multinational corporations) and for other awful reasons. The people who were skeptical or critical over this whole social trend knew damn well it was all a bad idea, and we said so. Society deserves to get what it voted for, with both its ballot box and its dollars, and on some level, it deserves to get it good and hard without scapegoating the hired help, when it was them in the first place who told us skeptical, critical engineers and scientists to shut up and produce what society wanted from us.

stlHusker 8y ago
I agree but have a bit different take... "For instance, machine learning is now widely used for surveillance and advertising in ways that violate basic presumptions about privacy rights." Those same mathematical techniques are also being used to more accurately detect cancer in MRI scans. Saying everyone needs to be held responsible for second order effects is unrealistic and ultimately damaging to society. Blaming the toolmaker for things they cannot control will only cause no tools to be made. To me it points to a larger issue in our society that we are on a trend of completely removing context. Context no longer matters, what matters are the effects, regardless of whether you can reasonably be held accountable for them or not. It is thought-crime induced by those who in the same breath rail against that totalitarian behavior and then skip to implementing a version of it themselves. A number of people on this thread immediately jump to the argument on surveillance and data privacy which is a useful discussion but tangential to what is being proposed in the original article. The article is moral hand-wringing with zero clarity or practicality. And the judge of who is culpable or not is left to a hysterical press, driven by a hysterical and sometimes uninformed public. Look at what this behavior has done to political (and personal) discourse. Now imagine it being applied to science. No thank you.

627467 8y ago

There are more knowledge creators/workers than ever before. I don't think that expecting more information from any given study (especially if it is applied research) is detrimental to society or the study itself. It's about raising the standards because we can. Standard publication sections: intro, material and methods, results, discussion, conclusion. It feels like you can add a "speculative impact" section somewhere between discussion and conclusion.

thedancollins 8y ago

I have to agree. I think that calling it "patronizing" is harsh but science generally does not progress on a timeline that depends on and adapts to humans - it is the other way around. And whereas history may judge the oil industry negatively it also judges theocratic regimes negatively for trying to "mitigate" progress.

thatjsguy 8y ago

> Engineers are not philosophers and should not be placed in this role. Which is how we wind up with companies hell-bent on vacuuming up as much data as possible and selling it to anyone who asks. Perhaps if some software developers (or, at this point, some legislators) stopped to think about the ethical considerations of the tech industry’s actions, we all wouldn’t have to have conversations about how other countries can buy elections on the same platform where you share pictures of your food.

Nasrudith 8y ago
You're kidding yourself if you think that electioneering and lobbying are a new thing. Look at Saudi role in promoting Wahhabism, religious fundimentalists worsening AIDs epidemics and promoting homophobia by pushing bullshit on the third world for one alone let alone actual CIA or KGB influence. Before people were falling for literal fake news they were falling for Satanic Ritual Panic because the truth wasn't what they were concerned about but how it made them feel. Tech was never the issue here - people were. The drive to scapegoat the upstarts should be regarded with great suspicion. While there may be questionable ethics involved with companies it looks to be a classic attempt to scapegoat the troublesome for matters that are peripheral at best and any action they could have taken would have dissatisfied people like how both the left and right are convinced Facebook is biased against them because they don't reflect the world as they wish to see it.

hodgesrm 8y ago

> Engineers are not philosophers and should not be placed in this role. We do not have the tools to do it. As with any human we have a duty to consider first order effects - but even the most astute philosopher will have trouble anticipating second order effects. This burden should not be placed on those least qualified to perform it. Here are two rebuttals to the above statement. 1. At the level of projects it's asking engineers to think a bit more broadly about solutions. For instance, when designing a user comment service for social media you can trade-off between designs that encourage informed debate vs. ranting flame wars. In many cases both designs have similar implementation costs and meet business requirements, so why not pick the one that is more socially responsible? As these choices become better understood they should just be part of user experience design. 2.) To develop the trade-offs we should address more research to social consequences of technology and take the results seriously. Great engineers are philosophers or at least more broad-minded about the interactions between technology and society than some of the current leaders in Silicon Valley. Andy Grove, David Hewlett, Bill Packard, and Bill Gates were/are not just problem solvers but examples of engineers who have thought about how to benefit society as a whole.

vegetablepotpie 8y ago

I disagree because part of the job of engineering is stake holder analysis, where engineers consider all the groups that will be affected by a system before developing requirements of the system. This is one of the things that differentiates engineers from technicians. For CS to be more than a field of coders they should consider the larger impact of their work.