7 ms·

I get that they're trying to encourage ethical behavior, but this proposal seems incredibly arrogant. It implicitly states that computer programmers are, as a

I get that they're trying to encourage ethical behavior, but this proposal seems incredibly arrogant. It implicitly states that computer programmers are, as a field, competent to predict how a technology will impact society. Whether or not ANY field is competent to do that is questionable, but whether or not computer programmers are able to do that is not questionable; it's clearly incorrect. We don't have the background, training, skillset, or in some cases even personality to be making insightful predictions about how society will respond to and be impacted by a new technology. I'm not saying the person advocating it is a bad person, but this particular proposal is arrogant (and founded on an incorrect understanding of what kind of thing programmers are good at).

eli_gottlieb 8y ago
Isn't society supposed to democratically control the impacts of various things upon itself?
- TeMPOraL 8y ago
  If by "democratically" you mean through the market, then yes, it's how this works. EDIT: what I mean is, roughly, markets are how we usually screw ourselves up in the first place.
  - dekhn 8y ago
    No, they mean through legislation.
  - onceKnowable 8y ago
    Bullshit, the market doesn’t demand your right to privacy. Nor does the market care about your property right. The democracy you live in enshrines those rights on your behalf.
    - mindcrime 8y ago
      The democracy you live in enshrines those rights on your behalf. And it's doing a great job of it.[1][2][3][4][5][6][7][8] [1]:https://en.wikipedia.org/wiki/NSA_warrantless_surveillance_(2001%E2%80%932007) https://en.wikipedia.org/wiki/NSA_warrantless_surveillance_(... [2]: https://en.wikipedia.org/wiki/Room_641A https://en.wikipedia.org/wiki/Room_641A [3]: https://en.wikipedia.org/wiki/National_security_letter https://en.wikipedia.org/wiki/National_security_letter [4]: https://en.wikipedia.org/wiki/COINTELPRO https://en.wikipedia.org/wiki/COINTELPRO [5]: https://en.wikipedia.org/wiki/Project_MKUltra https://en.wikipedia.org/wiki/Project_MKUltra [6]: https://en.wikipedia.org/wiki/J._Edgar_Hoover https://en.wikipedia.org/wiki/J._Edgar_Hoover [7]: https://www.eff.org/deeplinks/2014/05/how-nsa-transforming-law-enforcement https://www.eff.org/deeplinks/2014/05/how-nsa-transforming-l... [8]: https://en.wikipedia.org/wiki/Parallel_construction https://en.wikipedia.org/wiki/Parallel_construction
      - stevew20 8y ago
        Beautiful timing.
      - onceKnowable 8y ago
        None of those actions were the result of an informed public debate. In fact, they’re all actions that show that an informed public debate regarding new technologies is needed to protect the public from not only unscrupulous businesses or external threats but to protect the public from their own authorities. Without public debate and a public that demands robust protection in the fit of laws, misused technologies represent a threat to everyone until that debate is had.
- 2close4comfort 8y ago
  Not when the market has no moral compass with which to guide. Democracy is controlled by the market, and when you teach a computer to learn to pick the cars out of a picture so you can "prove you are not a bot" watch Real Genius you do have a moral obligation to what you take part in creating. If you say my piece is small and I can't stop them from making a facial recognition system/plate reader/password cracker then you are putting your financial gain above what your work does to others. You are part of the problem.
  - eli_gottlieb 8y ago
    > Democracy is controlled by the market This is the problem, because you know what else is controlled by the market? Your employment. I fully encourage going on strike if asked to make surveillance tech, but you will be fired for it. Then some other firm will be hired, and, oh well. If "the market" controls democracy, to the extent that the "democratic" government demands surveillance tech because someone on the market wants to sell it, that's the problem to be solved, not a shortage of research scientists martyring their careers for no effective gain against the larger social pathologies.
    - 2close4comfort 8y ago
      YES! Totally agree but they can't continue to give the market the weapons that are being used to maintain that repression.
tremon 8y ago
They're not talking about computer programmers though (I agree they don't have the ability). The more interesting question is whether Computer Science (the academic discipline) should focus more on the societal impact of computing instead of solely on the technical implementation of computing (i.e. applied mathematics).
- bartread 8y ago
  > They're not talking about computer programmers though (I agree they don't have the ability). This pigeon-holing was way beyond tiresome, not to mention grossly inaccurate, 20 years ago: please STOP IT. Computer programmers aren't in general some special class of person incapable of learning new skills and behaviours. Quite the opposite for the most part.
  - tremon 8y ago
    I'm not sure I understand why you call this pigeon-holing. The GP explained that the professional field of programming has very little overlap with the professional field of social dynamics, and I was simply agreeing with that observation. "Most programmers don't know how society will evolve" appears to me a similar observation as "most pilots don't know how to build a plane". I don't really see why that's an objectionable statement.
    - crankylinuxuser 8y ago
      This is, at its heart, a question of ethics in engineering disciplines. Software engineering is a combination of applied mathematics, data science, and significant engineering practices (do X with Y constraints). In Engineering proper, we have ethics classes. They entail in how things can fail and cause item, structural, and/or injury up to death. We have all read about the Therac-25. That was a software and hardware design choice that lead to the death of many. Computer science seems to skirt the ethics discussions just by being too new of a field. We (royal) aren't looking at the failure modes of having social media. Nor are we considering the ramifications of the code we write. Intelligence is the ability to write the code. Wisdom is the ability to understand its ramifications.
robertk 8y ago
Get off your high horse man. Disclosing a worry or concern is not arrogance even if you do not wield the expertise to evaluate said concern. It is another input that may or may not be beneficial to humanity to assimilate, but let the reader be the judge. It is impossible to assess the risk of unknown unknowns, so rather than assailing someone with adjectives and persecuting them with the crime of “arrogance” and “hubris,” just judge less and listen. If they’re wrong, move on. Look at the mirror before you call another one arrogant.
- PunchTornado 8y ago
  > If they’re wrong, move on why? aren't we allowed to judge the wrong? nothing wrong in calling someone arrogant if he's arrogant.
  - robertk 8y ago
    Except complete hypocrisy I suppose, which it falls upon the judge to assert as vice or virtue. Better than invective is a genuine and sincere mutual examination of the facts — actual content rather than mindless reputational slander of an entire field of study. http://www.paulgraham.com/disagree.html http://www.paulgraham.com/disagree.html
  - deleted 8y ago
    [deleted]
  - asd9873 8y ago
    Their post is more ad hominem cynicism than reasoned critique.
- dang 8y ago
  Please don't put personal barbs in your HN comments, regardless of how arrogant some other comment seems. Your comment would be much better without its first and last sentences. The inverse of a shit sandwich!
rb808 8y ago
Ah yes we've invented this iPhone thingie, we predict it will cause people to get addicted to cat memes, sending dick picks and the Russians to meddle with our election.
- cubano 8y ago
  Russians have been meddling with our elections for a long, long time.
Bromskloss 8y ago
Even with perfect prediction, who is to decide, for everyone, what is good and what is bad?
- DoofusOfDeath 8y ago
  Excellent point. Someone else brought up the culpability of fission researchers for Hiroshima. That's one example of a major technical endeavor where even with historical hindsight people won't all agree on its ethics.
  - onceKnowable 8y ago
    Regardless of the ethics of the initial developers of the individual fission researchers, they were experts in their narrow field. They could never be held responsible for not realizing that MAD was the only outcome that their work would result in. The development of the MAD determination was made by the rest of the general public once the full facts of what those initial fission researchers had developed came to public knowledge. The article is proposing that that public debate happen during or before development so that laws can be made to robustly protect the public from a new technology in the event that it’s misused.
    - DoofusOfDeath 8y ago
      Yeah, I was just trying to add a reason that the OP's proposal wasn't workable.
Nasrudith 8y ago
I think it is even worse in its arrogance but not of the programmer but of the would be "ethical reformers" - it thinks the ethics of /theory/ can be determined beforehand, that it can be halted and following these judgements would lead to a positive outcome by holding the whole world back when others may stumble upon the same train of thought later. The fact that there is relatively little resistance between theory and practice does not change that. Under that framework USSR's suppression of genetics in favor of Lysenkoist bullshit would be justified because an incomplete understanding of genetics could lead to eugenics travesties. It is like King Cnut trying to stop the tides only totally unironically and unsarcastically.
daveFNbuck 8y ago
I don't think you would have to believe that we're competent at predicting there impact on society to support this. There main benefit would probably just be getting people to consider the potential negative impacts of their work.
ggggtez 8y ago
Though, to be fair, even if they aren't good at predicting societal impact, even asking the question may be enough to identify low-hanging fruit, even without expertise in sociology.
- dllthomas 8y ago
  Further, as we get practiced at asking the question, maybe we'll get better at it.
asd9873 8y ago
I think you inferred a whole lot that isn't there on some pretty trite grounds. It plainly advocates for a peer-review process. It's not advocating for the researched themselves to become a hardcore utilitarian over their work. And you're focused on "programmers". This is about computer science research. You're speaking more to nerds that hide in their basement, not academics figuring out more refined AI algorithms, which is mostly math. You're being too stubborn, for my tastes anyway.
4684499 8y ago
> It implicitly states that computer programmers are, as a field, competent to predict how a technology will impact society. Not always though, there are plenty of negative examples already, just don't make the society fall for the same mistakes.
oconnore 8y ago
Perfect is the enemy of good.
onceKnowable 8y ago
It’s not that hard. Take an example: your team has developed technology that can identify, with five nines accuracy, the yearly income of a user. That’s innovative, for sure. And nobody is taking the achievement away from your team. But you don’t need to be Socrates to realize that what you’ve created could be used in horrifying ways. The article is asking creators to take a step back, ignore the excitement of creation for one moment, and consider if what they’re doing should be done. And if the answer is Yes, then decide if a public debate is required to protect the public from that new technology’s misuse. Now the first reaction from many will be to scoff “fuck that, I’m just building an app here, I’ve got an investment to recoup and money to make”. That’s valid, but it’s an unethical way of looking at the act of creation. Feel free to hide behind the “I’m just doing my job” excuse, but we all know where that ended up. That is the crux here, politicians can only make laws to protect the public at large from misuse of your newly created technology if they know about it. And the public at large can only demand robust protection (in the form of laws made by their politicians) from misuse of your technology if the public at large knows about it. Without a public debate, new technology could be misunderstood by politicians such that even if they are aware of the technology and even if they legislate, they may not make laws that robustly protect the public. Thus, from an ethical standpoint, a public debate is needed so that robust laws are demanded and provided in order to protect the public at large from new technologies.
- overeater 8y ago
  The question is, who becomes the final arbiter? And if that decision is "we should hide this knowledge", how is that enacted? What are the consequences for the researchers? Can you name any specific examples of computer science researcher that you think clearly should be censored (in your example, should the software or algorithm to do accurate prediction become "illegal")? And in those cases, what should be the penalty for publishing the work? I think if we start to delve into the actual implementation of this, and look at real examples, it will be clear that this idea of "research that can be used for bad should be not published" leads to a bad type of society.
  - onceKnowable 8y ago
jsonne 8y ago
If you go a bit deeper in the paper it suggests engaging social scientists to predict them as well as computer science folks.
inimino 8y ago
It's right there in the answer to the very first interview question. > The idea is not to try to predict the future, but, on the basis of the literature, to identify the expected side effects or unintended uses of this technology. Besides being (ironically) arrogant, your dismissal sounds to me like you want to disavow any responsibility computer programmers have for what the technology they develop is used for. > founded on an incorrect understanding of what kind of thing programmers are good at In my personal opinion, if you're not always thinking of the applications and effects on society of your work, you're not a very fully-developed human being and certainly not the best programmer you could be, to say nothing of being a good citizen.
Cyclone_ 8y ago
I agree mostly with this, I would say it's difficult for someone of any profession to try and predict how a technology would be used.
- onceKnowable 8y ago
  That’s not what’s being asked. No one person, or team of people, can determine that. What the article is proposing is that creators of new technologies inform the public so that the general public can decide if new laws are required to prevent that new technology from being misused against the public at large. There’s precedents here and I’ve written a ton of comments about them if you want to read a bit about why such a proposition is the only ethical option.

eli_gottlieb 8y ago

Isn't society supposed to democratically control the impacts of various things upon itself?

TeMPOraL 8y ago
If by "democratically" you mean through the market, then yes, it's how this works. EDIT: what I mean is, roughly, markets are how we usually screw ourselves up in the first place.
- dekhn 8y ago
  No, they mean through legislation.
- onceKnowable 8y ago
  Bullshit, the market doesn’t demand your right to privacy. Nor does the market care about your property right. The democracy you live in enshrines those rights on your behalf.
  - mindcrime 8y ago
    The democracy you live in enshrines those rights on your behalf. And it's doing a great job of it.[1][2][3][4][5][6][7][8] [1]:https://en.wikipedia.org/wiki/NSA_warrantless_surveillance_(2001%E2%80%932007) https://en.wikipedia.org/wiki/NSA_warrantless_surveillance_(... [2]: https://en.wikipedia.org/wiki/Room_641A https://en.wikipedia.org/wiki/Room_641A [3]: https://en.wikipedia.org/wiki/National_security_letter https://en.wikipedia.org/wiki/National_security_letter [4]: https://en.wikipedia.org/wiki/COINTELPRO https://en.wikipedia.org/wiki/COINTELPRO [5]: https://en.wikipedia.org/wiki/Project_MKUltra https://en.wikipedia.org/wiki/Project_MKUltra [6]: https://en.wikipedia.org/wiki/J._Edgar_Hoover https://en.wikipedia.org/wiki/J._Edgar_Hoover [7]: https://www.eff.org/deeplinks/2014/05/how-nsa-transforming-law-enforcement https://www.eff.org/deeplinks/2014/05/how-nsa-transforming-l... [8]: https://en.wikipedia.org/wiki/Parallel_construction https://en.wikipedia.org/wiki/Parallel_construction
    - stevew20 8y ago
      Beautiful timing.
    - onceKnowable 8y ago
      None of those actions were the result of an informed public debate. In fact, they’re all actions that show that an informed public debate regarding new technologies is needed to protect the public from not only unscrupulous businesses or external threats but to protect the public from their own authorities. Without public debate and a public that demands robust protection in the fit of laws, misused technologies represent a threat to everyone until that debate is had.
2close4comfort 8y ago
Not when the market has no moral compass with which to guide. Democracy is controlled by the market, and when you teach a computer to learn to pick the cars out of a picture so you can "prove you are not a bot" watch Real Genius you do have a moral obligation to what you take part in creating. If you say my piece is small and I can't stop them from making a facial recognition system/plate reader/password cracker then you are putting your financial gain above what your work does to others. You are part of the problem.
- eli_gottlieb 8y ago
  > Democracy is controlled by the market This is the problem, because you know what else is controlled by the market? Your employment. I fully encourage going on strike if asked to make surveillance tech, but you will be fired for it. Then some other firm will be hired, and, oh well. If "the market" controls democracy, to the extent that the "democratic" government demands surveillance tech because someone on the market wants to sell it, that's the problem to be solved, not a shortage of research scientists martyring their careers for no effective gain against the larger social pathologies.
  - 2close4comfort 8y ago
    YES! Totally agree but they can't continue to give the market the weapons that are being used to maintain that repression.

tremon 8y ago

They're not talking about computer programmers though (I agree they don't have the ability). The more interesting question is whether Computer Science (the academic discipline) should focus more on the societal impact of computing instead of solely on the technical implementation of computing (i.e. applied mathematics).

bartread 8y ago
> They're not talking about computer programmers though (I agree they don't have the ability). This pigeon-holing was way beyond tiresome, not to mention grossly inaccurate, 20 years ago: please STOP IT. Computer programmers aren't in general some special class of person incapable of learning new skills and behaviours. Quite the opposite for the most part.
- tremon 8y ago
  I'm not sure I understand why you call this pigeon-holing. The GP explained that the professional field of programming has very little overlap with the professional field of social dynamics, and I was simply agreeing with that observation. "Most programmers don't know how society will evolve" appears to me a similar observation as "most pilots don't know how to build a plane". I don't really see why that's an objectionable statement.
  - crankylinuxuser 8y ago
    This is, at its heart, a question of ethics in engineering disciplines. Software engineering is a combination of applied mathematics, data science, and significant engineering practices (do X with Y constraints). In Engineering proper, we have ethics classes. They entail in how things can fail and cause item, structural, and/or injury up to death. We have all read about the Therac-25. That was a software and hardware design choice that lead to the death of many. Computer science seems to skirt the ethics discussions just by being too new of a field. We (royal) aren't looking at the failure modes of having social media. Nor are we considering the ramifications of the code we write. Intelligence is the ability to write the code. Wisdom is the ability to understand its ramifications.

robertk 8y ago

Get off your high horse man. Disclosing a worry or concern is not arrogance even if you do not wield the expertise to evaluate said concern. It is another input that may or may not be beneficial to humanity to assimilate, but let the reader be the judge. It is impossible to assess the risk of unknown unknowns, so rather than assailing someone with adjectives and persecuting them with the crime of “arrogance” and “hubris,” just judge less and listen. If they’re wrong, move on. Look at the mirror before you call another one arrogant.

PunchTornado 8y ago
> If they’re wrong, move on why? aren't we allowed to judge the wrong? nothing wrong in calling someone arrogant if he's arrogant.
- robertk 8y ago
  Except complete hypocrisy I suppose, which it falls upon the judge to assert as vice or virtue. Better than invective is a genuine and sincere mutual examination of the facts — actual content rather than mindless reputational slander of an entire field of study. http://www.paulgraham.com/disagree.html http://www.paulgraham.com/disagree.html
- deleted 8y ago
  [deleted]
- asd9873 8y ago
  Their post is more ad hominem cynicism than reasoned critique.
dang 8y ago
Please don't put personal barbs in your HN comments, regardless of how arrogant some other comment seems. Your comment would be much better without its first and last sentences. The inverse of a shit sandwich!

rb808 8y ago

Ah yes we've invented this iPhone thingie, we predict it will cause people to get addicted to cat memes, sending dick picks and the Russians to meddle with our election.

cubano 8y ago
Russians have been meddling with our elections for a long, long time.

Bromskloss 8y ago

Even with perfect prediction, who is to decide, for everyone, what is good and what is bad?

DoofusOfDeath 8y ago
Excellent point. Someone else brought up the culpability of fission researchers for Hiroshima. That's one example of a major technical endeavor where even with historical hindsight people won't all agree on its ethics.
- onceKnowable 8y ago
  Regardless of the ethics of the initial developers of the individual fission researchers, they were experts in their narrow field. They could never be held responsible for not realizing that MAD was the only outcome that their work would result in. The development of the MAD determination was made by the rest of the general public once the full facts of what those initial fission researchers had developed came to public knowledge. The article is proposing that that public debate happen during or before development so that laws can be made to robustly protect the public from a new technology in the event that it’s misused.
  - DoofusOfDeath 8y ago
    Yeah, I was just trying to add a reason that the OP's proposal wasn't workable.

Nasrudith 8y ago

I think it is even worse in its arrogance but not of the programmer but of the would be "ethical reformers" - it thinks the ethics of /theory/ can be determined beforehand, that it can be halted and following these judgements would lead to a positive outcome by holding the whole world back when others may stumble upon the same train of thought later. The fact that there is relatively little resistance between theory and practice does not change that. Under that framework USSR's suppression of genetics in favor of Lysenkoist bullshit would be justified because an incomplete understanding of genetics could lead to eugenics travesties. It is like King Cnut trying to stop the tides only totally unironically and unsarcastically.

daveFNbuck 8y ago

I don't think you would have to believe that we're competent at predicting there impact on society to support this. There main benefit would probably just be getting people to consider the potential negative impacts of their work.

ggggtez 8y ago

Though, to be fair, even if they aren't good at predicting societal impact, even asking the question may be enough to identify low-hanging fruit, even without expertise in sociology.

dllthomas 8y ago
Further, as we get practiced at asking the question, maybe we'll get better at it.

asd9873 8y ago

I think you inferred a whole lot that isn't there on some pretty trite grounds. It plainly advocates for a peer-review process. It's not advocating for the researched themselves to become a hardcore utilitarian over their work. And you're focused on "programmers". This is about computer science research. You're speaking more to nerds that hide in their basement, not academics figuring out more refined AI algorithms, which is mostly math. You're being too stubborn, for my tastes anyway.

4684499 8y ago

> It implicitly states that computer programmers are, as a field, competent to predict how a technology will impact society. Not always though, there are plenty of negative examples already, just don't make the society fall for the same mistakes.

oconnore 8y ago

Perfect is the enemy of good.

onceKnowable 8y ago

It’s not that hard. Take an example: your team has developed technology that can identify, with five nines accuracy, the yearly income of a user. That’s innovative, for sure. And nobody is taking the achievement away from your team. But you don’t need to be Socrates to realize that what you’ve created could be used in horrifying ways. The article is asking creators to take a step back, ignore the excitement of creation for one moment, and consider if what they’re doing should be done. And if the answer is Yes, then decide if a public debate is required to protect the public from that new technology’s misuse. Now the first reaction from many will be to scoff “fuck that, I’m just building an app here, I’ve got an investment to recoup and money to make”. That’s valid, but it’s an unethical way of looking at the act of creation. Feel free to hide behind the “I’m just doing my job” excuse, but we all know where that ended up. That is the crux here, politicians can only make laws to protect the public at large from misuse of your newly created technology if they know about it. And the public at large can only demand robust protection (in the form of laws made by their politicians) from misuse of your technology if the public at large knows about it. Without a public debate, new technology could be misunderstood by politicians such that even if they are aware of the technology and even if they legislate, they may not make laws that robustly protect the public. Thus, from an ethical standpoint, a public debate is needed so that robust laws are demanded and provided in order to protect the public at large from new technologies.

overeater 8y ago
The question is, who becomes the final arbiter? And if that decision is "we should hide this knowledge", how is that enacted? What are the consequences for the researchers? Can you name any specific examples of computer science researcher that you think clearly should be censored (in your example, should the software or algorithm to do accurate prediction become "illegal")? And in those cases, what should be the penalty for publishing the work? I think if we start to delve into the actual implementation of this, and look at real examples, it will be clear that this idea of "research that can be used for bad should be not published" leads to a bad type of society.
- onceKnowable 8y ago

jsonne 8y ago

If you go a bit deeper in the paper it suggests engaging social scientists to predict them as well as computer science folks.

inimino 8y ago

It's right there in the answer to the very first interview question. > The idea is not to try to predict the future, but, on the basis of the literature, to identify the expected side effects or unintended uses of this technology. Besides being (ironically) arrogant, your dismissal sounds to me like you want to disavow any responsibility computer programmers have for what the technology they develop is used for. > founded on an incorrect understanding of what kind of thing programmers are good at In my personal opinion, if you're not always thinking of the applications and effects on society of your work, you're not a very fully-developed human being and certainly not the best programmer you could be, to say nothing of being a good citizen.

Cyclone_ 8y ago

I agree mostly with this, I would say it's difficult for someone of any profession to try and predict how a technology would be used.

onceKnowable 8y ago
That’s not what’s being asked. No one person, or team of people, can determine that. What the article is proposing is that creators of new technologies inform the public so that the general public can decide if new laws are required to prevent that new technology from being misused against the public at large. There’s precedents here and I’ve written a ton of comments about them if you want to read a bit about why such a proposition is the only ethical option.