10 ms·

The Moral Character of Cryptographic Work

240 points by cscheid 11y ago

moyix 11y ago
This is an important message to consider, and not just for cryptography. Everyone can benefit from thinking about the moral and social consequences of what problems they choose to work on, who they do them for, and what values the institutions they contract with hold.
tptacek 11y ago
For anyone here who doesn't know who the author is, Phil Rogaway is one of the most important academic cryptographers; he's responsible for OAEP, PSS, OCB, UMAC, FPE, and the constructions behind XTS, the universal standard for disk encryption. The meat of the paper is in Sections 2 (where the unintended power dynamics of some modern academic crypto research projects is discussed) and 4 (where he provides suggestions for important practical projects academic cryptographers should tackle). Sections 1 and 3 are written for an audience of academics who might be less familiar with the political implications of crypto than the typical HN reader. Essentially, Rogaway is trying to convince mathematicians to embrace the practical and political impact of their work. Colin Percival gets a nice mention towards the end of the paper. I'd be over the moon if I were him. Congrats, Colin!
- nickpsecurity 11y ago
  It's a nice paper. Turns out, he grudgingly believes what I've been saying all along: "I am not optimistic. The gure of the heroic cryptographer sweeping in to save the world from totalitarian surveillance is ludicrous. And in a world where intelligence agencies stockpile and exploit countless vulnerabilities, obtain CA secret keys, subvert software-update mechanisms, infltrate private companies with moles, redirect online discussions in favored directions, and exert enormous in uence on standards bodies, cryptography alone will be an ineectual response. At best, cryptography might be a tool for creating possibilities within contours circumscribed by other forces." Exactly. Gotta deal with those "other forces," which are social and legal. Otherwise, the opponents vast resources work around the few that try to resist with the crypto. Doesn't stop many technical people from thinking crypto will magically save the day.
- ropman76 11y ago
  This is a very good paper and it's disappointing that it didn't get higher on HN. That being said I disagree with some of paper's points. I don't believe that cryptographers should take on moral responsibility for how they perceive their work may or may not be used. If a malware writer uses XTS to lock up a hard drive should Dr Rogeway be morally responsible for that since he helped create the constructions for XTS? I would argue "no" because there is a moral separation between idea and implementation. We should not burden cryptographers with moral baggage that should be placed on the people who implement or set policy to implement.
  - dvanduzer 11y ago
    There is a clear distinction between a moral responsibility and a legal liability. The moral duties of scientists versus policy makers have little to do with the moral separation between idea and implementation.
  - shkkmo 11y ago
    Are you arguing that the scientists who developed the first atomic bomb share no moral responsibility for the (positive or negative) effects of its use? Dr. Rogeway is morally responsible both for the use of his encryption for cyber ransom and for the use of his encryption to allow private democratic and economic discourse. If you think the negative effects of publicly available encryption outweigh the positive effects, it would be immoral to be working on making better encryption publicly available. Obviously, there is a complicated moral calculus behind which effects are foreseeable/likely relative to your other available actions and their foreseeable/likely effects.
AdmiralAsshat 11y ago
They lead one to ask if our inability to effectively address mass surveillance constitutes a failure of our field. I believe that it does. I call for a community-wide effort to develop more effective means to resist mass surveillance. What's funny is that you could talk to any number of law enforcement officials who believe that the moral failing is on the cryptography community for not providing a "backdoor" into encrypted communications. Or to restate, "Please, Apple, think of the children!" Morality is, unfortunately, subjective. Part of the argument is in convincing your opponent that your morality is superior to theirs. Or, perhaps, that their stance violates their own sense of morality.
- DennisP 11y ago
  Part three of the paper contains his justification for opposing mass surveillance.
- policestate101 11y ago
  When your opponent condones torture, indefinite imprisonment of innocents, war crimes, and genocide, there is little point in having anbopen dialog or argument with them.
  - hueving 11y ago
    There are many people that don't condone those things but still think law enforcement should be able to get into people's computers or read communications with a warrant. It's more nuanced than you would like to believe.
    - policestate101 11y ago
      Nauance is really serving all those blacks in jail for fabricated evidence. Nauance has done wonders for the people of Syria. Nauanc as in "disposition matrix" instead of targeted assassinations. Sometimes things are so nauanced hospitals filled with doctors without borders get bombed, so nuanced we may not even consider it a war crime! But do on about how much more nuanced the world really is, how incredibly morally bankrupt, what weak moral fiber to defend torture through the weasel words of nauace, the millions of dead surely are glad for such a reasonable and nauceced view you present. Again, debating people who condone torture and mass murder is a waste of time.
pcl 11y ago
From the abstract: Cryptography rearranges power: it configures who can do what, from what. I'd argue that the reverse is really the issue that needs more attention. Online systems that do not provide strong cryptography rearrange power, as compared to their offline equivalents. It was not feasible to scan all phone calls for keywords in 1970, since that required effort from humans to do the patching and listening. The power dynamic changed when our industry brought those calls into a centralized, trivially-storable clear-text format. Encrypting the conversations is simply a partial return to the status quo of a few decades ago.
- mangeletti 11y ago
  +100 What powerful tool is I incrementalism that the whole population has forgotten that we used to be private individuals. The unification of the human race by the Internet now threatens the relevance of overgrown governments and banks. That's why they fight us.
  - dasil003 11y ago
    I wouldn't say they do it because they feel threatened—how often do bureaucrats genuinely feel threatened? Most likely they are just legitimately trying to do their jobs, and the natural tendency will be to seek any power available that seems relevant to that task. Does any individual or organization ever voluntarily cede power?
- nathan_long 11y ago
  Good point. And even longer ago, all conversations required meeting, and were private unless an eavesdropper was physically nearby. Tech has created more extreme possibilities on both sides. On the privacy side, it's possible to exchange messages at a distance in an unreadable and nearly undetectable way. On the surveillance side, it's possible to eavesdrop on nearly everyone (except the very sophisticated). We can't and (don't want to) go back in time. The real question is "which danger is greater: conspiracy or oppression?"
  - PhaseMage 11y ago
sneak 11y ago
So I guess we have a moral imperative to fork Chrome to actually enforce cert pinning even against locally-installed roots, then?
- shkkmo 11y ago
  Enforce cert pinning? No. Notify when it is overridden by locally-installed roots? Yes.
- magicalist 11y ago
  ...as opposed to Firefox, Edge, and Safari which all do the same?
dang 11y ago
Posted a few days ago at https://news.ycombinator.com/item?id=10655418 https://news.ycombinator.com/item?id=10655418, but got so little discussion that we won't treat it as a dupe but have instead merged the threads.
- cscheid 11y ago
  Huh, thank you. I missed it back then, and assumed that when the submission went through it meant that the dupe detector okayed it.
NHQ 11y ago
OBVIOUSLY CHAOTIC NEUTRAL
PhaseMage 11y ago
I completely agree that Cryptography researchers should evaluate their work against their moral values. I feel the same thing about pretty much all engineering... I've been focusing on trying to design moral networking protocols.
pdkl95 11y ago
> This makes cryptography an inherently political tool. It always was. Many developers like to stay out of politics. Concentrating on difficult technical problems is hard enough; adding in politics is therefor adding in unnecessary complexity. As the wonderful Tom Lehrer put it in his song "Wernher Von Braun"[1], Don't say that he's hypocritical, Say rather that he's apolitical. "once the rockets are up, who cares where they come down? That's not my department," says Wernher von Braun. The problem with this is similar to the problem of abstaining from the vote: it's absolutely not a neutral position. Choosing to abstain from politics in general, like those that choose to abstain from the vote, is de facto a vote for the status quo and majority rule. Not only is cryptography an inherently political tool, almost all software is political. Software does not exist in a vacuum; the entire point of most software is that it has an impact on business, society, and the world. With the discovery of the General Purpose Computer, this impact can be very large. It's easy to see why cryptography disrupts existing power structures. It should be similarly easy to see how software already overturned the traditional power structures in places like the stock market, manufacturing, and retail. So please, consider what impact your software might have when you are writing it, or if someone already has a goal in mind. Maybe, in some cases, it's better to walk away. It;'s a hard question, but the answer is not to say "I'm staying out of politics". To quote Quinn Norton and Eleanor Saitta from their talk[2] at 30c3, there is "no neutral ground in a burning world". [1] https://www.youtube.com/watch?v=QEJ9HrZq7Ro#t=16 https://www.youtube.com/watch?v=QEJ9HrZq7Ro#t=16 [2] https://www.youtube.com/watch?v=DWg2qEEa9CE https://www.youtube.com/watch?v=DWg2qEEa9CE
bryanhm 11y ago
Here we have what I think is a display of an intelligent mind specialized in one area, funded by a state institution, and weak at resolving moral conflicts. Computer scientists (and any self-respecting scientists) HAVE to separate their ethics from the interests of state institutions. Phrases in the paper resembling something like "where the cryptographer has a duty to serve the public and keep their self-interest in check" indicate this. I've read a paper recently on designing systems to have security exceptions for law enforcement and calling them "exception requirements" or something to that effect. This is the sort of thing a good study of ethics can help to resolve.
- jvdh 11y ago
  Computer scientists (and any self-respecting scientists) HAVE to separate their ethics from the interests of state institutions. Why? The paper argues exactly to the opposite in the first part, describing the atomic-bomb scientists, and the Russell-Einstein Manifesto. In my opinion, cryptographers and computer scientists have ignored morally questioning their work for too long. Reality is now catching up on this, with techniques for surveillance. The paper argues this for cryptography, in my view this is even more generally applicable.
  - bryanhm 11y ago
    Why do scientists/cryptographers have to judge ethics for themselves? Not sure what you're asking. Bertrand Russell judged the bomb to be a danger for humanity and chose to express his own views on it.
    - jvdh 11y ago
      Medical ethics committees have ethicists, but the main advice is from medical experts. I would not want anybody else judging the ethics of medical experiments, as no one else has the expert knowledge. I see absolutely no reason why this should different for computer scientists and cryptographers. No one else understands the implications of an experiment or a new methodology. I was at a meeting recently discussing ethics committees for computer science. A medical expert gave his opinion and said (paraphrasing): I fail to see the problem with digitalisation, we have had medical records on paper for years, now they are on a computer, what is the difference? I don't mean to say that every scientist should do this individually. They should discuss this with colleagues, and with an ethics committee, which should contain subject matter experts, but also ethicists.
zby 11y ago
Cyber-security in general is political. This guy is a cryptographer - so it is natural that he formulated this for his own area - but it is too narrow. Hackers are now routinely the foot soldiers of the cyber-war of everyone against everyone - we need to think more about our own rules instead of following orders.
kluck 11y ago
It is the responsibility of the inventor to think ahead as far as possible about how an invention may do good and/or harm.
jroitgrund 11y ago
The biggest thing I took away from this was reading the slides and seeing the FBI's suicide letter to Martin Luther King. I didn't know about that, and reading that right after seeing the jury dury article on the front page today is chilling to say the least.
gcb0 11y ago
everyone has those talks 30yrs ago when crypto was labeled munition by the usa. if this article has any content that warrant discussion, is how out of touch with reality the social sciences are. ... and sadly, yet another proof of how necessary Snowden was.
jvvw 11y ago
This was one of the factors (certainly not the only one) that made me move out of the field of cryptography - I was doing work related to attempting to break a major cryptosystem and I realised that I wasn't completely sure what the right course of action was ethically in the slim chance that I succeeded. My background was in pure mathematics and up until I moved into cryptography, it seemed obvious that openess of information was an obvious good. However, once in the crypto field, it became much a much more ambiguous issue.

240 points by cscheid 11y ago

moyix 11y ago

This is an important message to consider, and not just for cryptography. Everyone can benefit from thinking about the moral and social consequences of what problems they choose to work on, who they do them for, and what values the institutions they contract with hold.

tptacek 11y ago

For anyone here who doesn't know who the author is, Phil Rogaway is one of the most important academic cryptographers; he's responsible for OAEP, PSS, OCB, UMAC, FPE, and the constructions behind XTS, the universal standard for disk encryption. The meat of the paper is in Sections 2 (where the unintended power dynamics of some modern academic crypto research projects is discussed) and 4 (where he provides suggestions for important practical projects academic cryptographers should tackle). Sections 1 and 3 are written for an audience of academics who might be less familiar with the political implications of crypto than the typical HN reader. Essentially, Rogaway is trying to convince mathematicians to embrace the practical and political impact of their work. Colin Percival gets a nice mention towards the end of the paper. I'd be over the moon if I were him. Congrats, Colin!

nickpsecurity 11y ago
It's a nice paper. Turns out, he grudgingly believes what I've been saying all along: "I am not optimistic. The gure of the heroic cryptographer sweeping in to save the world from totalitarian surveillance is ludicrous. And in a world where intelligence agencies stockpile and exploit countless vulnerabilities, obtain CA secret keys, subvert software-update mechanisms, infltrate private companies with moles, redirect online discussions in favored directions, and exert enormous in uence on standards bodies, cryptography alone will be an ineectual response. At best, cryptography might be a tool for creating possibilities within contours circumscribed by other forces." Exactly. Gotta deal with those "other forces," which are social and legal. Otherwise, the opponents vast resources work around the few that try to resist with the crypto. Doesn't stop many technical people from thinking crypto will magically save the day.
ropman76 11y ago
This is a very good paper and it's disappointing that it didn't get higher on HN. That being said I disagree with some of paper's points. I don't believe that cryptographers should take on moral responsibility for how they perceive their work may or may not be used. If a malware writer uses XTS to lock up a hard drive should Dr Rogeway be morally responsible for that since he helped create the constructions for XTS? I would argue "no" because there is a moral separation between idea and implementation. We should not burden cryptographers with moral baggage that should be placed on the people who implement or set policy to implement.
- dvanduzer 11y ago
  There is a clear distinction between a moral responsibility and a legal liability. The moral duties of scientists versus policy makers have little to do with the moral separation between idea and implementation.
- shkkmo 11y ago
  Are you arguing that the scientists who developed the first atomic bomb share no moral responsibility for the (positive or negative) effects of its use? Dr. Rogeway is morally responsible both for the use of his encryption for cyber ransom and for the use of his encryption to allow private democratic and economic discourse. If you think the negative effects of publicly available encryption outweigh the positive effects, it would be immoral to be working on making better encryption publicly available. Obviously, there is a complicated moral calculus behind which effects are foreseeable/likely relative to your other available actions and their foreseeable/likely effects.

AdmiralAsshat 11y ago

They lead one to ask if our inability to effectively address mass surveillance constitutes a failure of our field. I believe that it does. I call for a community-wide effort to develop more effective means to resist mass surveillance. What's funny is that you could talk to any number of law enforcement officials who believe that the moral failing is on the cryptography community for not providing a "backdoor" into encrypted communications. Or to restate, "Please, Apple, think of the children!" Morality is, unfortunately, subjective. Part of the argument is in convincing your opponent that your morality is superior to theirs. Or, perhaps, that their stance violates their own sense of morality.

DennisP 11y ago
Part three of the paper contains his justification for opposing mass surveillance.
policestate101 11y ago
When your opponent condones torture, indefinite imprisonment of innocents, war crimes, and genocide, there is little point in having anbopen dialog or argument with them.
- hueving 11y ago
  There are many people that don't condone those things but still think law enforcement should be able to get into people's computers or read communications with a warrant. It's more nuanced than you would like to believe.
  - policestate101 11y ago
    Nauance is really serving all those blacks in jail for fabricated evidence. Nauance has done wonders for the people of Syria. Nauanc as in "disposition matrix" instead of targeted assassinations. Sometimes things are so nauanced hospitals filled with doctors without borders get bombed, so nuanced we may not even consider it a war crime! But do on about how much more nuanced the world really is, how incredibly morally bankrupt, what weak moral fiber to defend torture through the weasel words of nauace, the millions of dead surely are glad for such a reasonable and nauceced view you present. Again, debating people who condone torture and mass murder is a waste of time.

pcl 11y ago

From the abstract: Cryptography rearranges power: it configures who can do what, from what. I'd argue that the reverse is really the issue that needs more attention. Online systems that do not provide strong cryptography rearrange power, as compared to their offline equivalents. It was not feasible to scan all phone calls for keywords in 1970, since that required effort from humans to do the patching and listening. The power dynamic changed when our industry brought those calls into a centralized, trivially-storable clear-text format. Encrypting the conversations is simply a partial return to the status quo of a few decades ago.

mangeletti 11y ago
+100 What powerful tool is I incrementalism that the whole population has forgotten that we used to be private individuals. The unification of the human race by the Internet now threatens the relevance of overgrown governments and banks. That's why they fight us.
- dasil003 11y ago
  I wouldn't say they do it because they feel threatened—how often do bureaucrats genuinely feel threatened? Most likely they are just legitimately trying to do their jobs, and the natural tendency will be to seek any power available that seems relevant to that task. Does any individual or organization ever voluntarily cede power?
nathan_long 11y ago
Good point. And even longer ago, all conversations required meeting, and were private unless an eavesdropper was physically nearby. Tech has created more extreme possibilities on both sides. On the privacy side, it's possible to exchange messages at a distance in an unreadable and nearly undetectable way. On the surveillance side, it's possible to eavesdrop on nearly everyone (except the very sophisticated). We can't and (don't want to) go back in time. The real question is "which danger is greater: conspiracy or oppression?"
- PhaseMage 11y ago

sneak 11y ago

So I guess we have a moral imperative to fork Chrome to actually enforce cert pinning even against locally-installed roots, then?

shkkmo 11y ago
Enforce cert pinning? No. Notify when it is overridden by locally-installed roots? Yes.
magicalist 11y ago
...as opposed to Firefox, Edge, and Safari which all do the same?

dang 11y ago

Posted a few days ago at https://news.ycombinator.com/item?id=10655418 https://news.ycombinator.com/item?id=10655418, but got so little discussion that we won't treat it as a dupe but have instead merged the threads.

cscheid 11y ago
Huh, thank you. I missed it back then, and assumed that when the submission went through it meant that the dupe detector okayed it.

NHQ 11y ago

OBVIOUSLY CHAOTIC NEUTRAL

PhaseMage 11y ago

I completely agree that Cryptography researchers should evaluate their work against their moral values. I feel the same thing about pretty much all engineering... I've been focusing on trying to design moral networking protocols.

pdkl95 11y ago

> This makes cryptography an inherently political tool. It always was. Many developers like to stay out of politics. Concentrating on difficult technical problems is hard enough; adding in politics is therefor adding in unnecessary complexity. As the wonderful Tom Lehrer put it in his song "Wernher Von Braun"[1], Don't say that he's hypocritical, Say rather that he's apolitical. "once the rockets are up, who cares where they come down? That's not my department," says Wernher von Braun. The problem with this is similar to the problem of abstaining from the vote: it's absolutely not a neutral position. Choosing to abstain from politics in general, like those that choose to abstain from the vote, is de facto a vote for the status quo and majority rule. Not only is cryptography an inherently political tool, almost all software is political. Software does not exist in a vacuum; the entire point of most software is that it has an impact on business, society, and the world. With the discovery of the General Purpose Computer, this impact can be very large. It's easy to see why cryptography disrupts existing power structures. It should be similarly easy to see how software already overturned the traditional power structures in places like the stock market, manufacturing, and retail. So please, consider what impact your software might have when you are writing it, or if someone already has a goal in mind. Maybe, in some cases, it's better to walk away. It;'s a hard question, but the answer is not to say "I'm staying out of politics". To quote Quinn Norton and Eleanor Saitta from their talk[2] at 30c3, there is "no neutral ground in a burning world". [1] https://www.youtube.com/watch?v=QEJ9HrZq7Ro#t=16 https://www.youtube.com/watch?v=QEJ9HrZq7Ro#t=16 [2] https://www.youtube.com/watch?v=DWg2qEEa9CE https://www.youtube.com/watch?v=DWg2qEEa9CE

bryanhm 11y ago

Here we have what I think is a display of an intelligent mind specialized in one area, funded by a state institution, and weak at resolving moral conflicts. Computer scientists (and any self-respecting scientists) HAVE to separate their ethics from the interests of state institutions. Phrases in the paper resembling something like "where the cryptographer has a duty to serve the public and keep their self-interest in check" indicate this. I've read a paper recently on designing systems to have security exceptions for law enforcement and calling them "exception requirements" or something to that effect. This is the sort of thing a good study of ethics can help to resolve.

jvdh 11y ago
Computer scientists (and any self-respecting scientists) HAVE to separate their ethics from the interests of state institutions. Why? The paper argues exactly to the opposite in the first part, describing the atomic-bomb scientists, and the Russell-Einstein Manifesto. In my opinion, cryptographers and computer scientists have ignored morally questioning their work for too long. Reality is now catching up on this, with techniques for surveillance. The paper argues this for cryptography, in my view this is even more generally applicable.
- bryanhm 11y ago
  Why do scientists/cryptographers have to judge ethics for themselves? Not sure what you're asking. Bertrand Russell judged the bomb to be a danger for humanity and chose to express his own views on it.
  - jvdh 11y ago
    Medical ethics committees have ethicists, but the main advice is from medical experts. I would not want anybody else judging the ethics of medical experiments, as no one else has the expert knowledge. I see absolutely no reason why this should different for computer scientists and cryptographers. No one else understands the implications of an experiment or a new methodology. I was at a meeting recently discussing ethics committees for computer science. A medical expert gave his opinion and said (paraphrasing): I fail to see the problem with digitalisation, we have had medical records on paper for years, now they are on a computer, what is the difference? I don't mean to say that every scientist should do this individually. They should discuss this with colleagues, and with an ethics committee, which should contain subject matter experts, but also ethicists.

zby 11y ago

Cyber-security in general is political. This guy is a cryptographer - so it is natural that he formulated this for his own area - but it is too narrow. Hackers are now routinely the foot soldiers of the cyber-war of everyone against everyone - we need to think more about our own rules instead of following orders.

kluck 11y ago

It is the responsibility of the inventor to think ahead as far as possible about how an invention may do good and/or harm.

jroitgrund 11y ago

The biggest thing I took away from this was reading the slides and seeing the FBI's suicide letter to Martin Luther King. I didn't know about that, and reading that right after seeing the jury dury article on the front page today is chilling to say the least.

gcb0 11y ago

everyone has those talks 30yrs ago when crypto was labeled munition by the usa. if this article has any content that warrant discussion, is how out of touch with reality the social sciences are. ... and sadly, yet another proof of how necessary Snowden was.

jvvw 11y ago

This was one of the factors (certainly not the only one) that made me move out of the field of cryptography - I was doing work related to attempting to break a major cryptosystem and I realised that I wasn't completely sure what the right course of action was ethically in the slim chance that I succeeded. My background was in pure mathematics and up until I moved into cryptography, it seemed obvious that openess of information was an obvious good. However, once in the crypto field, it became much a much more ambiguous issue.