8 ms·

We need a code of ethics for software developers, with actual teeth (violating it means you can't commercially build software any more). Edit: it's amazing how

We need a code of ethics for software developers, with actual teeth (violating it means you can't commercially build software any more). Edit: it's amazing how much the alt-right has infiltrated Hacker News. The notion that we, as developers, should have a professional obligation to engage ethically should not be so controversial (yet here I am with negative karma for proposing it). To expand on my point: 1. This is hardly a novel suggestion. Lots of positions have professional associations with ethics rules. See engineers, doctors, lawyers, librarians, etc. 2. This is not a union. Lawyers don't bargain collectively. 3. Nobody is talking about illegal immigration. The worry here is around proposals for a database of Muslim-Americans (who are almost all here legally). 4. Nobody is coming for your code. Obviously you won't need to enter your professional PIN to compile code. It just means you'd have to be a member to work for a major tech company.

anigbrowl 10y ago
That's no more possible than a code of ethics for painters that would prevent them from creating new pictures. But the technical impossibility of this is no barrier to stringent social sanctions that would make it virtually impossible to get hired to any position of responsibility in the future. A code cannot have teeth without being backed up by some organization to do the biting, as it were, meaning some sort of union or professional association whose bylaws enshrine these ethical standards. Otherwise the 21st century equivalent of 'I was only following orders' will be 'that's above my pay grade.'
- tptacek 10y ago
  All that's needed to enforce a code of ethics is shop rules requiring membership in the relevant professional organization. That organization needn't be a union.
- morgante 10y ago
  It's not a technical problem, it's a social one. Obviously we're not going to make it impossible to code without being a member of the developer guild. But it's totally feasible to make it so you can't get hired by any self-respecting tech company without being a member. Start at the top and things will trickle down quickly. I feel like these sorts of discussions too frequently get entangled with unions and collective bargaining, which they really shouldn't. Lawyers don't bargain collectively, but they're all members of the bar.
GuiA 10y ago
How do you enforce that? How do you avoid it making software even costlier to build by several orders of magnitude? How do you make sure that outsourced developers that will inevitably be used due to this rise in cost follow the code of ethics?
- tptacek 10y ago
  How do ASME members enforce their code of ethics? How do doctors enforce the Hippocratic Oath? We should resist the temptation of assuming that just because software development is new, the professional ethics problems we face are novel or especially challenging.
  - yarou 10y ago
    IMO, you'd be solving the wrong problem. The real problem that needs to be solved is licensing software engineers. This comes with the added benefit of standardizing competence. If lawyers and doctors need licensing, why shouldn't we also have a licensing body for software engineers? Then you get the ethics enforcement for free.
    - tptacek 10y ago
      I'm happy to take on one problem at a time.
  - stale2002 10y ago
    The issue is that very few people would be willing to be treated by an unlicensed doctor. The same is not true for software and will NEVER be true for software. Nobody is ever going to care if the guy fiddling around with CSS and HTML to make a marketing landing page is licensed or not. Hell, I know small startups that will pay a bright high school student to do stuff like that. Vast swafts of software engineering can be reduced to basic crud web dev that a bootcamper college dropout is capable of doing. There is no "high school intern" level equivalent of medical work that the world has a massive shortage of.
    - 10y ago
tptacek 10y ago
I was thinking about this on the drive home. A minimal set: * That we be mindful of the difficulty of our craft and not work on systems where failures can harm others when we're uncertain of our ability to execute reliably. For instance: that we not work on projects that seem likely to produce another Therac-25 disaster. * That we be mindful of the outsized impact technology can have, especially when put to unexpected use, and thus be obligated to design systems with all reasonable safeguards against misuse. For instance: that we not work on projects that are benign when used as designed, but that are easily turned into instruments of mass surveillance --- like: a big chunk of the network security product market. * That we design and build systems that ordinary users can reliably use safely and, where applicable, securely. For instance: that we don't repeat the mistakes of Adobe Flash.
- quesera 10y ago
  Numbers 1 and 3 are great. Number 2 is effectively impossible. You correctly point out the entire netsec industry, but it goes much much further. Cisco, Oracle, Google, and Facebook would all immediately fail that test. I would shed no tears, but it'd be a tough sell with most people.
  - tptacek 10y ago
    You'd be surprised. Sit down with a group of smart people, isolate a single problem, and pick it apart. For instance: you've got a giant database of sensitive information that could be used to build a(nother) national Muslim registry. So: * Aggressively aggregate data into statistical aggregates that address your business questions while discarding, the way we already do for credit card data, sensitive information we don't immediately need. * Seed the database with false information in a way that allows it to answer the business logic questions your application needs, but that would respond to an unexpected compelled production request with data that would be untenably expensive to true back up. * Design the system so that it can efficiently answer the questions the system was designed to answer, but that takes proof-of-work-scale effort to answer other questions. There are already database systems with these designs. There are plenty of other answers; this was like a 5 minute conversation in a bar in San Francisco a few weeks ago. The point is that rule #2 isn't a professional obligation today, so there's little incentive to design comprehensive safeguards --- or really even to conduct the misuse modeling exercises that would even suggest what kinds of countermeasures are needed. What I think is the case is that we won't come up with good solutions to these problems until we make them professionally mandatory to solve, so that their solutions enable new revenue by enabling applications to be built that could not be built ethically otherwise.
intransigent 10y ago
Wouldn't that require the building of a database that is used to discriminate against individuals listed therein?
- morgante 10y ago
  I feel like you're trying to make some witty point, but it's actually totally nonsensical. Yes, I absolutely want a database of ethical software engineers and wholeheartedly support discriminating against people who aren't in it.
- tptacek 10y ago
  Almost every other major profession has rules like these. You can't even prepare taxes without complying with a code of ethics. This isn't the crazy logistical challenge you're making it out to be.
  - intransigent 10y ago
    - Taking the hippocratic oath - Passing the bar - Obtaining a pilot's license - Completing series 7,(63,66) exams Yes, these are things people do. But these things are each different from each other, and different from a petition. Some of them are not centralized lists, and others are not calls to political action. Meanwhile others are discriminatory, in that they distinguish those who have proven proficiencies, as capable practicioners of valuable skills. Simply affirming a willingness to act in a certain way only means so much. Refusing to act means less. Indicating an intent to prevent others from acting is yet another behavior. Advertising any political intentions, regardless of actual skills or rationale, may carry sentimental value, and that's nice, I guess. Meanwhile the possession of technical skills, which rest on the firmament of electrified silicon substrates, with open-ended usage patterns, which may eventually lead to these objects ...behaving with some semblance of ...sentient free will? Deciding thier own ethics? In ways not obvious to the inventor? Are their parenting licenses? Just kidding. That last part's a can of worms all its own.
    - tptacek 10y ago
abstractbeliefs 10y ago
But who decides what's ethical? I struggle with the same issues here in the UK, where the professional body for tech, the British Computing Society (BCS) is not just toothless, but in my eyes, actively harmful (for example, chose to vote _in favour_ of NSA-like practices at GCHQ - including giving evidence (in the form of an expert statement) to the government). I think what is important to realise is that something approximating the majority of people do actually go against the values typically espoused by the people who visit HN, not just in the wider public but also within tech. And even discounting those, how many people are quite happy to just do as told? How do we work to distill the concept of professional ethics in tech from these people, and then instill it?
- idlewords 10y ago
  I think a good first step is to learn from groups—doctors, librarians, lawyers, engineers—who have such a working system of professional ethics in place.
  - brighteyes 10y ago
    Definitely true, and all the ones I am familiar with are purposefully non-partisan. That is, they define as ethical things that both Democrats and Republicans agree on.
    - idlewords 10y ago
      That's very much not the case. Consider the epic battle between librarians and John Ashcroft in the wake of 9/11, over patron privacy.
      - brighteyes 10y ago
        I am less familiar with the librarian code of ethics, but yes, from what I've heard they are quite unique and different from the others.
        10y ago
ar15saveslives 10y ago
What do you mean "you can't build"? What does "build software" mean? You can't run gcc anymore? Contribute to opensource? Publish your app? What would you add to that "code of ethics"? Should I be banned from "build software" if my library is used in non-ethical (in someone's opinion) project? Is working for institutions in countries that violate human rights (UAE?) should be considered "unethical"? Just take your witch hunt ideas and leave us alone.
- idlewords 10y ago
  A reasonable code of professional ethics is not a witch hunt.
  - flamedoge 10y ago
    based on whose "reason" is reasonable?
    - idlewords 10y ago
      Based on broad consensus within the profession, like happens in journalism, law, medicine, social science etc. etc.
      - flamedoge 10y ago
        Negative enforcement by punishment is not likely to hold well for programmers. What are you gonna do, ban all hacking? good luck trying to enforce that.
        idlewords 10y ago
        You think of this in terms of a ban, but what a professional code of ethics really does is offer cover to people who don't want to behave unethically.
        flamedoge 10y ago
sean_patel 10y ago
> We need a code of ethics for software developers Since you went there, we also need a Union to protect our rights and prevent overtime abuse (esp. at Startups).
- morgante 10y ago
  No we don't. Please don't bring in unrelated issues to muddle the waters. We can have a professional association without having a union.
  - adrienne 10y ago
    Sure, but we still need a union, too. Just because a large proportion of capitalism-worshipping Hacker News bros think unions are eeeeeevil is no reason we should shut up about them.
    - morgante 10y ago
      For the time being I'd rather focus on a professional association, since talk of unions will send those capitalist bros running for the hills (and I think ethics are more important than protecting our salaries). Also I think we're professionals. Doctors and engineers are doing just fine without unionizing.
      - adrienne 10y ago
        Unions aren't just about salaries, they're about resisting all sorts of unfair management bullshit. And developers may be "professionals" (that's kind of a loaded fucking word; carpenters are professionals too, and they have unions), but we are also the victims of a lot of unfair management bullshit, including unpaid on-call time and mandatory unpaid overtime (oh, but we're exempt, that makes it okay? Fuck that.) I'm more concerned with ethics than salaries, too. Which is precisely why i think unions are important. Corporations have no ethics. Management has no ethics.
        10y ago
slavik81 10y ago
Any self-regulating professional organization ultimately is granted its status by the government. If its ethics deviate significantly from that of wider society, it will lose support in government and will probably not survive as an organization. Trying to create a government-appointed organization to oppose a government policy is simply foolish. It's not going to happen.
- morgante 10y ago
  There are plenty of examples where legal ethics or librarian ethics have conflicted with government policy.
  - slavik81 10y ago
    While that is true, they largely agree with policy. There's only occasional conflicts. Those organisations have long histories and clearly defined, non-partisan principles that most people largely agree with. Congress is not going to pass a law to deport Mexicans and Muslims, then pass a law to establish an organization that is designed to thwart their previous law.
    - morgante 10y ago
      Perhaps Congress won't, but that's assuming that government is unitary. Professional licensing is done at the state level anyways. It would be very easy to imagine, for example, California passing a law to establish/bolster a developer guild at the same time that Congress is trying to deport Muslims.
      - slavik81 10y ago
        I don't think state governments could bar programmers from working on federal projects. I suppose they might be able to bar them from working on anything but federal projects, though. Maybe. I'm not exactly a constitutional scholar.
        10y ago
carsongross 10y ago
We need a code of ethics for writers, with actual teeth (violating it means you can't commercially write any more). We need a code of ethics for directors, with actual teeth (violating it means you can't commercially direct films any more). This is fun!
- morgante 10y ago
  We need a code of ethics for lawyers, with actual teeth (violating it means you can't practice law any more). We need a code of ethics for doctors, with actual teeth (violating it means you can't practice medicine any more). You're right, this is fun! (Also, really poor choice of examples. Both writers and directors have large and successful unions.)
  - deleted 10y ago
    [deleted]
threatofrain 10y ago
The AMA or ABA is useful for censuring activities which are obviously noxious to broader society, which helps it manage its standing within society, but I think that is almost the entirety of the motivation, utility, and scope of professional ethics. Nevertheless I think there are advantages to organization and I hope for more of it. That being said, You can't stop a metaphorical mass database of Muslims at a time when there's sufficient political support to do so -- and there's plenty at the moment. A lot of people see <it's just information, what do you have to hide if you're righteous> as a good enough reason, enough to generate broad acceptance. There's also enough political support to build illegal mass surveillance machinery, and then lie to Congress about it. The aggregate of the US is okay with that, just like how the aggregate of the UK is okay with even more in-your-face surveillance. Likewise, you cannot stop the documented abuses of psychiatry when there's all too much political support for it. There are useful reasons for a union, and I hope it so for the health of the technology community, but I don't think it can dent the issues which people mean by this "never again" political speech (big databases of people -> risk to vulnerable minorities).

anigbrowl 10y ago

That's no more possible than a code of ethics for painters that would prevent them from creating new pictures. But the technical impossibility of this is no barrier to stringent social sanctions that would make it virtually impossible to get hired to any position of responsibility in the future. A code cannot have teeth without being backed up by some organization to do the biting, as it were, meaning some sort of union or professional association whose bylaws enshrine these ethical standards. Otherwise the 21st century equivalent of 'I was only following orders' will be 'that's above my pay grade.'

tptacek 10y ago
All that's needed to enforce a code of ethics is shop rules requiring membership in the relevant professional organization. That organization needn't be a union.
morgante 10y ago
It's not a technical problem, it's a social one. Obviously we're not going to make it impossible to code without being a member of the developer guild. But it's totally feasible to make it so you can't get hired by any self-respecting tech company without being a member. Start at the top and things will trickle down quickly. I feel like these sorts of discussions too frequently get entangled with unions and collective bargaining, which they really shouldn't. Lawyers don't bargain collectively, but they're all members of the bar.

GuiA 10y ago

How do you enforce that? How do you avoid it making software even costlier to build by several orders of magnitude? How do you make sure that outsourced developers that will inevitably be used due to this rise in cost follow the code of ethics?

tptacek 10y ago
How do ASME members enforce their code of ethics? How do doctors enforce the Hippocratic Oath? We should resist the temptation of assuming that just because software development is new, the professional ethics problems we face are novel or especially challenging.
- yarou 10y ago
  IMO, you'd be solving the wrong problem. The real problem that needs to be solved is licensing software engineers. This comes with the added benefit of standardizing competence. If lawyers and doctors need licensing, why shouldn't we also have a licensing body for software engineers? Then you get the ethics enforcement for free.
  - tptacek 10y ago
    I'm happy to take on one problem at a time.
- stale2002 10y ago
  The issue is that very few people would be willing to be treated by an unlicensed doctor. The same is not true for software and will NEVER be true for software. Nobody is ever going to care if the guy fiddling around with CSS and HTML to make a marketing landing page is licensed or not. Hell, I know small startups that will pay a bright high school student to do stuff like that. Vast swafts of software engineering can be reduced to basic crud web dev that a bootcamper college dropout is capable of doing. There is no "high school intern" level equivalent of medical work that the world has a massive shortage of.
  - 10y ago

tptacek 10y ago

I was thinking about this on the drive home. A minimal set: * That we be mindful of the difficulty of our craft and not work on systems where failures can harm others when we're uncertain of our ability to execute reliably. For instance: that we not work on projects that seem likely to produce another Therac-25 disaster. * That we be mindful of the outsized impact technology can have, especially when put to unexpected use, and thus be obligated to design systems with all reasonable safeguards against misuse. For instance: that we not work on projects that are benign when used as designed, but that are easily turned into instruments of mass surveillance --- like: a big chunk of the network security product market. * That we design and build systems that ordinary users can reliably use safely and, where applicable, securely. For instance: that we don't repeat the mistakes of Adobe Flash.

quesera 10y ago
Numbers 1 and 3 are great. Number 2 is effectively impossible. You correctly point out the entire netsec industry, but it goes much much further. Cisco, Oracle, Google, and Facebook would all immediately fail that test. I would shed no tears, but it'd be a tough sell with most people.
- tptacek 10y ago
  You'd be surprised. Sit down with a group of smart people, isolate a single problem, and pick it apart. For instance: you've got a giant database of sensitive information that could be used to build a(nother) national Muslim registry. So: * Aggressively aggregate data into statistical aggregates that address your business questions while discarding, the way we already do for credit card data, sensitive information we don't immediately need. * Seed the database with false information in a way that allows it to answer the business logic questions your application needs, but that would respond to an unexpected compelled production request with data that would be untenably expensive to true back up. * Design the system so that it can efficiently answer the questions the system was designed to answer, but that takes proof-of-work-scale effort to answer other questions. There are already database systems with these designs. There are plenty of other answers; this was like a 5 minute conversation in a bar in San Francisco a few weeks ago. The point is that rule #2 isn't a professional obligation today, so there's little incentive to design comprehensive safeguards --- or really even to conduct the misuse modeling exercises that would even suggest what kinds of countermeasures are needed. What I think is the case is that we won't come up with good solutions to these problems until we make them professionally mandatory to solve, so that their solutions enable new revenue by enabling applications to be built that could not be built ethically otherwise.

intransigent 10y ago

Wouldn't that require the building of a database that is used to discriminate against individuals listed therein?

morgante 10y ago
I feel like you're trying to make some witty point, but it's actually totally nonsensical. Yes, I absolutely want a database of ethical software engineers and wholeheartedly support discriminating against people who aren't in it.
tptacek 10y ago
Almost every other major profession has rules like these. You can't even prepare taxes without complying with a code of ethics. This isn't the crazy logistical challenge you're making it out to be.
- intransigent 10y ago
  - Taking the hippocratic oath - Passing the bar - Obtaining a pilot's license - Completing series 7,(63,66) exams Yes, these are things people do. But these things are each different from each other, and different from a petition. Some of them are not centralized lists, and others are not calls to political action. Meanwhile others are discriminatory, in that they distinguish those who have proven proficiencies, as capable practicioners of valuable skills. Simply affirming a willingness to act in a certain way only means so much. Refusing to act means less. Indicating an intent to prevent others from acting is yet another behavior. Advertising any political intentions, regardless of actual skills or rationale, may carry sentimental value, and that's nice, I guess. Meanwhile the possession of technical skills, which rest on the firmament of electrified silicon substrates, with open-ended usage patterns, which may eventually lead to these objects ...behaving with some semblance of ...sentient free will? Deciding thier own ethics? In ways not obvious to the inventor? Are their parenting licenses? Just kidding. That last part's a can of worms all its own.
  - tptacek 10y ago

abstractbeliefs 10y ago

But who decides what's ethical? I struggle with the same issues here in the UK, where the professional body for tech, the British Computing Society (BCS) is not just toothless, but in my eyes, actively harmful (for example, chose to vote _in favour_ of NSA-like practices at GCHQ - including giving evidence (in the form of an expert statement) to the government). I think what is important to realise is that something approximating the majority of people do actually go against the values typically espoused by the people who visit HN, not just in the wider public but also within tech. And even discounting those, how many people are quite happy to just do as told? How do we work to distill the concept of professional ethics in tech from these people, and then instill it?

idlewords 10y ago
I think a good first step is to learn from groups—doctors, librarians, lawyers, engineers—who have such a working system of professional ethics in place.
- brighteyes 10y ago
  Definitely true, and all the ones I am familiar with are purposefully non-partisan. That is, they define as ethical things that both Democrats and Republicans agree on.
  - idlewords 10y ago
    That's very much not the case. Consider the epic battle between librarians and John Ashcroft in the wake of 9/11, over patron privacy.
    - brighteyes 10y ago
      I am less familiar with the librarian code of ethics, but yes, from what I've heard they are quite unique and different from the others.
      - 10y ago

ar15saveslives 10y ago

What do you mean "you can't build"? What does "build software" mean? You can't run gcc anymore? Contribute to opensource? Publish your app? What would you add to that "code of ethics"? Should I be banned from "build software" if my library is used in non-ethical (in someone's opinion) project? Is working for institutions in countries that violate human rights (UAE?) should be considered "unethical"? Just take your witch hunt ideas and leave us alone.

idlewords 10y ago
A reasonable code of professional ethics is not a witch hunt.
- flamedoge 10y ago
  based on whose "reason" is reasonable?
  - idlewords 10y ago
    Based on broad consensus within the profession, like happens in journalism, law, medicine, social science etc. etc.
    - flamedoge 10y ago
      Negative enforcement by punishment is not likely to hold well for programmers. What are you gonna do, ban all hacking? good luck trying to enforce that.
      - idlewords 10y ago
        You think of this in terms of a ban, but what a professional code of ethics really does is offer cover to people who don't want to behave unethically.
        flamedoge 10y ago

sean_patel 10y ago

> We need a code of ethics for software developers Since you went there, we also need a Union to protect our rights and prevent overtime abuse (esp. at Startups).

morgante 10y ago
No we don't. Please don't bring in unrelated issues to muddle the waters. We can have a professional association without having a union.
- adrienne 10y ago
  Sure, but we still need a union, too. Just because a large proportion of capitalism-worshipping Hacker News bros think unions are eeeeeevil is no reason we should shut up about them.
  - morgante 10y ago
    For the time being I'd rather focus on a professional association, since talk of unions will send those capitalist bros running for the hills (and I think ethics are more important than protecting our salaries). Also I think we're professionals. Doctors and engineers are doing just fine without unionizing.
    - adrienne 10y ago
      Unions aren't just about salaries, they're about resisting all sorts of unfair management bullshit. And developers may be "professionals" (that's kind of a loaded fucking word; carpenters are professionals too, and they have unions), but we are also the victims of a lot of unfair management bullshit, including unpaid on-call time and mandatory unpaid overtime (oh, but we're exempt, that makes it okay? Fuck that.) I'm more concerned with ethics than salaries, too. Which is precisely why i think unions are important. Corporations have no ethics. Management has no ethics.
      - 10y ago

slavik81 10y ago

Any self-regulating professional organization ultimately is granted its status by the government. If its ethics deviate significantly from that of wider society, it will lose support in government and will probably not survive as an organization. Trying to create a government-appointed organization to oppose a government policy is simply foolish. It's not going to happen.

morgante 10y ago
There are plenty of examples where legal ethics or librarian ethics have conflicted with government policy.
- slavik81 10y ago
  While that is true, they largely agree with policy. There's only occasional conflicts. Those organisations have long histories and clearly defined, non-partisan principles that most people largely agree with. Congress is not going to pass a law to deport Mexicans and Muslims, then pass a law to establish an organization that is designed to thwart their previous law.
  - morgante 10y ago
    Perhaps Congress won't, but that's assuming that government is unitary. Professional licensing is done at the state level anyways. It would be very easy to imagine, for example, California passing a law to establish/bolster a developer guild at the same time that Congress is trying to deport Muslims.
    - slavik81 10y ago
      I don't think state governments could bar programmers from working on federal projects. I suppose they might be able to bar them from working on anything but federal projects, though. Maybe. I'm not exactly a constitutional scholar.
      - 10y ago

carsongross 10y ago

We need a code of ethics for writers, with actual teeth (violating it means you can't commercially write any more). We need a code of ethics for directors, with actual teeth (violating it means you can't commercially direct films any more). This is fun!

morgante 10y ago
We need a code of ethics for lawyers, with actual teeth (violating it means you can't practice law any more). We need a code of ethics for doctors, with actual teeth (violating it means you can't practice medicine any more). You're right, this is fun! (Also, really poor choice of examples. Both writers and directors have large and successful unions.)
- deleted 10y ago
  [deleted]

threatofrain 10y ago

The AMA or ABA is useful for censuring activities which are obviously noxious to broader society, which helps it manage its standing within society, but I think that is almost the entirety of the motivation, utility, and scope of professional ethics. Nevertheless I think there are advantages to organization and I hope for more of it. That being said, You can't stop a metaphorical mass database of Muslims at a time when there's sufficient political support to do so -- and there's plenty at the moment. A lot of people see <it's just information, what do you have to hide if you're righteous> as a good enough reason, enough to generate broad acceptance. There's also enough political support to build illegal mass surveillance machinery, and then lie to Congress about it. The aggregate of the US is okay with that, just like how the aggregate of the UK is okay with even more in-your-face surveillance. Likewise, you cannot stop the documented abuses of psychiatry when there's all too much political support for it. There are useful reasons for a union, and I hope it so for the health of the technology community, but I don't think it can dent the issues which people mean by this "never again" political speech (big databases of people -> risk to vulnerable minorities).