5 ms·

I’ve been saying this a lot lately, but software needs a professional associstion (like the AMA) with teeth. Ethics are not easy, and without collective governi

I’ve been saying this a lot lately, but software needs a professional associstion (like the AMA) with teeth. Ethics are not easy, and without collective governing power, they simply won’t exist. https://news.ycombinator.com/item?id=16423328 https://news.ycombinator.com/item?id=16423328 https://news.ycombinator.com/item?id=16420659 https://news.ycombinator.com/item?id=16420659

whataretensors 9y ago
I disagree. Not in principle, but because I think it won't work. If an organization has teeth in this market it will be turned into a weapon to gate out or remove competition.
- IntronExon 9y ago
  What makes software so different from every other endeavor in that regard?
  - whataretensors 9y ago
    Scope
    - IntronExon 9y ago
      Medicine encompasses everything from practice to research, humans to non-human animals, procedures and devices to drugs and therapies, and touches nearly every human alive. Engineering is even more diverse and penetrating, and software is merely a subset. It ain’t scope.
  - Bartweiss 9y ago
    If I understand your question: nothing, professional associations become anti-competitive gatekeepers in every endeavor. Roughly 30% of US workers have mandatory occupational licensing before they can do their jobs. Licensed services have roughly 15% higher prices than nonlicensed ones. Most of those workers are not in safety critical jobs - on average an interior designer faces more stringent licensing requirements than an EMT. Unions and professional associations have consistently been the main lobbyists advocating professional licensing as a way to keep wages high (by keeping supply low) and guarantee a role for the organization. Even in medicine, the number of residency slots is controlled in a way that creates a stable, minor shortage of doctors. I'm not against all licensing and oversight, obviously. I'd rather have a cap on residencies than untrained doctors, if those are the only two choices. But the history of 'every other endeavor' looks to me like a lesson in how oversight organizations will cause exactly this problem.
    - IntronExon 9y ago
      There is always a trade off, but each occupation learns the hard way thst it’s worth it. Hopefully software will figure that out in time to regulate itself before the government steps in with agendas and a heavy hand.
- carapace 9y ago
  So... competition > ethics? Ethical software with less competition seems better than unethical software with more competition, eh?
  - deleted 9y ago
    [deleted]
  - rocqua 9y ago
    I worry that Ethical software will lose out to unethical software. There is too much pressure and low-hanging fruit to do any kind of gate-keeping that won't be circumvented. If you want to take the required measures to not circumvent the gate-keeper, that is strangling the entire sector. Since most developed countries see this sector as the basis for future growth, so doing that seems unlikely. Beyond that, the growth would probably move to less restrictive places.
  - Bartweiss 9y ago
    I suppose the question is how much faith you have in oversight boards to actually produce ethical outcomes. My worry wouldn't be "they'll cut down on competition to ensure ethical behavior", it would be "they'll block competition and protect their member instead of ensuring ethics." I think the top-level essay here is extremely cogent: oversight is almost always limited to ensuring competence. In medicine, that gets us 'ethics' when ethics means "providing good care and avoiding bad care". (It hasn't gotten us ethics when the question is anything complicated, like defining mental illness.) Competence in data science and software is largely orthogonal to ethics, so I expect an oversight board would achieve very little in that regard.
schaunwheeler 9y ago
A think a governing body solves the wrong problem. Most of the ethical issues that have arisen around data science in particular are actually competency issues. It's not that someone deliberately set out to target already-vulnerable populations. It's that they designed systems without building in checks for unintended consequences. A governing body accomplishes what any legal framework accomplishes: it provides instruction and incentives to limit liability. A legal framework is different than an ethical framework. What we need is an ethical framework. The original Hippocratic Oath did a fairly good job of this by stipulating ways that a doctor could prove his competence. Doctors who adhered to the oath weren't better doctors because they had some kind of internal moral compass or external adjudicating body. They were better doctors because only the doctors who had competency to spare were willing to make the sacrifices that adherence to the oath required. https://hackernoon.com/on-the-difficulty-of-creating-a-data-science-code-of-ethics-9ab44f71af52 https://hackernoon.com/on-the-difficulty-of-creating-a-data-... Ethical problems get solved (as much as that type of problem can ever really be "solved") by individual practitioners refusing to work with other individual practitioners who refuse to adhere to some basic best practices. That creates a network of competent and trustworthy individuals, who are still totally fallible, but who put their own practice up toe constant public scrutiny. Ethical problems are problems of systemic risk. Systemic risk doesn't get solved through organizations.
- s73v3r_ 9y ago
  The governing body would not just be there to arbitrate, advocate, or negotiate. They would also be there for promoting training and sharing of knowledge.
  - mr_spothawk 9y ago
    i want to stress that i'm not challenging the person, just the text. > The governing body would not just be there to arbitrate, advocate, or negotiate. They would also be there for promoting training and sharing of knowledge. this reads like duct-tape solutions. it's always the case that a governing body grows and grows and gets in the way. "it's not just there to XYZ, it also BLAHBLAHBLAH". a governing body is one possible solution to the problem of people not understanding ethics (or how to figure out what ethics are), that I don't think will solve the larger problem. a governing body will need to be run by practitioners of governance, and enforced by practitioners of enforcement. ethics needs to be practiced everywhere, all the time. and a governing body won't make that happen. people need to be learning about why their activities are "good" or "bad" or "questionable". in my (simplified version) view: we're not lacking incentive to work (money!), we're lacking incentive to not work (people shouting at us for being assholes for doing asshole work [e.g. bombs, baby killing, evil algos]). we need to cultivate behaviors of ethical culture... instead we're cultivating behaviors that promote short-term-ism (buy your way to happiness, bomb your way to safety, build your dream home on loan!, opportunity cost services saving us from having to cleaning up after ourselves, trash removal services that vanish things away to faraway dumps we never think of)

whataretensors 9y ago

I disagree. Not in principle, but because I think it won't work. If an organization has teeth in this market it will be turned into a weapon to gate out or remove competition.

IntronExon 9y ago
What makes software so different from every other endeavor in that regard?
- whataretensors 9y ago
  Scope
  - IntronExon 9y ago
    Medicine encompasses everything from practice to research, humans to non-human animals, procedures and devices to drugs and therapies, and touches nearly every human alive. Engineering is even more diverse and penetrating, and software is merely a subset. It ain’t scope.
- Bartweiss 9y ago
  If I understand your question: nothing, professional associations become anti-competitive gatekeepers in every endeavor. Roughly 30% of US workers have mandatory occupational licensing before they can do their jobs. Licensed services have roughly 15% higher prices than nonlicensed ones. Most of those workers are not in safety critical jobs - on average an interior designer faces more stringent licensing requirements than an EMT. Unions and professional associations have consistently been the main lobbyists advocating professional licensing as a way to keep wages high (by keeping supply low) and guarantee a role for the organization. Even in medicine, the number of residency slots is controlled in a way that creates a stable, minor shortage of doctors. I'm not against all licensing and oversight, obviously. I'd rather have a cap on residencies than untrained doctors, if those are the only two choices. But the history of 'every other endeavor' looks to me like a lesson in how oversight organizations will cause exactly this problem.
  - IntronExon 9y ago
    There is always a trade off, but each occupation learns the hard way thst it’s worth it. Hopefully software will figure that out in time to regulate itself before the government steps in with agendas and a heavy hand.
carapace 9y ago
So... competition > ethics? Ethical software with less competition seems better than unethical software with more competition, eh?
- deleted 9y ago
  [deleted]
- rocqua 9y ago
  I worry that Ethical software will lose out to unethical software. There is too much pressure and low-hanging fruit to do any kind of gate-keeping that won't be circumvented. If you want to take the required measures to not circumvent the gate-keeper, that is strangling the entire sector. Since most developed countries see this sector as the basis for future growth, so doing that seems unlikely. Beyond that, the growth would probably move to less restrictive places.
- Bartweiss 9y ago
  I suppose the question is how much faith you have in oversight boards to actually produce ethical outcomes. My worry wouldn't be "they'll cut down on competition to ensure ethical behavior", it would be "they'll block competition and protect their member instead of ensuring ethics." I think the top-level essay here is extremely cogent: oversight is almost always limited to ensuring competence. In medicine, that gets us 'ethics' when ethics means "providing good care and avoiding bad care". (It hasn't gotten us ethics when the question is anything complicated, like defining mental illness.) Competence in data science and software is largely orthogonal to ethics, so I expect an oversight board would achieve very little in that regard.

schaunwheeler 9y ago

A think a governing body solves the wrong problem. Most of the ethical issues that have arisen around data science in particular are actually competency issues. It's not that someone deliberately set out to target already-vulnerable populations. It's that they designed systems without building in checks for unintended consequences. A governing body accomplishes what any legal framework accomplishes: it provides instruction and incentives to limit liability. A legal framework is different than an ethical framework. What we need is an ethical framework. The original Hippocratic Oath did a fairly good job of this by stipulating ways that a doctor could prove his competence. Doctors who adhered to the oath weren't better doctors because they had some kind of internal moral compass or external adjudicating body. They were better doctors because only the doctors who had competency to spare were willing to make the sacrifices that adherence to the oath required. https://hackernoon.com/on-the-difficulty-of-creating-a-data-science-code-of-ethics-9ab44f71af52 https://hackernoon.com/on-the-difficulty-of-creating-a-data-... Ethical problems get solved (as much as that type of problem can ever really be "solved") by individual practitioners refusing to work with other individual practitioners who refuse to adhere to some basic best practices. That creates a network of competent and trustworthy individuals, who are still totally fallible, but who put their own practice up toe constant public scrutiny. Ethical problems are problems of systemic risk. Systemic risk doesn't get solved through organizations.

s73v3r_ 9y ago
The governing body would not just be there to arbitrate, advocate, or negotiate. They would also be there for promoting training and sharing of knowledge.
- mr_spothawk 9y ago
  i want to stress that i'm not challenging the person, just the text. > The governing body would not just be there to arbitrate, advocate, or negotiate. They would also be there for promoting training and sharing of knowledge. this reads like duct-tape solutions. it's always the case that a governing body grows and grows and gets in the way. "it's not just there to XYZ, it also BLAHBLAHBLAH". a governing body is one possible solution to the problem of people not understanding ethics (or how to figure out what ethics are), that I don't think will solve the larger problem. a governing body will need to be run by practitioners of governance, and enforced by practitioners of enforcement. ethics needs to be practiced everywhere, all the time. and a governing body won't make that happen. people need to be learning about why their activities are "good" or "bad" or "questionable". in my (simplified version) view: we're not lacking incentive to work (money!), we're lacking incentive to not work (people shouting at us for being assholes for doing asshole work [e.g. bombs, baby killing, evil algos]). we need to cultivate behaviors of ethical culture... instead we're cultivating behaviors that promote short-term-ism (buy your way to happiness, bomb your way to safety, build your dream home on loan!, opportunity cost services saving us from having to cleaning up after ourselves, trash removal services that vanish things away to faraway dumps we never think of)