5 ms·

> Software people not philosophers promulgated the nonsense idea that "algorithms can't be biased", so I think software people need plenty of help understanding

> Software people not philosophers promulgated the nonsense idea that "algorithms can't be biased", so I think software people need plenty of help understanding what an algorithm is, with help from philosophy and humanity. (E: throwaway894345's comment is of much higher quality than mine.) Very few serious software engineers -- and virtually zero computer scientists -- have ever held this opinion. It's an extreme minority view. And, I know a few philosophers who also repeat this nonsense. Those that do hold this view IME are almost always folks who are invested in a personal politics that is congruent with dismissing concerns about bias. Without exception, IME, they're also knee-jerk antagonistic toward other ideas about bias that don't involve computers (eg they're the first ones to complain about implicit bias training). These people also tend to be the annoyingly pedantic type ("bias has a technical meaning in statistics!" sort of thing). In other words, they're politically invested, and no amount of input from philosophers -- or technicians/scientists -- is going to change their strongly held personal politic beliefs. Again, anyone who has spent a moment around philosophers knows that they are just humans who are not immune from the exact same problem.

jimktrains2 6y ago
> Very few serious software engineers -- and virtually zero computer scientists -- have ever held this opinion. It's an extreme minority view. And yet here we are selling software, like facial recognition, to organizations that can ruin people and have little oversight based on how well those same algorithms are supposed to work?
- throwawaygh 6y ago
  > And yet here we are selling software, like facial recognition Speak for yourself. (Also, last I checked, most CEOs, boards, and sales forces don't have technical backgrounds.) > to organizations that can ruin people and have little oversight based on how well those same algorithms are supposed to work? And how do you get oversight? That's a political problem, not a philosophical problem. If you want to sway the opinions of regulators and legislators, please do not reach for the philosophy dept. Convincing regulators and legislators to do stuff is not the job of a philosopher. So what is the job of a philosopher? This kind of brings us back to my original point from which we've now strayed. Philosophy is a field of study. The job of a philosopher is to do philosophy. In order to do that job well, philosophers require more input from technical experts than they currently receive. This matters precisely because philosophy is important; good philosophy is the lifeblood of a healthy society, but it's increasingly difficult to do good philosophy without some scientific training.
  - jimktrains2 6y ago
    I do not sell such software. You blame companies, but is that to mean that engineers are supposed to turn a blind eye to ethical issues in the products their employers sell? Other engineering disciplins whole-heartedly disagree. Regulation is besides the point. The comment I replied to said that software engineers have never held the belief that their algorithms aren't biased and I was simply pointing out that that is either false on a large scale or those engineers are complicit with the use of their software in unethical ways because of those bias issues in the software.
    - throwawaygh 6y ago
      > but is that to mean that engineers are supposed to turn a blind eye to ethical issues in the products their employers sell? Those engineers understand the obvious problems with facial recognition software. Everyone does. The engineers building this software don't need philosophical training. Instead, they need a personal/professional ethics (or, in lieu of such self-regulatory instincts, government regulation). Again, neither of those presents a philosophical problem. > Other engineering disciplins whole-heartedly disagree. Regulation is besides the point. I disagree. Other engineers disagree precisely because they operate in highly regulated disciplines. PE codes of ethics are not (just) personal ethical commitments; rather, they are (also) backed by the force of law. BTW, lots of engineers build WMDs.
    - throwaway894345 6y ago
      I think it's helpful to talk about "bias issues" specifically. When we talk about them in the abstract, they always seem to carry this boogeyman-ish quality, but when we talk about them as concrete instances, we can almost always (and rather easily) come up with concrete mitigations. For example, some people have rightly expressed concern about statistical models of criminality derived from law enforcement data, so we augment and weigh that data with other sources, such as victim surveys, hospital records, and so forth. Of course, that doesn't eliminate all bias (we can never completely eliminate bias) but it's a significant improvement, especially when the alternative to "using a formal statistical model" is "using law enforcement officers' personal judgment". We can also make available certain information about the training data set and the methodology to provide an audit trail. Very rarely are these ethical issues really as insurmountable as critics seem to imply when they speak about "biased algorithms" in the abstract.
- deleted 6y ago
  [deleted]
- YawningAngel 6y ago
  We don't sell that software because it is the consensus of those qualified to judge that it's a good idea, we sell it because we live in a market economy, it is legal, and institutions want to buy it. The problem here is with the law and the institutions.
  - jimktrains2 6y ago
    You blame companies and the market economy, but is that to mean that engineers are supposed to turn a blind eye to ethical issues in the products their employers sell? Other engineering disciplins whole-heartedly disagree. My point is that either the statement that it's widely held that software can be biased is wrong (or more generously that the concern is not considered), or software engineers are complicit in unethical uses of their software because of known biases.
    - YawningAngel 6y ago
      I think individual engineers absolutely have a duty to act ethically, and that that duty includes not working on unethical things. However, I don't think that stating that the behaviour is unethical and that people shouldn't do it helps solve the problem, whereas I do think that we could ban bullshit ML woo or reform our institutions so that they don't use it to spy on people
throwaway894345 6y ago
To add more context to your point, originally the "algorithms are biased" people were arguing (or strongly implying) that all algorithms were biased and that they would always be biased so we should never use them (or something to that end, anyway). I don't know that their opponents were arguing that algorithms "aren't biased" as much as for a clearer understanding of what is meant by the claim "algorithms are biased" because there's no obvious definition for "bias" that lends itself to the conclusion that algorithms will always be more bias (or "worsely biased" in case that's not the same as "more bias") than people. The latter camp specifically argued that algorithms aren't inherently biased, but rather they "inherit" any biases encoded in the training data, and thus the "algorithms are biased" group would do well to frame their concerns in terms of concerns about training data sets and focus on how we can mitigate those concerns. Over time, as is often the case with healthy debate, the "algorithms are biased" people did start to frame their concerns this way and now we have robust and productive conversations about ethically training algorithms (instead of a strict prohibition on algorithms). This is strictly a good outcome, except that some people are misremembering the argument as one group arguing that "algorithms can't be biased" in any sense of the phrase (although I'm sure there are some fringe few people for whom that's true as well, but as you point out, that was never a prominent position) and we forget that the "algorithms are biased" people weren't always making the nuanced arguments that they've been pushed to make today.
- throwawaygh 6y ago
  Excellent clarification; thanks for taking the time to write this up!

jimktrains2 6y ago

> Very few serious software engineers -- and virtually zero computer scientists -- have ever held this opinion. It's an extreme minority view. And yet here we are selling software, like facial recognition, to organizations that can ruin people and have little oversight based on how well those same algorithms are supposed to work?

throwawaygh 6y ago
> And yet here we are selling software, like facial recognition Speak for yourself. (Also, last I checked, most CEOs, boards, and sales forces don't have technical backgrounds.) > to organizations that can ruin people and have little oversight based on how well those same algorithms are supposed to work? And how do you get oversight? That's a political problem, not a philosophical problem. If you want to sway the opinions of regulators and legislators, please do not reach for the philosophy dept. Convincing regulators and legislators to do stuff is not the job of a philosopher. So what is the job of a philosopher? This kind of brings us back to my original point from which we've now strayed. Philosophy is a field of study. The job of a philosopher is to do philosophy. In order to do that job well, philosophers require more input from technical experts than they currently receive. This matters precisely because philosophy is important; good philosophy is the lifeblood of a healthy society, but it's increasingly difficult to do good philosophy without some scientific training.
- jimktrains2 6y ago
  I do not sell such software. You blame companies, but is that to mean that engineers are supposed to turn a blind eye to ethical issues in the products their employers sell? Other engineering disciplins whole-heartedly disagree. Regulation is besides the point. The comment I replied to said that software engineers have never held the belief that their algorithms aren't biased and I was simply pointing out that that is either false on a large scale or those engineers are complicit with the use of their software in unethical ways because of those bias issues in the software.
  - throwawaygh 6y ago
    > but is that to mean that engineers are supposed to turn a blind eye to ethical issues in the products their employers sell? Those engineers understand the obvious problems with facial recognition software. Everyone does. The engineers building this software don't need philosophical training. Instead, they need a personal/professional ethics (or, in lieu of such self-regulatory instincts, government regulation). Again, neither of those presents a philosophical problem. > Other engineering disciplins whole-heartedly disagree. Regulation is besides the point. I disagree. Other engineers disagree precisely because they operate in highly regulated disciplines. PE codes of ethics are not (just) personal ethical commitments; rather, they are (also) backed by the force of law. BTW, lots of engineers build WMDs.
  - throwaway894345 6y ago
    I think it's helpful to talk about "bias issues" specifically. When we talk about them in the abstract, they always seem to carry this boogeyman-ish quality, but when we talk about them as concrete instances, we can almost always (and rather easily) come up with concrete mitigations. For example, some people have rightly expressed concern about statistical models of criminality derived from law enforcement data, so we augment and weigh that data with other sources, such as victim surveys, hospital records, and so forth. Of course, that doesn't eliminate all bias (we can never completely eliminate bias) but it's a significant improvement, especially when the alternative to "using a formal statistical model" is "using law enforcement officers' personal judgment". We can also make available certain information about the training data set and the methodology to provide an audit trail. Very rarely are these ethical issues really as insurmountable as critics seem to imply when they speak about "biased algorithms" in the abstract.
deleted 6y ago
[deleted]
YawningAngel 6y ago
We don't sell that software because it is the consensus of those qualified to judge that it's a good idea, we sell it because we live in a market economy, it is legal, and institutions want to buy it. The problem here is with the law and the institutions.
- jimktrains2 6y ago
  You blame companies and the market economy, but is that to mean that engineers are supposed to turn a blind eye to ethical issues in the products their employers sell? Other engineering disciplins whole-heartedly disagree. My point is that either the statement that it's widely held that software can be biased is wrong (or more generously that the concern is not considered), or software engineers are complicit in unethical uses of their software because of known biases.
  - YawningAngel 6y ago
    I think individual engineers absolutely have a duty to act ethically, and that that duty includes not working on unethical things. However, I don't think that stating that the behaviour is unethical and that people shouldn't do it helps solve the problem, whereas I do think that we could ban bullshit ML woo or reform our institutions so that they don't use it to spy on people

throwaway894345 6y ago

To add more context to your point, originally the "algorithms are biased" people were arguing (or strongly implying) that all algorithms were biased and that they would always be biased so we should never use them (or something to that end, anyway). I don't know that their opponents were arguing that algorithms "aren't biased" as much as for a clearer understanding of what is meant by the claim "algorithms are biased" because there's no obvious definition for "bias" that lends itself to the conclusion that algorithms will always be more bias (or "worsely biased" in case that's not the same as "more bias") than people. The latter camp specifically argued that algorithms aren't inherently biased, but rather they "inherit" any biases encoded in the training data, and thus the "algorithms are biased" group would do well to frame their concerns in terms of concerns about training data sets and focus on how we can mitigate those concerns. Over time, as is often the case with healthy debate, the "algorithms are biased" people did start to frame their concerns this way and now we have robust and productive conversations about ethically training algorithms (instead of a strict prohibition on algorithms). This is strictly a good outcome, except that some people are misremembering the argument as one group arguing that "algorithms can't be biased" in any sense of the phrase (although I'm sure there are some fringe few people for whom that's true as well, but as you point out, that was never a prominent position) and we forget that the "algorithms are biased" people weren't always making the nuanced arguments that they've been pushed to make today.

throwawaygh 6y ago
Excellent clarification; thanks for taking the time to write this up!