5 ms·

Software people not philosophers promulgated the nonsense idea that "algorithms can't be biased", so I think software people need plenty of help understanding w

Software people not philosophers promulgated the nonsense idea that "algorithms can't be biased", so I think software people need plenty of help understanding what an algorithm is, with help from philosophy and humanity. Software, being mostly based on math is a prime example of "map that is not the territory".

throwawaygh 6y ago
> Software people not philosophers promulgated the nonsense idea that "algorithms can't be biased", so I think software people need plenty of help understanding what an algorithm is, with help from philosophy and humanity. (E: throwaway894345's comment is of much higher quality than mine.) Very few serious software engineers -- and virtually zero computer scientists -- have ever held this opinion. It's an extreme minority view. And, I know a few philosophers who also repeat this nonsense. Those that do hold this view IME are almost always folks who are invested in a personal politics that is congruent with dismissing concerns about bias. Without exception, IME, they're also knee-jerk antagonistic toward other ideas about bias that don't involve computers (eg they're the first ones to complain about implicit bias training). These people also tend to be the annoyingly pedantic type ("bias has a technical meaning in statistics!" sort of thing). In other words, they're politically invested, and no amount of input from philosophers -- or technicians/scientists -- is going to change their strongly held personal politic beliefs. Again, anyone who has spent a moment around philosophers knows that they are just humans who are not immune from the exact same problem.
- jimktrains2 6y ago
  > Very few serious software engineers -- and virtually zero computer scientists -- have ever held this opinion. It's an extreme minority view. And yet here we are selling software, like facial recognition, to organizations that can ruin people and have little oversight based on how well those same algorithms are supposed to work?
  - throwawaygh 6y ago
    > And yet here we are selling software, like facial recognition Speak for yourself. (Also, last I checked, most CEOs, boards, and sales forces don't have technical backgrounds.) > to organizations that can ruin people and have little oversight based on how well those same algorithms are supposed to work? And how do you get oversight? That's a political problem, not a philosophical problem. If you want to sway the opinions of regulators and legislators, please do not reach for the philosophy dept. Convincing regulators and legislators to do stuff is not the job of a philosopher. So what is the job of a philosopher? This kind of brings us back to my original point from which we've now strayed. Philosophy is a field of study. The job of a philosopher is to do philosophy. In order to do that job well, philosophers require more input from technical experts than they currently receive. This matters precisely because philosophy is important; good philosophy is the lifeblood of a healthy society, but it's increasingly difficult to do good philosophy without some scientific training.
    - jimktrains2 6y ago
      I do not sell such software. You blame companies, but is that to mean that engineers are supposed to turn a blind eye to ethical issues in the products their employers sell? Other engineering disciplins whole-heartedly disagree. Regulation is besides the point. The comment I replied to said that software engineers have never held the belief that their algorithms aren't biased and I was simply pointing out that that is either false on a large scale or those engineers are complicit with the use of their software in unethical ways because of those bias issues in the software.
      - throwawaygh 6y ago
        > but is that to mean that engineers are supposed to turn a blind eye to ethical issues in the products their employers sell? Those engineers understand the obvious problems with facial recognition software. Everyone does. The engineers building this software don't need philosophical training. Instead, they need a personal/professional ethics (or, in lieu of such self-regulatory instincts, government regulation). Again, neither of those presents a philosophical problem. > Other engineering disciplins whole-heartedly disagree. Regulation is besides the point. I disagree. Other engineers disagree precisely because they operate in highly regulated disciplines. PE codes of ethics are not (just) personal ethical commitments; rather, they are (also) backed by the force of law. BTW, lots of engineers build WMDs.
- throwaway894345 6y ago
  To add more context to your point, originally the "algorithms are biased" people were arguing (or strongly implying) that all algorithms were biased and that they would always be biased so we should never use them (or something to that end, anyway). I don't know that their opponents were arguing that algorithms "aren't biased" as much as for a clearer understanding of what is meant by the claim "algorithms are biased" because there's no obvious definition for "bias" that lends itself to the conclusion that algorithms will always be more bias (or "worsely biased" in case that's not the same as "more bias") than people. The latter camp specifically argued that algorithms aren't inherently biased, but rather they "inherit" any biases encoded in the training data, and thus the "algorithms are biased" group would do well to frame their concerns in terms of concerns about training data sets and focus on how we can mitigate those concerns. Over time, as is often the case with healthy debate, the "algorithms are biased" people did start to frame their concerns this way and now we have robust and productive conversations about ethically training algorithms (instead of a strict prohibition on algorithms). This is strictly a good outcome, except that some people are misremembering the argument as one group arguing that "algorithms can't be biased" in any sense of the phrase (although I'm sure there are some fringe few people for whom that's true as well, but as you point out, that was never a prominent position) and we forget that the "algorithms are biased" people weren't always making the nuanced arguments that they've been pushed to make today.
  - throwawaygh 6y ago
    Excellent clarification; thanks for taking the time to write this up!
true_religion 6y ago
Algorithms are just encodings of their methodology of their creators. Here's an algorithm to detect foreigners in my country: Step 1: Calculate skin tone ratio of person Step 2: If light skinned then assign person to bin "foreigner". This will work in 99% of cases, so by one standard its a good algorithm that requires little effort to give valid results. However, it's just encoded the bias that light skinned people can't be citizens, and thus fails for mixed children, refugees and the thousands multigenerational dependents of expats. No one can really say computers are unbiased. They simply do what you tell them to do. In my opinion, we computer scientists and practicing engineers understand that, but the general population has the mystique that the machine is somehow more right than a person, simply because you can't see the person(s) who programmed the machine.
- adamsea 6y ago
  Very good point. And, this algorithm would totally fail in my country! As the U.S. has a wide variety of citizens of various skin tones. Which I'm very happy about : ). Which I think supports what you're saying; not only do algorithms have biases, but we also can't assume they are universally applicable across a domain (in this case that a single facial recognition algorithm will have the same degree of accuracy in all places across the globe).
  - true_religion 6y ago
    Yep, I agree. Facial recognition is a field where you commonly see overfitting to culture where the tech package was created in. One dead stupid example, I can think of is biometric cameras being unable to differentiate between faces accurately on the default settings. The cameras are sourced from Europe, and universally use an exposure setting that's too short to do anything more than bring out the whites of someones eyes if you have dark skin tones and are in a normally lit room. Just setting the explore to be a little longer, results in decent results for dark skin tones (and of course, worse results for light skin tones). Consumer technologies like iPhone camera's on the other hand get it right, because they were forced to start testing on different culture groups due to bad press, but their ubiquity leads to people forgetting that every single setting in a camera can lead to better or worse outcomes, and you can't always rely on having a fast processor like an iPhone to autocorrect for you.

throwawaygh 6y ago

> Software people not philosophers promulgated the nonsense idea that "algorithms can't be biased", so I think software people need plenty of help understanding what an algorithm is, with help from philosophy and humanity. (E: throwaway894345's comment is of much higher quality than mine.) Very few serious software engineers -- and virtually zero computer scientists -- have ever held this opinion. It's an extreme minority view. And, I know a few philosophers who also repeat this nonsense. Those that do hold this view IME are almost always folks who are invested in a personal politics that is congruent with dismissing concerns about bias. Without exception, IME, they're also knee-jerk antagonistic toward other ideas about bias that don't involve computers (eg they're the first ones to complain about implicit bias training). These people also tend to be the annoyingly pedantic type ("bias has a technical meaning in statistics!" sort of thing). In other words, they're politically invested, and no amount of input from philosophers -- or technicians/scientists -- is going to change their strongly held personal politic beliefs. Again, anyone who has spent a moment around philosophers knows that they are just humans who are not immune from the exact same problem.

jimktrains2 6y ago
> Very few serious software engineers -- and virtually zero computer scientists -- have ever held this opinion. It's an extreme minority view. And yet here we are selling software, like facial recognition, to organizations that can ruin people and have little oversight based on how well those same algorithms are supposed to work?
- throwawaygh 6y ago
  > And yet here we are selling software, like facial recognition Speak for yourself. (Also, last I checked, most CEOs, boards, and sales forces don't have technical backgrounds.) > to organizations that can ruin people and have little oversight based on how well those same algorithms are supposed to work? And how do you get oversight? That's a political problem, not a philosophical problem. If you want to sway the opinions of regulators and legislators, please do not reach for the philosophy dept. Convincing regulators and legislators to do stuff is not the job of a philosopher. So what is the job of a philosopher? This kind of brings us back to my original point from which we've now strayed. Philosophy is a field of study. The job of a philosopher is to do philosophy. In order to do that job well, philosophers require more input from technical experts than they currently receive. This matters precisely because philosophy is important; good philosophy is the lifeblood of a healthy society, but it's increasingly difficult to do good philosophy without some scientific training.
  - jimktrains2 6y ago
    I do not sell such software. You blame companies, but is that to mean that engineers are supposed to turn a blind eye to ethical issues in the products their employers sell? Other engineering disciplins whole-heartedly disagree. Regulation is besides the point. The comment I replied to said that software engineers have never held the belief that their algorithms aren't biased and I was simply pointing out that that is either false on a large scale or those engineers are complicit with the use of their software in unethical ways because of those bias issues in the software.
    - throwawaygh 6y ago
      > but is that to mean that engineers are supposed to turn a blind eye to ethical issues in the products their employers sell? Those engineers understand the obvious problems with facial recognition software. Everyone does. The engineers building this software don't need philosophical training. Instead, they need a personal/professional ethics (or, in lieu of such self-regulatory instincts, government regulation). Again, neither of those presents a philosophical problem. > Other engineering disciplins whole-heartedly disagree. Regulation is besides the point. I disagree. Other engineers disagree precisely because they operate in highly regulated disciplines. PE codes of ethics are not (just) personal ethical commitments; rather, they are (also) backed by the force of law. BTW, lots of engineers build WMDs.
throwaway894345 6y ago
To add more context to your point, originally the "algorithms are biased" people were arguing (or strongly implying) that all algorithms were biased and that they would always be biased so we should never use them (or something to that end, anyway). I don't know that their opponents were arguing that algorithms "aren't biased" as much as for a clearer understanding of what is meant by the claim "algorithms are biased" because there's no obvious definition for "bias" that lends itself to the conclusion that algorithms will always be more bias (or "worsely biased" in case that's not the same as "more bias") than people. The latter camp specifically argued that algorithms aren't inherently biased, but rather they "inherit" any biases encoded in the training data, and thus the "algorithms are biased" group would do well to frame their concerns in terms of concerns about training data sets and focus on how we can mitigate those concerns. Over time, as is often the case with healthy debate, the "algorithms are biased" people did start to frame their concerns this way and now we have robust and productive conversations about ethically training algorithms (instead of a strict prohibition on algorithms). This is strictly a good outcome, except that some people are misremembering the argument as one group arguing that "algorithms can't be biased" in any sense of the phrase (although I'm sure there are some fringe few people for whom that's true as well, but as you point out, that was never a prominent position) and we forget that the "algorithms are biased" people weren't always making the nuanced arguments that they've been pushed to make today.
- throwawaygh 6y ago
  Excellent clarification; thanks for taking the time to write this up!

true_religion 6y ago

Algorithms are just encodings of their methodology of their creators. Here's an algorithm to detect foreigners in my country: Step 1: Calculate skin tone ratio of person Step 2: If light skinned then assign person to bin "foreigner". This will work in 99% of cases, so by one standard its a good algorithm that requires little effort to give valid results. However, it's just encoded the bias that light skinned people can't be citizens, and thus fails for mixed children, refugees and the thousands multigenerational dependents of expats. No one can really say computers are unbiased. They simply do what you tell them to do. In my opinion, we computer scientists and practicing engineers understand that, but the general population has the mystique that the machine is somehow more right than a person, simply because you can't see the person(s) who programmed the machine.

adamsea 6y ago
Very good point. And, this algorithm would totally fail in my country! As the U.S. has a wide variety of citizens of various skin tones. Which I'm very happy about : ). Which I think supports what you're saying; not only do algorithms have biases, but we also can't assume they are universally applicable across a domain (in this case that a single facial recognition algorithm will have the same degree of accuracy in all places across the globe).
- true_religion 6y ago
  Yep, I agree. Facial recognition is a field where you commonly see overfitting to culture where the tech package was created in. One dead stupid example, I can think of is biometric cameras being unable to differentiate between faces accurately on the default settings. The cameras are sourced from Europe, and universally use an exposure setting that's too short to do anything more than bring out the whites of someones eyes if you have dark skin tones and are in a normally lit room. Just setting the explore to be a little longer, results in decent results for dark skin tones (and of course, worse results for light skin tones). Consumer technologies like iPhone camera's on the other hand get it right, because they were forced to start testing on different culture groups due to bad press, but their ubiquity leads to people forgetting that every single setting in a camera can lead to better or worse outcomes, and you can't always rely on having a fast processor like an iPhone to autocorrect for you.