3 ms·

Algorithms are just encodings of their methodology of their creators. Here's an algorithm to detect foreigners in my country: Step 1: Calculate skin tone ratio

Algorithms are just encodings of their methodology of their creators. Here's an algorithm to detect foreigners in my country: Step 1: Calculate skin tone ratio of person Step 2: If light skinned then assign person to bin "foreigner". This will work in 99% of cases, so by one standard its a good algorithm that requires little effort to give valid results. However, it's just encoded the bias that light skinned people can't be citizens, and thus fails for mixed children, refugees and the thousands multigenerational dependents of expats. No one can really say computers are unbiased. They simply do what you tell them to do. In my opinion, we computer scientists and practicing engineers understand that, but the general population has the mystique that the machine is somehow more right than a person, simply because you can't see the person(s) who programmed the machine.

adamsea 6y ago
Very good point. And, this algorithm would totally fail in my country! As the U.S. has a wide variety of citizens of various skin tones. Which I'm very happy about : ). Which I think supports what you're saying; not only do algorithms have biases, but we also can't assume they are universally applicable across a domain (in this case that a single facial recognition algorithm will have the same degree of accuracy in all places across the globe).
- true_religion 6y ago
  Yep, I agree. Facial recognition is a field where you commonly see overfitting to culture where the tech package was created in. One dead stupid example, I can think of is biometric cameras being unable to differentiate between faces accurately on the default settings. The cameras are sourced from Europe, and universally use an exposure setting that's too short to do anything more than bring out the whites of someones eyes if you have dark skin tones and are in a normally lit room. Just setting the explore to be a little longer, results in decent results for dark skin tones (and of course, worse results for light skin tones). Consumer technologies like iPhone camera's on the other hand get it right, because they were forced to start testing on different culture groups due to bad press, but their ubiquity leads to people forgetting that every single setting in a camera can lead to better or worse outcomes, and you can't always rely on having a fast processor like an iPhone to autocorrect for you.
throwaway894345 6y ago
That's not how any real-world algorithms work (and maybe this is exactly your point); rather, provided they learned on a representative sample, they'd learn that there's a 99% chance that a given light-skinned person is a non-citizen (or more likely, that the lighter the skin, the less likely they are to be a citizen--i.e., continuous and not discrete). For a given light-skinned person, they'd return a confidence of 99% that the person in question is a non-citizen. "Bias" would be a person looking at that confidence interval and misinterpreting it as 100%. Additionally, it's problematic because it's easily conned--for example, if it's used to determine who is eligible to vote (even if we take care not to round the confidence interval up to 100% by classifying 1% of light-skinned people as eligible and 1% of dark-skinned people as ineligible), someone could haul in a bunch of foreigners with dark skin to commit voter fraud. This also isn't "algorithmic bias", but rather foolishness in the application of the algorithm (of course, it's impossible to conceive of a legitimate purpose for an algorithm that maps statistically between skin color and citizenship).
- true_religion 6y ago
  > provided they learned on a representative sample From this, I think you're talking about deep learning algorithms. However, the term algorithm predates practical AI, deep learning, neural nets, etc. I would know because I sat for an algorithms exam when the fastest consumer computers had speeds measured in single-digit MHz. An algorithm is essentially just a series of steps that a person (or more commonly now adays, a computer) can follow along with to achieve a specific goal. Talking about stochastic systems, back propagation, or even simple probability and confidence intervals just muddies the issue. At its core, algorithms can merely be something defined deterministically, and programmed by hand by a human being. By the way, the system I described isn't something in actual use by machines (I hope). It's just the thought pattern of biased people in my country, a sort of mental short cut used to determine what level of service you get.
  - throwaway894345 6y ago
    I thought we were talking about machine algorithms, but it doesn't really matter. We're talking about a simple statistical mapping between two boolean properties: skin tone and citizenship. You stipulated already that, for a given light-skinned person, the algorithm (machine or human) will return "non-citizen" with 99% confidence, and that that confidence interval is correct. As far as I can tell, we can conclude from those facts that there is no bias (maybe I'm wrong, but I consider bias to be an errant belief); however, there are still potential issues: * For what applications is 99% confidence acceptable? Can we do better by avoiding a statistical analysis altogether, e.g., by using citizenship records? * Do we really believe/trust that humans are capable of 99% confidence (calling into question the fact that you established in your original post)? * Are users of this algorithm rounding 99% up to 100%, thus disenfranchising one group 1% of the time (as opposed to distributing the error proportionately between the two groups)?

adamsea 6y ago

Very good point. And, this algorithm would totally fail in my country! As the U.S. has a wide variety of citizens of various skin tones. Which I'm very happy about : ). Which I think supports what you're saying; not only do algorithms have biases, but we also can't assume they are universally applicable across a domain (in this case that a single facial recognition algorithm will have the same degree of accuracy in all places across the globe).

true_religion 6y ago
Yep, I agree. Facial recognition is a field where you commonly see overfitting to culture where the tech package was created in. One dead stupid example, I can think of is biometric cameras being unable to differentiate between faces accurately on the default settings. The cameras are sourced from Europe, and universally use an exposure setting that's too short to do anything more than bring out the whites of someones eyes if you have dark skin tones and are in a normally lit room. Just setting the explore to be a little longer, results in decent results for dark skin tones (and of course, worse results for light skin tones). Consumer technologies like iPhone camera's on the other hand get it right, because they were forced to start testing on different culture groups due to bad press, but their ubiquity leads to people forgetting that every single setting in a camera can lead to better or worse outcomes, and you can't always rely on having a fast processor like an iPhone to autocorrect for you.

throwaway894345 6y ago

That's not how any real-world algorithms work (and maybe this is exactly your point); rather, provided they learned on a representative sample, they'd learn that there's a 99% chance that a given light-skinned person is a non-citizen (or more likely, that the lighter the skin, the less likely they are to be a citizen--i.e., continuous and not discrete). For a given light-skinned person, they'd return a confidence of 99% that the person in question is a non-citizen. "Bias" would be a person looking at that confidence interval and misinterpreting it as 100%. Additionally, it's problematic because it's easily conned--for example, if it's used to determine who is eligible to vote (even if we take care not to round the confidence interval up to 100% by classifying 1% of light-skinned people as eligible and 1% of dark-skinned people as ineligible), someone could haul in a bunch of foreigners with dark skin to commit voter fraud. This also isn't "algorithmic bias", but rather foolishness in the application of the algorithm (of course, it's impossible to conceive of a legitimate purpose for an algorithm that maps statistically between skin color and citizenship).

true_religion 6y ago
> provided they learned on a representative sample From this, I think you're talking about deep learning algorithms. However, the term algorithm predates practical AI, deep learning, neural nets, etc. I would know because I sat for an algorithms exam when the fastest consumer computers had speeds measured in single-digit MHz. An algorithm is essentially just a series of steps that a person (or more commonly now adays, a computer) can follow along with to achieve a specific goal. Talking about stochastic systems, back propagation, or even simple probability and confidence intervals just muddies the issue. At its core, algorithms can merely be something defined deterministically, and programmed by hand by a human being. By the way, the system I described isn't something in actual use by machines (I hope). It's just the thought pattern of biased people in my country, a sort of mental short cut used to determine what level of service you get.
- throwaway894345 6y ago
  I thought we were talking about machine algorithms, but it doesn't really matter. We're talking about a simple statistical mapping between two boolean properties: skin tone and citizenship. You stipulated already that, for a given light-skinned person, the algorithm (machine or human) will return "non-citizen" with 99% confidence, and that that confidence interval is correct. As far as I can tell, we can conclude from those facts that there is no bias (maybe I'm wrong, but I consider bias to be an errant belief); however, there are still potential issues: * For what applications is 99% confidence acceptable? Can we do better by avoiding a statistical analysis altogether, e.g., by using citizenship records? * Do we really believe/trust that humans are capable of 99% confidence (calling into question the fact that you established in your original post)? * Are users of this algorithm rounding 99% up to 100%, thus disenfranchising one group 1% of the time (as opposed to distributing the error proportionately between the two groups)?