2 ms·
That's not how any real-world algorithms work (and maybe this is exactly your point); rather, provided they learned on a representative sample, they'd learn tha
by throwaway894345 6y ago
That's not how any real-world algorithms work (and maybe this is exactly your point); rather, provided they learned on a representative sample, they'd learn that there's a 99% chance that a given light-skinned person is a non-citizen (or more likely, that the lighter the skin, the less likely they are to be a citizen--i.e., continuous and not discrete). For a given light-skinned person, they'd return a confidence of 99% that the person in question is a non-citizen. "Bias" would be a person looking at that confidence interval and misinterpreting it as 100%.
Additionally, it's problematic because it's easily conned--for example, if it's used to determine who is eligible to vote (even if we take care not to round the confidence interval up to 100% by classifying 1% of light-skinned people as eligible and 1% of dark-skinned people as ineligible), someone could haul in a bunch of foreigners with dark skin to commit voter fraud. This also isn't "algorithmic bias", but rather foolishness in the application of the algorithm (of course, it's impossible to conceive of a legitimate purpose for an algorithm that maps statistically between skin color and citizenship).
- true_religion 6y ago> provided they learned on a representative sample From this, I think you're talking about deep learning algorithms. However, the term algorithm predates practical AI, deep learning, neural nets, etc. I would know because I sat for an algorithms exam when the fastest consumer computers had speeds measured in single-digit MHz. An algorithm is essentially just a series of steps that a person (or more commonly now adays, a computer) can follow along with to achieve a specific goal. Talking about stochastic systems, back propagation, or even simple probability and confidence intervals just muddies the issue. At its core, algorithms can merely be something defined deterministically, and programmed by hand by a human being. By the way, the system I described isn't something in actual use by machines (I hope). It's just the thought pattern of biased people in my country, a sort of mental short cut used to determine what level of service you get.
- throwaway894345 6y agoI thought we were talking about machine algorithms, but it doesn't really matter. We're talking about a simple statistical mapping between two boolean properties: skin tone and citizenship. You stipulated already that, for a given light-skinned person, the algorithm (machine or human) will return "non-citizen" with 99% confidence, and that that confidence interval is correct. As far as I can tell, we can conclude from those facts that there is no bias (maybe I'm wrong, but I consider bias to be an errant belief); however, there are still potential issues: * For what applications is 99% confidence acceptable? Can we do better by avoiding a statistical analysis altogether, e.g., by using citizenship records? * Do we really believe/trust that humans are capable of 99% confidence (calling into question the fact that you established in your original post)? * Are users of this algorithm rounding 99% up to 100%, thus disenfranchising one group 1% of the time (as opposed to distributing the error proportionately between the two groups)?