4 ms·

> That's the entire point of the article, our biases, as reflected in the world, are learned by models. They are not biased if they represent reality based on

> That's the entire point of the article, our biases, as reflected in the world, are learned by models. They are not biased if they represent reality based on facts. In fact, the main issue is that while these algorithms fit reality as it actually is, the critics crying "bias" are complaining that these algorithms don't fit what they believe reality should be. In other words, the algorithms output predictions that matches reality but critics argue that they should instead be manipulated to output results that suits their ideals. But that has nothing to do with having a bias.

FisDugthop 7y ago
For example, the model under discussion, which can tell the difference between men and women and believes in differences between them, probably doesn't know that intersex people exist. It probably cannot learn such a distinction, either. Therefore this algorithm probably doesn't fit what we know reality to resemble. So there's at least two sorts of biases. There's both biases inside the model, which is what you're thinking of when you use the word "bias", and also biases outside the model, which inform the model's design and construction in ways that cannot even be quantified with only the model's metrics.
- rumanator 7y ago
  > For example, the model under discussion, which can tell the difference between men and women and believes in differences between them, probably doesn't know that intersex people exist. Considering that the frequency of intersex people in a population lies somewhere between 0.05% and 0.07%, not accounting for a secondary trait such as whether a biological M or F happens to be intersex is an irrelevant classification error that has a negligible (if any) impact on a model's predictive ability. And by the way, any model can be regenerated if any attribute left out is found to be significant.
ncallaway 7y ago
The post you are replying to has three clear bullet points which indicate where the problem lies. It also directly addresses the point you're making about accurate depictions of reality. Would you mind responding to the third bullet point from the parent post, that discusses the dangers of naive and biased applications of facts that are based on reality?
- rumanator 7y ago
  > The post you are replying to has three clear bullet points which indicate where the problem lies. No, it does not. All it does is point possible explanations of why reality is the way it is. Yet, reality is still reality. If models are expected to predict reality then their output will match what we observe in reality. If someone expects predictive models to not predict reality and instead output results that do not match reality and instead comply with someone's personal ideals then that's an entirely different problem: predicting someone's personal goals instead of reality.
  - ncallaway 7y ago
    The third bullet point of the above question (which you still haven't addressed) directly addresses this. The problem is not that the model outputs reality. The problem is that the model _incorrectly_ produces not reality, based on facts it _did_ learn from reality. That is, there is a fact pattern in reality, but the model draws an illogical conclusion based on that fact pattern. So, I'll ask again. Could you directly address this point from the parent post: "A recruiting agency uses a predictive model to recruit computer programmers. Due to the way the model was trained, it excludes qualified women (not OK, a clear misapplication of an algorithm)"
    - thereisnospork 7y ago
      > "A recruiting agency uses a predictive model to recruit computer programmers. Due to the way the model was trained, it excludes qualified women (not OK, a clear misapplication of an algorithm)" Philosophically wrong or not, perhaps excluding those hypothetical women from consideration is the quote unquote optimal approach to allocate recruiting resources. Why this is the case is irrelevant; the onus is on society to 'shape up' and remove the reasons the group 'women'[1] are disqualifying themselves from the fair model. Of course assuming good faith in model design, training, implementation, etc. [1] Ditto for men as kindergarten teachers - employment (e.g.) being a zero(ish) sum game, so long as more women are teachers, some other profession will contain an outsized amount of men (and vice versa). There are reasons[2] why more women are kindergarten teachers than men, and any fair algorithm and effective algorithm would be expected to make prediction on these reasons or proxies thereof. [2] Or there would be an even gender ratio.
StuffedParrot 7y ago
Mind giving an example of something that could either be bias or fact?
- rumanator 7y ago
  The definition of bias used in Machine Learning is quite clear, as is the definition of training, validation and test data sets.
  - StuffedParrot 7y ago
    So-you do mind giving an example. Telling. Never trust someone who says they deal only with facts.
tomrod 7y ago
Yes, it is explicitly statistical bias, and it comes from extrapolating a system of equations from single point observable outcomes, a situation Economics tends to solve with instrumental variables. If I can, let me explain using basic quantity supplied and demanded. Consider the supply equation: Q_s = a + bP + cX, and the demand equation, Q_d = e - fP + gZ. Data only observes where Q_s = Q_d given P, X, and Z. So what most ML models attempt to do is predict Q as a single fit function of P, X, and Z -- not entirely unreasonable, but misses the structure of the full system. So two things to consider: (1) Implementing a rule policy based on "measuring as it actually is" may result in a feedback cycle (personally I think there may be one in common real estate property valuation "AVM" models). By recruiters relying on a biased model the class imbalance may actually increase over doing nothing. (2) Normative ideals, like being blind to anything but skill and opportunity or equalizing background (two common political ideals in the US) cannot be appropriately evaluated. This is Judea Pearl's claim of Pr(Y|X) versus Pr(Y|do(X)).
- MayeulC 7y ago
  I wonder if a Bayesian approach couldn't be taken to try to remain free of that bias, since it's basically designed for this (a system you can only observe part of).
  - tomrod 7y ago
    Worth considering: https://www.tandfonline.com/doi/abs/10.1080/07474938.2013.807146 https://www.tandfonline.com/doi/abs/10.1080/07474938.2013.80...

FisDugthop 7y ago

For example, the model under discussion, which can tell the difference between men and women and believes in differences between them, probably doesn't know that intersex people exist. It probably cannot learn such a distinction, either. Therefore this algorithm probably doesn't fit what we know reality to resemble. So there's at least two sorts of biases. There's both biases inside the model, which is what you're thinking of when you use the word "bias", and also biases outside the model, which inform the model's design and construction in ways that cannot even be quantified with only the model's metrics.

rumanator 7y ago
> For example, the model under discussion, which can tell the difference between men and women and believes in differences between them, probably doesn't know that intersex people exist. Considering that the frequency of intersex people in a population lies somewhere between 0.05% and 0.07%, not accounting for a secondary trait such as whether a biological M or F happens to be intersex is an irrelevant classification error that has a negligible (if any) impact on a model's predictive ability. And by the way, any model can be regenerated if any attribute left out is found to be significant.

ncallaway 7y ago

The post you are replying to has three clear bullet points which indicate where the problem lies. It also directly addresses the point you're making about accurate depictions of reality. Would you mind responding to the third bullet point from the parent post, that discusses the dangers of naive and biased applications of facts that are based on reality?

rumanator 7y ago
> The post you are replying to has three clear bullet points which indicate where the problem lies. No, it does not. All it does is point possible explanations of why reality is the way it is. Yet, reality is still reality. If models are expected to predict reality then their output will match what we observe in reality. If someone expects predictive models to not predict reality and instead output results that do not match reality and instead comply with someone's personal ideals then that's an entirely different problem: predicting someone's personal goals instead of reality.
- ncallaway 7y ago
  The third bullet point of the above question (which you still haven't addressed) directly addresses this. The problem is not that the model outputs reality. The problem is that the model _incorrectly_ produces not reality, based on facts it _did_ learn from reality. That is, there is a fact pattern in reality, but the model draws an illogical conclusion based on that fact pattern. So, I'll ask again. Could you directly address this point from the parent post: "A recruiting agency uses a predictive model to recruit computer programmers. Due to the way the model was trained, it excludes qualified women (not OK, a clear misapplication of an algorithm)"
  - thereisnospork 7y ago
    > "A recruiting agency uses a predictive model to recruit computer programmers. Due to the way the model was trained, it excludes qualified women (not OK, a clear misapplication of an algorithm)" Philosophically wrong or not, perhaps excluding those hypothetical women from consideration is the quote unquote optimal approach to allocate recruiting resources. Why this is the case is irrelevant; the onus is on society to 'shape up' and remove the reasons the group 'women'[1] are disqualifying themselves from the fair model. Of course assuming good faith in model design, training, implementation, etc. [1] Ditto for men as kindergarten teachers - employment (e.g.) being a zero(ish) sum game, so long as more women are teachers, some other profession will contain an outsized amount of men (and vice versa). There are reasons[2] why more women are kindergarten teachers than men, and any fair algorithm and effective algorithm would be expected to make prediction on these reasons or proxies thereof. [2] Or there would be an even gender ratio.

StuffedParrot 7y ago

Mind giving an example of something that could either be bias or fact?

rumanator 7y ago
The definition of bias used in Machine Learning is quite clear, as is the definition of training, validation and test data sets.
- StuffedParrot 7y ago
  So-you do mind giving an example. Telling. Never trust someone who says they deal only with facts.

tomrod 7y ago

Yes, it is explicitly statistical bias, and it comes from extrapolating a system of equations from single point observable outcomes, a situation Economics tends to solve with instrumental variables. If I can, let me explain using basic quantity supplied and demanded. Consider the supply equation: Q_s = a + bP + cX, and the demand equation, Q_d = e - fP + gZ. Data only observes where Q_s = Q_d given P, X, and Z. So what most ML models attempt to do is predict Q as a single fit function of P, X, and Z -- not entirely unreasonable, but misses the structure of the full system. So two things to consider: (1) Implementing a rule policy based on "measuring as it actually is" may result in a feedback cycle (personally I think there may be one in common real estate property valuation "AVM" models). By recruiters relying on a biased model the class imbalance may actually increase over doing nothing. (2) Normative ideals, like being blind to anything but skill and opportunity or equalizing background (two common political ideals in the US) cannot be appropriately evaluated. This is Judea Pearl's claim of Pr(Y|X) versus Pr(Y|do(X)).

MayeulC 7y ago
I wonder if a Bayesian approach couldn't be taken to try to remain free of that bias, since it's basically designed for this (a system you can only observe part of).
- tomrod 7y ago
  Worth considering: https://www.tandfonline.com/doi/abs/10.1080/07474938.2013.807146 https://www.tandfonline.com/doi/abs/10.1080/07474938.2013.80...