4 ms·

IQ isn't the point though. I'm not a huge fan of IQ specifically either. Your original objection was intrinsic to the idea of detecting things from faces. These

by darawk 6y ago

IQ isn't the point though. I'm not a huge fan of IQ specifically either. Your original objection was intrinsic to the idea of detecting things from faces. These objections are to the quality of the measures. I'm not going to defend IQ as a useful measure, because it has a ton of problems and that's not my point here. > If the evaluation function is flawed (which is essentially your contention, and which I absolutely agree with), the trained model will exhibit biases that reflect the flaws of the evaluation function. An ML model isn't suddenly going to solve the cultural issues we have with measuring IQ. It will encode the same biases that society does, because the model will try as hard as it can to do exactly what the human proctors would have done. Of course. But so what? There are people using IQ now for things. An ML model isn't going to magically make those biases worse, either. What it's going to do is bring them to the surface, so that they are quantifiable and we can actually do something about them. ML is the solution to the bias problem. Right now these evaluations are being made by other humans. Humans who we cannot statistically debias. Humans who's biases we can't even effectively interrogate. The reason people are making all these memes about biased AI is not that it's more biased than humans, it's that the bias is more measurable.

joshuamorton 6y ago
> An ML model isn't going to magically make those biases worse, either. Well, actually, research shows that unless great care is taken it absolutely can. If you include for example race as a factor in a model, it can learn non-causal correlations between race and whatever the objective is. This can have a compounding effect in some cases. [0] > What it's going to do is bring them to the surface, so that they are quantifiable and we can actually do something about them. I don't follow this. If we have a biased objective function, the model won't surface any biases we weren't already cognizant of in the objective function. And they were already quantifiable: we had a function that we were using to evaluate the model. We could use that same function on whatever non-model evaluation we were doing. > ML is the solution to the bias problem. This is basically directly in contradiction to what leading experts on the subject say. ML cannot fix bias in human systems, unless we presuppose that those systems are biased, in which case we can often address the bias in the human systems directly without ML. > Humans who we cannot statistically debias. Humans who's biases we can't even effectively interrogate. You can still have decisions be made by objective expert systems without complex ML. If you want to learn someone's IQ, the best way is to debias the IQ test, not to try and infer it from their face bones. > it's that the bias is more measurable If we can measure the bias in the output of an ML model, we can equivalently measure the bias in the output of a human system. You're presupposing the existence of some unbiased objective function which we don't have, and that's at the core of the issue. [0]: https://www.wired.com/story/ideas-joi-ito-insurance-algorithms/ https://www.wired.com/story/ideas-joi-ito-insurance-algorith... has a few good examples here, like how naive bail and sentencing models encode racial bias that isn't present in humans. And to be clear the response here shouldn't be "well let's just build better models" but "why do we think a model will improve the situation here at all"? Removing agency from Judges has historically been bad for the average person convicted of a crime. This doesn't mean that individual judges can't make terrible rulings, but that the alternatives are usually worse on the whole.
- darawk 6y ago
  > I don't follow this. If we have a biased objective function, the model won't surface any biases we weren't already cognizant of in the objective function. And they were already quantifiable: we had a function that we were using to evaluate the model. We could use that same function on whatever non-model evaluation we were doing. We can actually follow the logic of the model. For instance, you can theoretically de-bias a dataset by building a racial classifier from it. What you need is an objective test for the presence of racial information, and that's easy to obtain: Build a classifier to explicitly predict race from your feature set. Train an adversarial model to reconstruct your dataset with maximum fidelity, subject to the constraint that race can no longer be predicted from it. > This is basically directly in contradiction to what leading experts on the subject say. ML cannot fix bias in human systems, unless we presuppose that those systems are biased, in which case we can often address the bias in the human systems directly without ML. These experts are just wrong, then. Naive ML won't fix bias in human systems, but that doesn't mean we can't use ML to fix it, if we do so thoughtfully. > You can still have decisions be made by objective expert systems without complex ML. If you want to learn someone's IQ, the best way is to debias the IQ test, not to try and infer it from their face bones. Sure, but there are a lot of things that we don't do in the best possible way because it's too expensive. There are lots of use cases for cheap, scalable, low precision models. > If we can measure the bias in the output of an ML model, we can equivalently measure the bias in the output of a human system. You're presupposing the existence of some unbiased objective function which we don't have, and that's at the core of the issue. Right, but we cannot fix the bias in a human. And humans are heterogenous and inconsistent. The same person may be more or less biased on different days. The ML model is consistent, and we can incrementally improve its bias in tangible and testable ways. The same is not true of humans.
  - joshuamorton 6y ago
    > de-bias a dataset And what does this get you? Let's look at a face recognition dataset. What happens when you debias it? Is it still useful? No. Because the faces no longer resemble real faces. > These experts are just wrong, then Perhaps, but you aren't making a strong case for that. > There are lots of use cases for cheap, scalable, low precision models. That involve facial recognition? > Right, but we cannot fix the bias in a human We don't need to. We just need to fix the bias in the system. And we absolutely can incrementally reduce bias in systems that involve humans.

by darawk 6y ago

joshuamorton 6y ago

> An ML model isn't going to magically make those biases worse, either. Well, actually, research shows that unless great care is taken it absolutely can. If you include for example race as a factor in a model, it can learn non-causal correlations between race and whatever the objective is. This can have a compounding effect in some cases. [0] > What it's going to do is bring them to the surface, so that they are quantifiable and we can actually do something about them. I don't follow this. If we have a biased objective function, the model won't surface any biases we weren't already cognizant of in the objective function. And they were already quantifiable: we had a function that we were using to evaluate the model. We could use that same function on whatever non-model evaluation we were doing. > ML is the solution to the bias problem. This is basically directly in contradiction to what leading experts on the subject say. ML cannot fix bias in human systems, unless we presuppose that those systems are biased, in which case we can often address the bias in the human systems directly without ML. > Humans who we cannot statistically debias. Humans who's biases we can't even effectively interrogate. You can still have decisions be made by objective expert systems without complex ML. If you want to learn someone's IQ, the best way is to debias the IQ test, not to try and infer it from their face bones. > it's that the bias is more measurable If we can measure the bias in the output of an ML model, we can equivalently measure the bias in the output of a human system. You're presupposing the existence of some unbiased objective function which we don't have, and that's at the core of the issue. [0]: https://www.wired.com/story/ideas-joi-ito-insurance-algorithms/ https://www.wired.com/story/ideas-joi-ito-insurance-algorith... has a few good examples here, like how naive bail and sentencing models encode racial bias that isn't present in humans. And to be clear the response here shouldn't be "well let's just build better models" but "why do we think a model will improve the situation here at all"? Removing agency from Judges has historically been bad for the average person convicted of a crime. This doesn't mean that individual judges can't make terrible rulings, but that the alternatives are usually worse on the whole.

darawk 6y ago
> I don't follow this. If we have a biased objective function, the model won't surface any biases we weren't already cognizant of in the objective function. And they were already quantifiable: we had a function that we were using to evaluate the model. We could use that same function on whatever non-model evaluation we were doing. We can actually follow the logic of the model. For instance, you can theoretically de-bias a dataset by building a racial classifier from it. What you need is an objective test for the presence of racial information, and that's easy to obtain: Build a classifier to explicitly predict race from your feature set. Train an adversarial model to reconstruct your dataset with maximum fidelity, subject to the constraint that race can no longer be predicted from it. > This is basically directly in contradiction to what leading experts on the subject say. ML cannot fix bias in human systems, unless we presuppose that those systems are biased, in which case we can often address the bias in the human systems directly without ML. These experts are just wrong, then. Naive ML won't fix bias in human systems, but that doesn't mean we can't use ML to fix it, if we do so thoughtfully. > You can still have decisions be made by objective expert systems without complex ML. If you want to learn someone's IQ, the best way is to debias the IQ test, not to try and infer it from their face bones. Sure, but there are a lot of things that we don't do in the best possible way because it's too expensive. There are lots of use cases for cheap, scalable, low precision models. > If we can measure the bias in the output of an ML model, we can equivalently measure the bias in the output of a human system. You're presupposing the existence of some unbiased objective function which we don't have, and that's at the core of the issue. Right, but we cannot fix the bias in a human. And humans are heterogenous and inconsistent. The same person may be more or less biased on different days. The ML model is consistent, and we can incrementally improve its bias in tangible and testable ways. The same is not true of humans.
- joshuamorton 6y ago
  > de-bias a dataset And what does this get you? Let's look at a face recognition dataset. What happens when you debias it? Is it still useful? No. Because the faces no longer resemble real faces. > These experts are just wrong, then Perhaps, but you aren't making a strong case for that. > There are lots of use cases for cheap, scalable, low precision models. That involve facial recognition? > Right, but we cannot fix the bias in a human We don't need to. We just need to fix the bias in the system. And we absolutely can incrementally reduce bias in systems that involve humans.