6 ms·

> Is it better to criticize the math that discovered that fact, or the root cause that made it true to begin with? The only place where math was involved was t

by jfasi 8y ago

> Is it better to criticize the math that discovered that fact, or the root cause that made it true to begin with? The only place where math was involved was the guts of the training stage of the model. A crucial stage to be sure, but one that's bookended in the front by problem definition, data selection, model selection, and a design for an evaluation process, and behind by the execution of that evaluation and the decision to launch the model. Literally every other stage of this process is driven by human decisions. I'll say it again, because apparently the point didn't sink in the first time around: Machine learning models are inference and pattern recognition devices, not scientific tools. The fact that it's inhuman and unthinking mathematics that produced a biased model offers no ethical or legal cover to the people who decide to put that model into use. The decision to apply the model is key here. Contrast two applications: one that takes in a patient's information and diagnosis to compute a dosage for a drug, and one that takes in a potential tenant's request and produces a rent/no rent decision. In the first, there are cases in which the bias is admissible if not necessary, e.g. [1]. However, the legitimacy of that model's application comes not from the supposed objectivity of the model's findings but from volumes of peer (i.e. human) reviewed research. In the second, there is no legal way in which this model can be applied, and I struggle to imagine a moral one. I can't imagine any court of law taking "the machine made me do it" as a defense in an FHA case. [1] https://www.nytimes.com/2005/06/24/health/fda-approves-a-heart-drug-for-africanamericans.html https://www.nytimes.com/2005/06/24/health/fda-approves-a-hea...

iguy 8y ago
pattern recognition devices, not scientific tools Isn't science all about pattern recognition? If the pattern exists, in the real world, then a good theory is one which encodes this. What you're asking for in a "moral" way of applying the model is that our actions should abide by your moral preferences. Perhaps even universal preferences. But it seems useful to me to keep logically separate these ideas about how we ought to do things. They don't flow naturally from observations of how things are. The example of adjusting drug & dosage based on race is a good one. The science backing this is exactly the same kind of statistical correlation as backs the rental decision. The training input is what race some test patients ticked on a form, and their tick mark sure as hell isn't the causal factor... that's some gene which is correlated, maybe, or some diet difference, or what's on TV, who knows. Nevertheless the correlation is there, as far as we can tell: the peer-reviewed science process isn't infallible. The reason we're OK with using this information is, I guess, that it aims to improve things for the patient. (Not every single patient, only statistically.) We make a moral judgement that this is more important than a landlord's wish to avoid bad tenants (again statistically).

by jfasi 8y ago

iguy 8y ago

pattern recognition devices, not scientific tools Isn't science all about pattern recognition? If the pattern exists, in the real world, then a good theory is one which encodes this. What you're asking for in a "moral" way of applying the model is that our actions should abide by your moral preferences. Perhaps even universal preferences. But it seems useful to me to keep logically separate these ideas about how we ought to do things. They don't flow naturally from observations of how things are. The example of adjusting drug & dosage based on race is a good one. The science backing this is exactly the same kind of statistical correlation as backs the rental decision. The training input is what race some test patients ticked on a form, and their tick mark sure as hell isn't the causal factor... that's some gene which is correlated, maybe, or some diet difference, or what's on TV, who knows. Nevertheless the correlation is there, as far as we can tell: the peer-reviewed science process isn't infallible. The reason we're OK with using this information is, I guess, that it aims to improve things for the patient. (Not every single patient, only statistically.) We make a moral judgement that this is more important than a landlord's wish to avoid bad tenants (again statistically).