5 ms·
I think I agree with everything on that page, but this seems to be incomplete: > It shouldn't be a mission impossible to find out how and why a decision about
by da39a3ee 6y ago
I think I agree with everything on that page, but this seems to be incomplete:
> It shouldn't be a mission impossible to find out how and why a decision about you was made.
That does sound attractive. But I don't see how it squares with technology. For certain technologies, the answer will be "the weights in the neural network, when applied to your attributes, yielded a score that fell above a cutoff" or "Your data attributes mapped to a certain location on an eigenvector and it fell above a cutoff defined on that eigenvector".
Is the answer to stop using such algorithms? If not, doesn't this webpage need to give us more of a hint as to how we are to overcome the apparent impasse?
- andersource 6y agoModel explainability is a very active area of research. For the example of an individual neural network prediction you could use SHAP values[0]. Explainability is far from a solved problem but there are certainly tools available to provide at least some transparency. I guess the website could provide references but that doesn't really seem to be the resolution being aimed for. [0] https://www.kaggle.com/dansbecker/shap-values https://www.kaggle.com/dansbecker/shap-values.
- jdashg 6y agoIf you can't guarantee that the algorithm respects protected classes (or other forbidden discriminations) then yes, you can't use it. Compare it to the hypothetical process where "we have Jim decide". You can't open up Jim's head and see why he made choices, but you can run tests against the system (Jim) to probe for forbidden discriminations.
- gus_massa 6y agoIt is very difficult to run test to be sure that Jim is not biased. The advantage of Jim is that you can ask Jim and then take at face value his answer. (And later, if there is a big problem you can just blame Jim and fire him.)