5 ms·
With current technology the problem of Bob (or Adolph or Mohamed) becoming associated with evil is insolvable by the fact that current deep learning systems are
by joe_the_user 9y ago
With current technology the problem of Bob (or Adolph or Mohamed) becoming associated with evil is insolvable by the fact that current deep learning systems are fundamentally characterized by not being able to distinguish causation from correlation.
The only solution I can see is forcing any company that imposes life-defining actions on people (credit bureaus, banks, parole boards, personnel offices, etc) to use only rules based on objective criteria and to prohibit systems based on a "lasagna" of ad-hoc data like present day AI systems. Indeed, if one looks at these in the light of day, one would have to describe such system as fundamentally evil, the definition of "playing games with people's lives." (just look at the racist parole-granting software, etc).
- pjlegato 9y agoThis isn't just a technology problem. Most humans are also really, really bad at distinguishing causation from correlation. The very notion that these are even different things is utterly alien to most people. An AI that was even slightly good at it would be far ahead of what we get when humans run things.
- nileshtrivedi 9y agoThere is no free lunch. Any forced adjustments we make to reduce false positives, will increase the false negatives. It's up to us to decide what trade-off we want and the algorithms will simply adapt to that. If we believe certain attributes to be irrelevant to a situation, we need to build systems that are completely blind to these attributes. This is how double-blind trials, or scientific peer review works.
- nl 9y agoAny forced adjustments we make to reduce false positives, will increase the false negatives. That's just not true. You can have systematic errors caused by bad training data which can be fixed without an increase in false negatives (otherwise new ML systems would never improve over old ones!) In the physical world a good analogy is crash testing of cars. For decades (until 2011!!), crash test dummies were all based on average sized American males. That led to hugely increased risks for female passengers: the female dummy in the front passenger seat registered a 20 to 40 percent risk of being killed or seriously injured, according to the test data. The average for that class of vehicle is 15 percent. Fixing that problem didn't case any increase in accident risk for men. It's the same in machine learning. There is no free lunch This isn't what the no free lunch theorem[2] says. That says that all optimization methods are theoretically equivalent. [1] https://www.washingtonpost.com/local/trafficandcommuting/female-dummy-makes-her-mark-on-male-dominated-crash-tests/2012/03/07/gIQANBLjaS_story.html https://www.washingtonpost.com/local/trafficandcommuting/fem... [2] https://en.wikipedia.org/wiki/No_free_lunch_theorem https://en.wikipedia.org/wiki/No_free_lunch_theorem
- gowld 9y ago"Causation" is defined as temporal correlation: changes in A cause changes in B. There's no reason to thin computers would be "bad" at this. People are only bad at this because they have small, lazy brains Even the greatest physicists and mathematicians can't tell the difference between correlation and causation. When people say "causation is not correlation", what they truly mean is "spurious, context-sensitive correlation is not strong universal correlation"
- joshuamorton 9y agoNo. Temporal correlation is "changes in A precede changes in B". This is not causation, you more or less correctly describe causation when you say "Changes in A cause changes in B", but a better way of putting it might be "all else equal, changes in A cause changes in B". The difficulty here is controlling for all possible C. As a simple example: If A causes a change in both B and C, but the change in B happens more quickly, "temporal correlation" would imply that B causes C, when that's not the case. This is especially obvious with cyclical phenomena. The tide going out does not cause the sun to rise, for example, even though I'm sure I could draw a temporal correlation between them. Nor does my car being in my garage cause me to go to work.
- oldandtired 9y agoIf it is insolvable for the current deep learning systems then they aren't worth the time and effort that has been spent on them. Your solution will not work, ever. All such companies will adjust their processes to maximise their own benefits, irrespective on any legal consequences. For most, the profits (in whatever way they define profit) will be the more important thing than any legal requirements placed on them. It is the nature of the people running these companies.
- AnthonyMouse 9y ago> The only solution I can see is forcing any company that imposes life-defining actions on people (credit bureaus, banks, parole boards, personnel offices, etc) to use only rules based on objective criteria and to prohibit systems based on a "lasagna" of ad-hoc data like present day AI systems. That is probably the exact opposite of what you really want. If the problem is that someone's name is Bob and the AI thinks Bobs are evil, what you want is for there to be 100,000 other factors for Bob to show the system that it isn't so. As many factors as possible, so that the one it gets wrong will have a very low weight. Even the objective criteria will have biases. There is a significant racial disparity in prior criminal convictions, income, credit history and nearly every other "objective" factor. The more factors you bring in, the more opportunities someone in a given demographic has to prove they still deserve a chance.
- joe_the_user 9y agoThat is probably the exact opposite of what you really want. If the problem is that someone's name is Bob and the AI thinks Bobs are evil, what you want is for there to be 100,000 other factors for Bob to show the system that it isn't so, as many factors as possible, so that the one it gets wrong will have a very low weight. You don't understand. My point is that institutions making such decision should not be able to make decisions according to these 100,000 unexplained factors. If you're a lender, you can look at employment history, records of payment and other objective related criteria. You can't look at, say, eye color, however useful you might think it is. Institutions should be able make these decisions arbitrarily, at the level that they effect lives. There should legal provisions for auditing these things (as there are, on occasions, provisions of auditing affirmative action, environmental protection behaviors, insurance decisions, etc).
- AnthonyMouse 9y ago> My point is that institutions making such decision should not be able to make decisions according to these 100,000 unexplained factors. But how does that help anything? The objective factors have the same potential for bias as the seemingly irrelevant ones. All you get by excluding factors is to increase bias by not considering information that could mitigate the bias in the factors you are considering. Suppose that 80% of black men would be rejected for a loan based on your preferred set of objective factors. Of that 80%, many would actually repay the loan, but none of the objective factors can distinguish them from those who wouldn't, and when averaged together the outcome is to refuse the loan. If you used some seemingly arbitrary factors that happen to correlate for unknown reasons, you could profitably make loans to 60% of them instead of 20%. How is it helping anyone to not do that?
- deleted 9y ago[deleted]
- kriro 9y agoThe main problem is that most other systems have their own biases. A typical suggestion I hear a lot goes along the lines of "if in doubt, let a human make the decision". If you think about ML in say a school context it's likely that certain biases are baked in but I'd argue that these biases tend to be more extreme when humans make decisions because they take into account less factors (this may not be true but it's my working hypothesis). I think a decent example is "first names". There's certain names that result in worse overall grades during a normal school career, even if the teachers are aware of this bias (in Germany the posterchild for this is "Kevin"). It's a tough problem. I think being aware that biases exist in ML is a good first step.
- Firadeoclus 9y ago> I think a decent example is "first names". There's certain names that result in worse overall grades during a normal school career, even if the teachers are aware of this bias (in Germany the posterchild for this is "Kevin"). There is a possible causal link with names which goes beyond "children are being treated worse because of their name".