3 ms·

We rely on human judgements that can't be described all the time in many of the above situations, so why would it be a "major concern" with machines? Machines c

by 0-_-0 5y ago

We rely on human judgements that can't be described all the time in many of the above situations, so why would it be a "major concern" with machines? Machines can be much more thoroughly tested than humans, so they should be more statistically predictable

shadowgovt 5y ago
Three reasons: one practical, two psychological. The practical one is that errors in a machine system scale, as do most things with machines. If I have a single bad X-ray tech who is applying the wrong medical process because I have a different race, for some reason, the damage that tech is doing is limited to whatever specific set of patients they are seeing. If a similar error occurs in a popular machine classification tool used widely by a hospital network, the damage is widespread. It is a plus that the machine can be corrected and the correction also scales, but with the (relatively speaking) stone tools we use to understand why a CNN makes its decisions these days, every fix risks breaking something else we're not testing for. The first psychological reason is that machine learning systems break in "alien" ways. They don't make the kind of mistakes humans make... They make mistakes as a product of their machinery, which means it's much much harder to predict what those mistakes will look like for an average operator. As a frequent example, it's pretty rare for humans to misclassify human beings in photographs as apes, or to fail to recognize a face in an image because the skin is too dark. That's a failure mode that happens over and over again with image recognition systems. And the second psychological reason is that humans don't trust machines to make human decisions yet. And that mistrust doesn't extend to other humans, even though we're incapable of cracking open another human's mind and understanding their thought process at the mechanistic level. It doesn't matter... we are the same organism and have a shared experience and empathy with them that we lack with machine recognition systems. It's semi-irrational, but it can't be wished away. A system for understanding why a machine makes decisions would be a step in the direction of addressing those concerns.

by 0-_-0 5y ago

shadowgovt 5y ago

Three reasons: one practical, two psychological. The practical one is that errors in a machine system scale, as do most things with machines. If I have a single bad X-ray tech who is applying the wrong medical process because I have a different race, for some reason, the damage that tech is doing is limited to whatever specific set of patients they are seeing. If a similar error occurs in a popular machine classification tool used widely by a hospital network, the damage is widespread. It is a plus that the machine can be corrected and the correction also scales, but with the (relatively speaking) stone tools we use to understand why a CNN makes its decisions these days, every fix risks breaking something else we're not testing for. The first psychological reason is that machine learning systems break in "alien" ways. They don't make the kind of mistakes humans make... They make mistakes as a product of their machinery, which means it's much much harder to predict what those mistakes will look like for an average operator. As a frequent example, it's pretty rare for humans to misclassify human beings in photographs as apes, or to fail to recognize a face in an image because the skin is too dark. That's a failure mode that happens over and over again with image recognition systems. And the second psychological reason is that humans don't trust machines to make human decisions yet. And that mistrust doesn't extend to other humans, even though we're incapable of cracking open another human's mind and understanding their thought process at the mechanistic level. It doesn't matter... we are the same organism and have a shared experience and empathy with them that we lack with machine recognition systems. It's semi-irrational, but it can't be wished away. A system for understanding why a machine makes decisions would be a step in the direction of addressing those concerns.