4 ms·
This is a fundamental problem with statistical approaches like NNs: we currently have no good idea when our how they will fail; it's the explanation problem. P
by mcguire 10y ago
This is a fundamental problem with statistical approaches like NNs: we currently have no good idea when our how they will fail; it's the explanation problem.
Putting a great deal of reliance on a system in that situation seems like a great mistake. "It looks like it works under normal circumstances" is not very reliable.
- scarmig 10y agoDevil's advocate: suppose humans succeed only 99% of the time, but we can understand and characterize the 1% of failures fairly well. Now suppose machine learning-based approaches succeed 99.9% of the time and only fail 0.1%, but that 0.1% is for totally unknowable reasons. Is the former better than the latter? It's fun to imagine someone constructing adversarial examples to cause self-driving cars to crash. But is that really a big deal? Humans can just as easily be tripped up by an "adversarial" example: I can pretty easily point a laser pointer from my window at random humans driving a car. The legal system has well-defined mechanisms for dealing with this.
- mcguire 10y agoCollectively, we have a lot of experience with the foibles of our visual system (to the extent that we don't notice what it does poorly, which is a problem, too). Further, we've arranged civilization to play to its strengths, which is one of the reasons we're so interested in facial recognition. The recognition systems look like they are doing the same thing we are, but they're not. Adding an adversary taking advantage of the difference is just sauce on top of the problem.