4 ms·
I hope someday soon we'll develop systems that allow us to "ask" a ML algorithm what factors led to a decision (diagnosis in this case). It would be interestin
by leereeves 10y ago
I hope someday soon we'll develop systems that allow us to "ask" a ML algorithm what factors led to a decision (diagnosis in this case).
It would be interesting to compare that with the current state of the art in the field, and see if ML can contribute new scientific/medical theory as well.
- derefr 10y agoI'm not so sure we could ask the algorithm itself, in any literal sense. An algorithm trained to introspect might actually be wrong about its own "memories" or "motives"—just like a human might! (Though, likely, without the penchant toward political rationalizing.) This is most simply because, whatever the algorithm is trained to do, it's certainly trained better to do that thing than to introspect. Introspection is a separate skill! But there's also a more insidious element: introspection (in humans, at least) tends to result in the creation of a lot of "personal concepts" that don't map to well-known common concepts. An introspection on one mind must necessarily result in a taxonomy that contains terms for the tiny, unique features that only that mind has—which makes it very, very hard to communicate one's personal introspections to others. (You might call this a kind of overfitting: the introspection capability becomes optimized for that one mind, but ceases to translate well to features in other minds—like human minds.) I'd place a much stronger bet on our ability to train one AI to "stare at the brain" of other AIs as they make decisions [tons of them, as its training data], with the expected output being a general theory on common AI features responsible for the given calculation step. A computer psychologist, of sorts. :) Of course, you could include such a pre-trained model as a "module" alongside the AI itself, and call the combined system "one AI" if you like.
- leereeves 10y ago> Introspection is a separate skill! Indeed it is. It's something that a second algorithm (perhaps ML, perhaps not) would do. And this is beginning to remind me of Society of Mind. https://en.wikipedia.org/wiki/Society_of_Mind https://en.wikipedia.org/wiki/Society_of_Mind
- romaniv 10y agoWell, there are ML algorithms that can be interrogated about their reasoning. One type is symbolic systems that use data to generate or "tune" their rules. Another one is stuff like this: https://www.youtube.com/watch?v=UqPcq0n59rQ https://www.youtube.com/watch?v=UqPcq0n59rQ (it's a kind of ensemble system that uses a linear combination of human-interpretable classifiers). Both kinds are pretty interesting, because you can manually inspect and tune them after training.
- kharms 10y agoDepending on the algorithm, there is this: https://homes.cs.washington.edu/~marcotcr/blog/lime/ https://homes.cs.washington.edu/~marcotcr/blog/lime/ It's a neat approach that I've used with some random forest classifiers.