5 ms·
I don't really understand your confusion? They say the following: - This model can distinguish photographs of a male retina from photographs of a female retin
by thaumasiotes 6y ago
I don't really understand your confusion?
They say the following:
- This model can distinguish photographs of a male retina from photographs of a female retina.
- We don't know, ourselves, how to do that.
- We would like to be able to determine, from looking at the model, what features it's using to draw the distinction.
What's weird?
- Traster 6y agoWhat I'm confused by is that they say this is important to do, but then don't actually seem to do it?
- joe_the_user 6y agoThey don't give an explanation because they don't know how to give an explanation - many if not most ML model lack an easy explanation presently, they just spit out answers. They are saying "someone should do this because it's important even though we don't (presently) know how to do this".
- aaron-santos 6y agoI'm open to learning why class activation maps (CAM) would or wouldn't be a good place to start.
- thegginthesky 6y agoWouldn't model explainers, such as Shap[0] or LIME[1], be able to assist in explaining these results? There are some very interesting advances in the space of model explainability that merit at least a try. [0]https://github.com/slundberg/shap https://github.com/slundberg/shap [1]https://github.com/marcotcr/lime https://github.com/marcotcr/lime
- Mehdi2277 6y agoHave you ever used them? They're pretty but not things I put much trust in and the other awkward issue is there's been a variety of papers that do an analysis of many interpretability algorithms and find some very weak properties. One example is train a model M, run something like SHAP, delete the features marked important in the original dataset, train a new model M2 and see minimal performance difference. At best that tells us many models have large number of ways to get there answer and the interpretability algorithm is giving you one. If your goal was understanding what distinguishes the data and not just specific model you failed. At worst it tells you the local way most interpretability algorithms work is just not revealing. The local nature (most are heavily gradient based) when working with data like images where individual pixels mean little also puts a lot of skepticism for me. There are a decent number of interpretability algorithms that look like they're just running edge detection which makes sense with gradient part but is poor for interpretability.
- JabavuAdams 6y agoImportant to do next.
- make3 6y agofuture work is almost always discussed in publications
- mattkrause 6y ago> We don't know, ourselves, how to do that. Have we actually tried? There is a cursory discussion about how this is "inconceivable to those who spent their careers looking at retinas". However, if it's not clinically useful (as the next sentence says), those experts probably haven't spent much--if any--training themselves to try. Humans can learn to detect surprisingly subtle features. For example, the right training regime can make you much better at reporting the tilt of a line, but it requires practice and feedback, just like the network got.
- barry-cotter 6y agoI bet this would end up like chicken sexers. Near 100% reliability, no ability to explain how they do it.
- spoonjim 6y agoHow do you learn to get good at the tilt of a line?
- mattkrause 6y agoPractice, practice, practice. The umbrella term for this is "perceptual learning" and it turns out that training can tweak the visual system: baseball players learn to read where an incoming pitch will go, radiologists can find subtle clues that indicate a tumor, and--as someone pointed out above--chicken sexers can tell the sex of a baby chicken somehow. These are fairly "high-level" phenomenon. Surprisingly, training also works on low-level visual phenomena, which you might think are limited by the eye itself or 'hard-wired' neural circuits. It's mostly just practice. One of the classic experiments looks at vernier (hyper-)acuity. You're shown pairs of lines that are slightly offset, like: | or | | | You report whether the top is shifted rightwards or leftwards. The computer gives you feedback, and shows you another pair (varying the distance to make it easier or harder). If you keep doing this, your threshold (i.e., the smallest offset you can reliably report) will about decrease 5-fold. The same thing happens for many other phenomena too--reporting the direction that some dots are moving, the tilt of a line, etc. In some cases, the improvements continue for days or even weeks of training. The wild thing is that they're often very specific to the training. For example, if you trained with the vertical stimuli above, the improvement does not transfer over to stimuli like: _ or _ _ _ There are "tricks" to designing a curriculum that generalizes and is relatively efficient, but there's still a lot to learn.