>"A strong example is a classifier that distinguishes between wolves and dogs. When you look at which pixels were the most impactful in 'Wolf' predictions, you

>"A strong example is a classifier that distinguishes between wolves and dogs. When you look at which pixels were the most impactful in 'Wolf' predictions, you can see that it is actually pixels of snow that are leading to predictions of 'Wolf'." Yes, but how common is a situation like this? I mean your classifier is then not much better than making a histogram of pixel intensities.