5 ms·
There was a machine learning system designed to produce interpretable results, called Eureqa. Eureqa is a fantastic piece of software that finds simple mathemat
by Houshalter 10y ago
There was a machine learning system designed to produce interpretable results, called Eureqa. Eureqa is a fantastic piece of software that finds simple mathematical equations that fit your data as good as possible. Emphasis on the "simple", it searches for the smallest equations it can find that works, and gives you a choice of different equations at different levels of complexity.
But still, the results are very difficult to interpret. Yes you can verify that the equation works, that it predicts the data. But why does it work? Well who knows? No one can answer that. Understanding even simple math expressions can be quite difficult. Imagine trying to learn physics from just reading the math equations involved and nothing else.
One biologist put his data into the program, and found, to his surprise, that it found a simple expression that almost perfectly explained one of the variables he was interested in. But he couldn't publish his result, because he couldn't understand it himself. You can't just publish a random equation with no explanation. What use is that?
I think the best method of understanding our models, is not going to come from making simpler models that we can compute by hand. Instead I think we should take advantage of our own neural networks. Try to train humans to predict what inputs, particularly in images, will activate a node in a neural network. We will learn that function ourselves, and then it's purpose will make sense to us. Just looking at the gradients of the input conveys a huge amount of information of which inout features are the most and least important. And by about how much.
But mostly I think the effort towards explainability is fundamentally misguided. In the domains where they are supposedly the most desirable, like medicine, accuracy should matter above all. A less accurate model could cost lives. Accuracy is easy to verify through cross validation, but explainability is a mysterious unmeasurable goal.
- paulsutter 10y ago> But mostly I think the effort towards explainability is fundamentally misguided... explainability is a mysterious unmeasurable goal. Do you think people should stop working on it? Most interesting things seem difficult to measure, until someone finds a way, and then it seems obvious. An example is search engine quality. At first this might seem too subjective to be measurable. But Google started measuring search quality using a panel of humans, and now everybody does that. The whole idea of these challenges is to broaden the search, to hope for key insights that by their nature seem elusive at first.
- Houshalter 10y agoWhether or not we can ever come up with a good definition of "explainability", I still think it's misguided. In almost all applications of ML, accuracy matters far, far more. Predicting whether a patient has cancer, for instance, it matters a great deal that you get the highest accuracy possible. Every point in accuracy you lose to "explainability" means people die.
- romaniv 10y agoUntil your model starts misdiagnosing people in real life because your training data wasn't 100% perfect (which it never will be for complex real-life problems). https://vimeo.com/125940125 https://vimeo.com/125940125 The problem with unexplainable AI isn't just the lack of explanations. It's the fact that the entire model is a black box, so you cannot somehow tweak a single part of it without retraining.
- woodman 10y agoEureqa uses (used? I haven't touched it in years) symbolic regression. Symbolic regression can certainly produce weird results, especially with low quality data, but I wouldn't rate it anywhere near classic ANNs in black-boxiness. Also, science is unfortunately full of magic numbers - but it is pretty amazing when you feed in pendulum motion data and the resulting equation ends up being Newton's second law: http://phys.org/news/2009-12-eureqa-robot-scientist-video.html http://phys.org/news/2009-12-eureqa-robot-scientist-video.ht...
- Houshalter 10y agoYes it's amazing that it can find an equation that fits the pendulum's motion. But if you didn't already know the physics math, it would just seem weird and arbitrary.
- woodman 10y agoNo more weird and arbitrary than the motion of the pendulum itself. I think you're underestimating the human intelligence component of the system. Include in the dataset an additional variable for rope length, record the data for multiple lengths - there, you've just isolated a variable in the equation and it is no longer arbitrary. The scientific method still works, but now you're working on a much more tightly bounded problem.