3 ms·

Shap values and other methods to parse out the inner workings for “black box” machine learning models. They’re good enough that I’ve grown fond of just throwing

Shap values and other methods to parse out the inner workings for “black box” machine learning models. They’re good enough that I’ve grown fond of just throwing a light gbm model at everything and calling it a day for that sweet spot of predictive power and ease of implementation.

hackernewds 5y ago
Would be so kind to share an example or resources to learn about this?
- screye 5y ago
  [1] Scott is the leading authority on all things SHAP (he wrote the seminal paper on it). [2] Chris Molnar's interpretable learning book has chapters on Shapeley values and SHAP. If you'd prefer text instead of video. [1] https://www.youtube.com/watch?v=B-c8tIgchu0 https://www.youtube.com/watch?v=B-c8tIgchu0 [2] https://christophm.github.io/interpretable-ml-book/shapley.html https://christophm.github.io/interpretable-ml-book/shapley.h...
- magneticnorth 5y ago
  Seconding Chris Molnar's excellent writeup. I also find the readme & example notebooks in Scott Lundberg's github repo to be a great way to get started. There are also references there for the original papers, which are surprisingly readable, imo. https://github.com/slundberg/shap https://github.com/slundberg/shap
hervature 5y ago
Strongly disagree. Shapley values and LIME give a very crude and extremely limited understanding of the model. They basically amount to a random selection of local slopes. For instance, if I tell you the (randomly selected) slopes of a function are [0.5, 0.2, 12.4, 1.1, 2.6] which the average is 3.3, can you guess anything? You might notice it is monotonic (maybe) but you certainly don't guess it is e^x.
- LeanderK 5y ago
  I would say that our ML models are not predictable enough yet at the local neighbourhood to really trust LIME. Adversarial examples prove that you just can't select a small enough range since you can always find those even for super tiny distances.
- oli5679 5y ago
  It depends on your domain. If you have a relatively small number of features and your features have relatively stable but not necessarily linear partial dependence, then shap can unpack them pretty well. E.g. value of a house it might not necessarily increase linearly in squarefoot, distance from city centre and other factors so a random-forest/gbt might do better than a linear model without careful feature engineering. Shap is a brilliant tool to unpack the dependencies this model has learned. There will be other cases where a linear model is adequate, so you can get the explainability just from the coefficients, or where there are very large numbers of features with complicated nested structure. In either case shap is not too useful. But the 'fit lightgbm and explain it with shap' is a decently pragmatic approach for lots of tabular data machine-learning problems.
- oneoff786 5y ago
  Maybe. But I don’t care about estimating the slopes. I just need to know big values of feature X pull the model prediction up and Small values do nothing. If you really need to unpack it, cluster your shaps. To see what kinds of predictions are being made
teruakohatu 5y ago
> I’ve grown fond of just throwing a light gbm model at everything and calling it a day It is not always a good idea to do that. Always try different methods, there is no ultimate method. At the very least OLS should be tried and some other fully explainable methods, even a simple CART like method.
- csee 5y ago
  OLS is my default go to. It outperforms a random forest so often in small data real world applications over nonstationary data, and model explainability is built into it. If I'm working in a domain with stationary data then I'd tilt more to the forest (due to not having to engineer features, and the inbuilt ability to detect non-linear relationships and interactions between features).
- deleted 5y ago
  [deleted]
sockpuppet69 5y ago

hackernewds 5y ago

Would be so kind to share an example or resources to learn about this?

screye 5y ago
[1] Scott is the leading authority on all things SHAP (he wrote the seminal paper on it). [2] Chris Molnar's interpretable learning book has chapters on Shapeley values and SHAP. If you'd prefer text instead of video. [1] https://www.youtube.com/watch?v=B-c8tIgchu0 https://www.youtube.com/watch?v=B-c8tIgchu0 [2] https://christophm.github.io/interpretable-ml-book/shapley.html https://christophm.github.io/interpretable-ml-book/shapley.h...
magneticnorth 5y ago
Seconding Chris Molnar's excellent writeup. I also find the readme & example notebooks in Scott Lundberg's github repo to be a great way to get started. There are also references there for the original papers, which are surprisingly readable, imo. https://github.com/slundberg/shap https://github.com/slundberg/shap

hervature 5y ago

Strongly disagree. Shapley values and LIME give a very crude and extremely limited understanding of the model. They basically amount to a random selection of local slopes. For instance, if I tell you the (randomly selected) slopes of a function are [0.5, 0.2, 12.4, 1.1, 2.6] which the average is 3.3, can you guess anything? You might notice it is monotonic (maybe) but you certainly don't guess it is e^x.

LeanderK 5y ago
I would say that our ML models are not predictable enough yet at the local neighbourhood to really trust LIME. Adversarial examples prove that you just can't select a small enough range since you can always find those even for super tiny distances.
oli5679 5y ago
It depends on your domain. If you have a relatively small number of features and your features have relatively stable but not necessarily linear partial dependence, then shap can unpack them pretty well. E.g. value of a house it might not necessarily increase linearly in squarefoot, distance from city centre and other factors so a random-forest/gbt might do better than a linear model without careful feature engineering. Shap is a brilliant tool to unpack the dependencies this model has learned. There will be other cases where a linear model is adequate, so you can get the explainability just from the coefficients, or where there are very large numbers of features with complicated nested structure. In either case shap is not too useful. But the 'fit lightgbm and explain it with shap' is a decently pragmatic approach for lots of tabular data machine-learning problems.
oneoff786 5y ago
Maybe. But I don’t care about estimating the slopes. I just need to know big values of feature X pull the model prediction up and Small values do nothing. If you really need to unpack it, cluster your shaps. To see what kinds of predictions are being made

teruakohatu 5y ago

> I’ve grown fond of just throwing a light gbm model at everything and calling it a day It is not always a good idea to do that. Always try different methods, there is no ultimate method. At the very least OLS should be tried and some other fully explainable methods, even a simple CART like method.

csee 5y ago
OLS is my default go to. It outperforms a random forest so often in small data real world applications over nonstationary data, and model explainability is built into it. If I'm working in a domain with stationary data then I'd tilt more to the forest (due to not having to engineer features, and the inbuilt ability to detect non-linear relationships and interactions between features).
deleted 5y ago
[deleted]

sockpuppet69 5y ago