2 ms·

yeah. you can run SHAP[0] on your xgboosted trees, results are kinda interesting, but it doesn't actually explain anything IME. [0] https://shap.readthedocs.io

by baq 2y ago

yeah. you can run SHAP[0] on your xgboosted trees, results are kinda interesting, but it doesn't actually explain anything IME. [0] https://shap.readthedocs.io/en/latest/index.html https://shap.readthedocs.io/en/latest/index.html

cubefox 2y ago
No wonder. "Shapley values" have the problem that they assume all necessary conditions are equally important. Say a successful surgery needs both a surgeon and a nurse, otherwise the patient dies. Shapley values will then assume that both have contributed equally to the successful surgery. Which isn't true, because surgeons are much less available (less replaceable) than nurses. If the nurse gets ill, a different nurse could probably do the task, while if the surgeon gets ill, the surgery may well have to be postponed. So the surgeons are more important for (contribute more to) a successful surgery.
- adammarples 2y ago
  Clearly both are equally important, 100% necessary. This doesn't account for rarity, nor does it account for wages, agreeability, smell or any of the other things it isn't trying to measure. You'll need a different metric for that and if you want to take both into account you should.
  - cubefox 2y ago
    Shapley values try to measure importance of contributions, and for this, bare necessity isn't a sufficient indicator. I think it comes down to probability. The task of the surgeon is, from a prior perspective, less likely to be fulfilled because it is harder to get hold of a surgeon. Similarly: What what was the main cause of the match getting lit? The match being struck? Or the atmosphere containing oxygen? Both are necessary in the sense that if either hadn't occurred the match wouldn't be lit. But it seems clear that the main cause was the match being struck, because matches being struck is relatively rare, and hence unlikely, while the atmosphere contains oxygen pretty much always. So I think the contributions calculated for Shapley values should be weighted by the inverse of their prior probabilities. Though it is possible that such probabilities are not typically available in the machine learning context in which SHAP operates.
__mharrison__ 2y ago
SHAP gets a bad rap. I use them all the time for global and feature interpretation. They can be a great springboard for diving into your data and doing additional feature engineering, monotonic constraints, and leveraging regression based models.

by baq 2y ago

cubefox 2y ago

No wonder. "Shapley values" have the problem that they assume all necessary conditions are equally important. Say a successful surgery needs both a surgeon and a nurse, otherwise the patient dies. Shapley values will then assume that both have contributed equally to the successful surgery. Which isn't true, because surgeons are much less available (less replaceable) than nurses. If the nurse gets ill, a different nurse could probably do the task, while if the surgeon gets ill, the surgery may well have to be postponed. So the surgeons are more important for (contribute more to) a successful surgery.

adammarples 2y ago
Clearly both are equally important, 100% necessary. This doesn't account for rarity, nor does it account for wages, agreeability, smell or any of the other things it isn't trying to measure. You'll need a different metric for that and if you want to take both into account you should.
- cubefox 2y ago
  Shapley values try to measure importance of contributions, and for this, bare necessity isn't a sufficient indicator. I think it comes down to probability. The task of the surgeon is, from a prior perspective, less likely to be fulfilled because it is harder to get hold of a surgeon. Similarly: What what was the main cause of the match getting lit? The match being struck? Or the atmosphere containing oxygen? Both are necessary in the sense that if either hadn't occurred the match wouldn't be lit. But it seems clear that the main cause was the match being struck, because matches being struck is relatively rare, and hence unlikely, while the atmosphere contains oxygen pretty much always. So I think the contributions calculated for Shapley values should be weighted by the inverse of their prior probabilities. Though it is possible that such probabilities are not typically available in the machine learning context in which SHAP operates.

__mharrison__ 2y ago

SHAP gets a bad rap. I use them all the time for global and feature interpretation. They can be a great springboard for diving into your data and doing additional feature engineering, monotonic constraints, and leveraging regression based models.