4 ms·
> (note: a decision tree is usually not explicitly trying to model the probability of each sample belonging to class X or Y). I realize that. Maybe my question
by radarsat1 4y ago
> (note: a decision tree is usually not explicitly trying to model the probability of each sample belonging to class X or Y).
I realize that. Maybe my question wasn't clear. I was specifically trying to ask if there exist interesting or useful models beyond standard decision trees that might do so.
- fjkdlsjflkds 4y agoI think I got your question right (but perhaps I was not totally clear in my answer). In a nutshell: sure, there are lots of such possibly useful models that use trees (you can even come up with new ones yourself). Random forests or GLM regression trees are two examples. My point is that, because standard decision trees do not attempt to explicitly model probabilities (they just try to partition the data in "homogeneous groups", in some sense), you have no pre-baked assurances that your "score" will actually be well calibrated (i.e., no assurances that it will actually be a good predictor of the raw probability/confidence). Sure, you could come up with some "decision tree"-like (i.e., partitioning algorithm) Bayesian method that would (maybe) ensure reasonable calibration of "confidence"... but, at that point, are you even using "decision trees" anymore? Probably not.