3 ms·

How is that different from allowing non-axis-aligned splits?

• tlarkworthy 9y ago
  its all much of a muchness. You can use a linear function at the nodes to project the space before applying a >. However, all these tweaks tend to just consume CPU for marginal benefit. You can still appoximate a non axis aligned decision boundary with a series of stepwise aligned splits, albeit less compactly. But if your aim is to just generate a ton of non linear distance functions ... a la random forest, focusing on tree expressivity is the wrong thing. Why not spend your CPU cycles on more trees instead instead, which leads to smoother decision boundaries in general when mixing their responses. There is a ton of tweaks you can do, but unless you are expressly researching ML algorithms, the pragmatist should not care and just try a few options from whatever comes in scipy.
• srean 9y ago
  That is a very interesting question and one can go deep into the woods with that, not everything is known about this question. There are two distinct effects at play here, computational cost and learning efficiency. A brutally shortened response would be that searching for non-axis aligned splits in the nodes can be very expensive CPU wise. This addresses the first effect at play. Applying a random rotation on the other hand is quite cheap. The second effect at play is that ensembling loses its power when you optimize over oblique splits because of lost diversity. It comes down to a trade off.