I've always heard that argument in favor of decision trees or random forests, yet those decision trees had 400k nodes :). So no one ever really looked at them,

I've always heard that argument in favor of decision trees or random forests, yet those decision trees had 400k nodes :). So no one ever really looked at them, but in theory you can could check the long node paths doing arbitrary splits on weird features :). Apart from that, the strength of DNNs is exactly that complex decision making compared to, say, the simple algorithms physicians learn and manually apply for diagnosis. Those are obviously vastly underfitting in many cases.