3 ms·
Interestingly I thought ILP is integer linear programming. (The first hit for ILP algorithm.) And then I read DeepMind's delta-ILP post you linked and was even
by pas 8y ago
Interestingly I thought ILP is integer linear programming. (The first hit for ILP algorithm.)
And then I read DeepMind's delta-ILP post you linked and was even more confused as that talked about gradient descent, and then looked at your Metagol example, and ...
So, decision trees are a great example, as they were the first thing I've encountered ~15 years ago about machine learning, and I think it's a first class ML citizen. But they are in their pure form absolutely vulnerable to noise.
And as soon as noise enters the picture it becomes optimization. And I think that's the model fitting that should be the primary concept. (I used curve-slash-model, but I should have used just model.) [And even if there's no "fitting", like with kNN, you still have data, and either you use a constant classification set of points, or you'll see bias, but then the question becomes what set to use.]
And you're right, that treating everything as X is folly, after all, it gives you no information, decreases no entropy. But I'm looking at it from the perspective of what's intelligence. And it seems to be always on this spectrum of how to optimize for something. (Making hard things easy, by learning rules/algorithms.) After all that's why are we really interested in ML, no?
> But you're missing lots of details and the result is actually hindering your undersanding. In a sense, by looking at everything from the point of view of "curve fitting" you're missing the forest for the um, trees.
If you were so kind to elaborate on this, I'm very interested. And in general, anything that comes to your mind about my ramblings! Thanks for your detailed comments!