4 ms·

I think the situation is more complex than the author states. For example if I have a linear model, Y = a + b * X, I will choose a and b to minimize in-sample

I think the situation is more complex than the author states. For example if I have a linear model, Y = a + b * X, I will choose a and b to minimize in-sample fit. Choosing a and b to maximize out of sample fit goes against all theory. However, if I want to choose which parameters go into my model, maximizing out of sample fit would be a good approach. So at the end of the day, there is not a huge philosophical difference between using in-sample and out-of-sample fit, only different approaches to the same problem. In both cases, the assumption is (usually) the the data is i.i.d., and in both cases, you are choosing some coefficients/parameters/hyperparameters with the intent of maximizing out of sample fit, but using different methods.

nkurz 13y ago
Are you coming from a theoretical math point-of-view or background? It's hard for me to say exactly why, but I feel your response is evidence of just that "huge philosophical difference" between traditional stats and machine learning. To me, even the statement "if I have a linear model" makes very little sense from the perspective of ML. Contrast with "if I think I'm dealing with a situation where a linear model might offer a good fit". Regarding "maximizing out of sample fit would be a good approach", I think ML is always and just-about-only concerned with maximizing out-of-sample fit, for if it wasn't, the solution would be a lookup table. I'm not trying to imply that you're wrong, rather that I think the 'gulf' is real. Or maybe I'm misunderstanding your point. For example, I feel that mjw's comment in this thread captures my view, which I think is more ML centered: https://news.ycombinator.com/item?id=6878336 https://news.ycombinator.com/item?id=6878336 Is that comment also in accord with your view, and it's me that's on the wrong side of that gulf?

nkurz 13y ago

Are you coming from a theoretical math point-of-view or background? It's hard for me to say exactly why, but I feel your response is evidence of just that "huge philosophical difference" between traditional stats and machine learning. To me, even the statement "if I have a linear model" makes very little sense from the perspective of ML. Contrast with "if I think I'm dealing with a situation where a linear model might offer a good fit". Regarding "maximizing out of sample fit would be a good approach", I think ML is always and just-about-only concerned with maximizing out-of-sample fit, for if it wasn't, the solution would be a lookup table. I'm not trying to imply that you're wrong, rather that I think the 'gulf' is real. Or maybe I'm misunderstanding your point. For example, I feel that mjw's comment in this thread captures my view, which I think is more ML centered: https://news.ycombinator.com/item?id=6878336 https://news.ycombinator.com/item?id=6878336 Is that comment also in accord with your view, and it's me that's on the wrong side of that gulf?