4 ms·
I'm confused. I agree that overfitting can lead to very bad models. But, what I don't understand is that I thought that "linear" in ML contexts was normally us
by EE84M3i 6y ago
I'm confused. I agree that overfitting can lead to very bad models.
But, what I don't understand is that I thought that "linear" in ML contexts was normally used in the sense of 'linear transformations', which is a sense of linear that 'line-fit' from excel isn't -- it's affine.
Is a linear model with thousands/millions of weights/parameters (like deep learning models) really substantially simpler to understand? Can it do anything useful?
[1]: https://en.wikipedia.org/wiki/Linear_map https://en.wikipedia.org/wiki/Linear_map
- cjhanks 6y agoI suppose from the perspective of someone implementing these models, yeah - it is linear, but it is not bijective. In a system with only one layer, that manifests as an alias (assuming the output dimensions are smaller). In a system with multiple layers of either `N->M` or `M->N`, those aliases tend to manifest as apparent "non-linearities". So, I guess looking from the bottom up the system may look non-continuous and linear. But if you look from the top down, it would look continuous and non-linear. Really, I am not sure which one is "true".