3 ms·
Auto-encoders have some big advantages over PCA, but they suffer the same shortcoming (sensitivity to units of measurement) described in the original post.
by dbecker 13y ago
Auto-encoders have some big advantages over PCA, but they suffer the same shortcoming (sensitivity to units of measurement) described in the original post.
- radarsat1 13y agoIn practice you have to scale data to [-1,1] or [0,1] for a neural net anyway, right? (Depending on the kernel function.)
- SatvikBeri 13y agoAnother common method is to scale to mean 0, variance 1. In my opinion this makes more sense since it handles outliers a bit better-e.g., consider a case where most of your values for a feature are scaled from 1 to 10 but there's one point with value 1,000,000.
- dbecker 13y agoI agree that you SHOULD normalize/scale data before running neural networks and autoencoders.... and this resolves the units issue in most cases (unless measurements in some units are non-linear functions of measurements in others). But this scaling also resolves the issue for PCA. So, I don't see much difference between autoencoders and PCA with regards to original post's "dimensional invalidity" concern. If anything, the scaling options you mention suggest "dimensional invalidity" isn't a big deal in practice for either method.