4 ms·
Can you elaborate? What other techniques do you find useful or better for either feature extraction or dimensionality reduction?
by shreyshnaccount 5y ago
Can you elaborate? What other techniques do you find useful or better for either feature extraction or dimensionality reduction?
- time_to_smile 5y agoThe entire field of deep learning is essentially performing non-linear dimensionality reduction, ultimately projecting the data on to a manifold where it is linearly separable. The last layer of most neural networks is essentially just logistic regression, so if you take that final layer before the soft max you have a lower dimensional representation of the original data. You can extend this things to more sophisticate example where you're not just learning a target but some other more general objective. For example you might want to learn a representation that minimizes the cosine distance between objects in a similar category. PCA is only useful for compression (which is rarely a problem in contemporary applications) as far as pure feature engineering goes. However when you realize that PCA is the same as a linear auto-encoder then it does serve as a good intellectual starting point for more sophisticated dimensionality reduction techniques.
- naijaboiler 5y ago> The entire field of deep learning is essentially performing non-linear dimensionality reduction, ultimately projecting the data on to a manifold where it is linearly separable. I will generalize this to every data problem. They all essentially boil down to reducing dimensions and projecting to a space where they are linearly separable. I was trying to show a junior colleague that this is essentially what any and every approach is doing
- shreyshnaccount 5y agoNYUs online deep learning course does a good job explaining the first two paragraphs.