4 ms·
Regarding the PCA before SVM step - PCA would help reduce dimensionality w.r.t linear correlations, kernels generally are used to obtain non-linear maps. So wou
by govg 9y ago
Regarding the PCA before SVM step - PCA would help reduce dimensionality w.r.t linear correlations, kernels generally are used to obtain non-linear maps. So wouldn't it be wrong to say doing PCA is redundant? Also, while kernel SVM optimization is dominated by number of data points, computing the value of the kernel function for individual data points might depend on the dimensionality itself, would it not? Another instance of lower dimensional representation possibly helping.
Please correct me if I'm wrong, I'm still just learning this stuff.
- screye 9y agoThe cost of computing similarities is definitely linear or lower, wrt. to features. But overall, the Kernel SVM will scale to the order of n^3 wrt. to the number of points. In that sense, focussing on the number of features may not make a lot of sense. PCA isn't lossless either. We are definitely trading off some richness in the data for efficiency. When the gain in efficiency is negligible (as is the case with SVMs) the trade off doesn't look as tempting. I am student in the area as well, so take my words with appropriate caution.