3 ms·
IMO, kernel-based computational methods are by far the most important overlooked advances in the statistical sciences. Kernel methods are linear methods on dat
by uoaei 5y ago
IMO, kernel-based computational methods are by far the most important overlooked advances in the statistical sciences.
Kernel methods are linear methods on data projected into very-high-dimensional spaces, and you get basically all the benefits of linear methods (convexity, access to analytical techniques/manipulations, etc.) while being much more computationally tractable and data-efficient than a naive approach. Maximum mean discrepancy (MMD) is a particularly shiny result from the last few years.
The tradeoff is that you must use an adequate kernel for whatever procedure you intend, and these can sometimes have sneaky pitfalls. A crass example would be the relative failure of tSNE and similar kernel-based visualization tools: in the case of tSNE the Cauchy kernel's tails are extremely fat which ends up degrading the representation of intra- vs inter-cluster distances.
- teshier-A 5y agoMy experience with MMD is that unless you've been using it and are familiar with other kernel methods, you probably won't know what to do with it : what kernel do I use ? how can I test for significance (in any sense of the word) ? Add the (last I checked) horrendous complexity, to me it looks like a less usable mutual information (or KL div) and without all the nice information theory around it.
- NavinF 5y agoI see a ton of papers about kernel-based stuff, but zero real applications. Feels like that whole subfield came and went before I got into ml.