2 ms·
I think I can help with that. The article nicely explains the data transformation so that it becomes linearly separable. But the trick to the kernel trick is n
by jwr 10y ago
I think I can help with that.
The article nicely explains the data transformation so that it becomes linearly separable. But the trick to the kernel trick is no to transform the data at all.
What you do is use a learning algorithm that doesn't need individual input vectors, but instead only needs their dot products. You then imagine a magical high-dimensional space where your data is (you suppose) linearly separable. The trick is that you never actually transform your data to that magical space — you don't need the input vectors, remember? You only need their dot products. So you define a function that given two vectors in your normal input space returns a scalar. Assuming your function behaves in a sane way (go read about the required properties if you need to), you can think of this function as a dot product. In some kind of magical space — you don't actually care much. You will never transform your data, it might not even be possible to: the most common gaussian kernel is defined over an infinite-dimensional space. But hey, who cares? You take your SVM, give it your kernel function and input data, and off it goes, working as usual, except your dot products are no longer computed in your input space, but in your magical infinite-dimensional space.
It's both really clever and really simple.