It's more than that. The SVM kernel maps your multi-dimensional data into an infinite dimensional data. Because of the way the math works you can essentially

It's more than that. The SVM kernel maps your multi-dimensional data into an infinite dimensional data. Because of the way the math works you can essentially learn from an infinite dimensional data without overfitting. The support vector is the data points that "support" the separating hyperplane, that is the points that meet the constraints. The other thing about SVMs is that they are computationally friendly. Just as a simple example (stolen from the Caltech course which I highly recommend) if you look at points on a plane that form a circle and try to separate them with a line you're going to fail. I.e. your points are (x,y) and those in the circle are your fake dollar bill and those outside aren't. But you can take all these points and apply a non-linear transform, e.g. (x, y, xy, xxy, yyx, x+y, xx+y2), you get the idea... It turns out that now you can* separate the data into what's inside and outside the circle. The problem is you just increased your so called VC dimension of the model and you might overfit the data and not learn anything. SVMs let you get infinite numbers of combinations, without overfitting and with cheap calculations... Pretty neat.