SVM comes from a theory called Vapnik–Chervonenkis dimension which talks about ways to balance best learner performance against likelihood of actually achieving

SVM comes from a theory called Vapnik–Chervonenkis dimension which talks about ways to balance best learner performance against likelihood of actually achieving that theoretical optimum. It's very nice theory. The nonlinear stuff is called the "kernel trick" and is essentially a recognition that an SVM's operation is depends on the data exclusively through measurements of how far apart the n^2 data points are from one another. Once you realize that you can twist the definitions around so as to insert whatever kind of non-linear notion of "distance" you like. Many ML algorithms can be augmented by the kernel trick. Finally, SVM's geometry allows you to think of it as a relatively nice optimization problem. Compare this to many other ML algorithms which have only heuristic optimization methods. It makes the algorithm surprisingly fast.