At the same time, I think it's astounding how often k-nearest neighbors (a method considerably simpler than SVMs) is either the best or second-best classifier f

At the same time, I think it's astounding how often k-nearest neighbors (a method considerably simpler than SVMs) is either the best or second-best classifier for a dataset. I'm not trying to pit SVMs vs. neural networks...rather I'm saying most methods of machine learning (neural nets, SVMs, gaussian mixture, kNN, what have you) can be effective and none stand out above the rest.