SGD with averaging [1] :) More seriously, SGD is pretty hard to beat for fitting linear models (SVM, logistic regression and other l1 penalized models with var

SGD with averaging [1] :) More seriously, SGD is pretty hard to beat for fitting linear models (SVM, logistic regression and other l1 penalized models with various loss functions) when the number of samples getting large. [1] http://leon.bottou.org/projects/sgd http://leon.bottou.org/projects/sgd