5 ms·
> We can see that the scheme that uses sequential batching actually performs worse than the CPU alone, whereas the new approach using DuHL achieves a 10× speed-
by web007 9y ago
> We can see that the scheme that uses sequential batching actually performs worse than the CPU alone, whereas the new approach using DuHL achieves a 10× speed-up over the CPU.
I had to get down to the graph to realize they're talking about SVM, not deep learning.
This could be pretty cool. Training a SVM has usually been "load ALL the data and go", and sequential implementations are almost non-existent. Even if this was 1x or 0.5x speed and didn't require the entire dataset at once it's a big win.
- jamesblonde 9y agoYes, i felt cheated when i read it was about training 1/10th of ImageNet on a SVM. I guess IBM are desparate not to be left behind in the race for distributed deep learning platforms.
- pdeburen 9y agoYeah they get you really hyped up first and drop the bomb at the end. Still quite impressive speedup, would be better though to show it on a benchmark where SVM's are used in practice
- carlmr 9y agoLooking at the normal stuff coming out of IBM they're not associated with good software in my mind. So the more outrageous their claim, the less I believe them. They need to earn a reputation first.
- hueving 9y agoThat's like judging all of Google based on the quality of one product. With 10x as many employees as Google and a very loose organization, expecting any kind of reputation is folly. A product from an IBM consultant is about as related to a product from IBM Watson as is a product from Microsoft being related to a product from Apple.
- kayoone 9y agoIBM has 400k employees and god knows how many subsidiaries and divisions, do you really think you can paint them all with one brush because of some negative experience you had with one of their products ?
- carlmr 9y agoNot one, we have multiple IBM software products in my company, and they're all consistently the most terrible software you can imagine making. Sure they might have some divisions that do better, but I have yet to see them.
- samfriedman 9y agoTo be honest, I'd readily cheer any groups working on traditional machine learning advancements despite all the current hype for neural methods.
- madenine 9y agoI'll second that. For all the attention DL/ANNs get... there's still a lot of legwork going on out there using linear models, basic trees, etc. IIRC this years kaggle survey ranked Logistic Regression as the #1 most used model by a long shot.
- make3 9y agoneural networks are stacked logistic regressions. a lot of the deep learning research benefits logistic regression
- make3 9y agoyou say that as if it's empty hype. but it's not: deep learning works, and works much better by any reasonable metric than SVMs in most problems that require high to very high model capacity.
- nlperguiy 9y agoI know two projects (fairly simple implementation) of sequential SVM. I believe vowpal wabbit can also do max-margin optimization. http://leon.bottou.org/projects/lasvm http://leon.bottou.org/projects/lasvm http://leon.bottou.org/projects/sgd http://leon.bottou.org/projects/sgd
- Barrin92 9y ago>I had to get down to the graph to realize they're talking about SVM, not deep learning. there's still a ton of usage for classical learning algorithms. I'd be a very happy camper if we could speed SVMs up by a magnitude