3 ms·
How much of this is solely due to the 1-bit gradient descent? It's a genuine breakthrough and MSR (or whoever came up with it) deserves all the credit, but assu
by Analemma_ 9y ago
How much of this is solely due to the 1-bit gradient descent? It's a genuine breakthrough and MSR (or whoever came up with it) deserves all the credit, but assuming it's not patented I imagine Google will be adding it to TF sooner or later and that will close most of the gap.
EDIT: My bad, I did not see at the end of the article that 1-bit SGD is not enabled on Keras yet, so the performance wins are coming from somewhere else. Neato.
- minimaxir 9y agoThe container specifically uses the non-1-bit SGD version CNTK after I learned it does not work, just to be sure. MSFT employees talk about LSTM gains on the original submission: https://news.ycombinator.com/item?id=14473255 https://news.ycombinator.com/item?id=14473255
- IshKebab 9y agoLast time I checked 1-bit gradient descent is one of the only bits of CNTK that is closed source. I expect it is patented.
- deleted 9y ago[deleted]