3 ms·
What's interesting is that Jeff Dean noted that ReLU was one of the big things why deep learning took off. Wikipedia summarizes tha advantages, but my guess is
by xiphias 10y ago
What's interesting is that Jeff Dean noted that ReLU was one of the big things why deep learning took off. Wikipedia summarizes tha advantages, but my guess is getting rid of the vanishing gradients problem:
https://en.m.wikipedia.org/wiki/Rectifier_%28neural_networks%29?wprov=sfla1 https://en.m.wikipedia.org/wiki/Rectifier_%28neural_networks...
- cr0sh 10y ago> Wikipedia summarizes tha advantages, but my guess is getting rid of the vanishing gradients problem Hmm - didn't think about that aspect, but that would be a big plus for backprop as I understand it! Thanks for the insight, and the link.