3 ms·
Didn't Schmidhuber's lab predate all of those in their inventions except ReLU (which I've heard is now out of favor as compute outpaces memory bandwidth and oth
by cma 3y ago
Didn't Schmidhuber's lab predate all of those in their inventions except ReLU (which I've heard is now out of favor as compute outpaces memory bandwidth and other activation functions have better properties).
- chubot 3y agoYeah I should have mentioned him too as evidence of the point!