4 ms·
> not any sort of fundamental algorithmic paradigm shift I think I cannot agree with this. There has been a lot of improvements to the algorithms to solve prob
by jorgemf 9y ago
> not any sort of fundamental algorithmic paradigm shift
I think I cannot agree with this. There has been a lot of improvements to the algorithms to solve problems and the pace has speed up thanks to GPUs. You just cannot make a neural network from 15 years ago bigger and think it is going to work with modern GPUs, it is not going to work at all. Moreover, new techniques have appeared to solve other type of problems.
I am talking about things like batch normalization, RELUs, LSTMs or GANs. Yes, neural networks still use gradient descent, but there are people working on other algorithms now and they seem to work but they are just less efficient.
> This is similar to how the integrated circuit enabled the personal computing "revolution" but down at the transistor level, it's still using the same principles of digital logic since the 1940s.
This claim has exactly the same problem as before. You can also say evolution has done nothing because the same principles that are in people they were there with the dinosaurs and even with the first cells. We are just a lot more cells than before.
- Amezarak 9y agoDo you have any suggestions on reading material?
- davidcuddeback 9y agoAssuming you're asking specifically about the history of DL/ML, I can recommend this (4-part) blog series: http://www.andreykurenkov.com/writing/ai/a-brief-history-of-neural-nets-and-deep-learning/ http://www.andreykurenkov.com/writing/ai/a-brief-history-of-.... It includes references for the relevant publications that identify problems (e.g., exploding gradients) and their solutions.
- jorgemf 9y agoI have read a lot of papers and usually, you end up in the original one if you check the references. But you have to check papers in specific areas. I am not aware of any good document where everything is there.