3 ms·
All I am saying is that there are other strategies to explore. It's ok to have deep learning. But the herding towards deep learning is just too much. Recall t
by max_ 2y ago
All I am saying is that there are other strategies to explore.
It's ok to have deep learning. But the herding towards deep learning is just too much.
Recall that before Hinton and his group presented thier record breaking ImageNet results, deep learning was a fringe technique that the likes of Marvin Minsky regarded as a failure in "Perceptions"
Scientific progress is made by researchers exploring other alternatives.
Herding can only slow down progress.
- jerpint 2y agoThe imagenet results’ “main” contribution was showing that scaling the compute on large datasets worked surprisingly well. They didn’t contribute any new models as much as they showed that scaling is what helped these models learn and obtain SOTA. It’s no surprise that industry has been pushing for more compute ever since.
- andsoitis 2y ago> All I am saying is that there are other strategies to explore. Are people not exploring other strategies?
- max_ 2y agoDo you know anyone in AI doing research on something that isn't Neural Networks? They are very few. Too few.
- dxbydt 2y ago> All I am saying is that there are other strategies to explore The “other strategies” have been completely sidelined. Even back when we had no transformers, only CNNs and RNNs, the other strategies were tossed aside because they didn’t show much promise in cifar or mnist or whatever benchmarks the NN community deemed the most suitable sota. Even before that, when we only had decision trees and catboost, people had already dismissed loess/splines etc. You can get a flavor of the dismissals here - https://news.ycombinator.com/item?id=19145706 https://news.ycombinator.com/item?id=19145706 At this point that horse has left the barn.
- lostmsu 2y ago> Herding can only slow down progress. You don't know that either. What if there are actually no alternatives.