> All I am saying is that there are other strategies to explore The “other strategies” have been completely sidelined. Even back when we had no transformers, o

> All I am saying is that there are other strategies to explore The “other strategies” have been completely sidelined. Even back when we had no transformers, only CNNs and RNNs, the other strategies were tossed aside because they didn’t show much promise in cifar or mnist or whatever benchmarks the NN community deemed the most suitable sota. Even before that, when we only had decision trees and catboost, people had already dismissed loess/splines etc. You can get a flavor of the dismissals here - https://news.ycombinator.com/item?id=19145706 https://news.ycombinator.com/item?id=19145706 At this point that horse has left the barn.