+1 on calling this BS, even though I think it is only partly BS: While it is true that training very large language models is very expensive, pre-trained model

+1 on calling this BS, even though I think it is only partly BS: While it is true that training very large language models is very expensive, pre-trained models + transfer learning allows interesting NLP work on a budget. For many types of deep learning a single computer with a fast and large memory GPU is enough. It is easy to under appreciate the importance of having a lot of human time to think, be creative, and try things out. I admit that new model architecture research is helped by AutoML, like AdaNet, etc. and being able to run many experiments in parallel becomes important. Teams that make breakthroughs can provide lots of human time, in addition to compute resources. There is another cost besides compute that favors companies: being able to pay very large salaries for top tier researchers, much more than what universities can pay. To me the end goal of what I have been working on since the 1980s is flexible general AI, and I don’t think we will get there with deep learning as it is now. I am in my 60s and I hope to see much more progress in my lifetime, but I expect we will need to catch several more “waves” of new technology like DL before we get there.