>> All this to say: I think there are fundamentals we need to view all of this progress through. Lower-bound complexities of problems that we learn in Algorithm

>> All this to say: I think there are fundamentals we need to view all of this progress through. Lower-bound complexities of problems that we learn in Algorithms 101 shouldn’t get thrown out the window because linear algebra, ReLUs and attention mechanisms are involved. Sorting, for example, has a provable lower-bound complexity of Ω(NlogN). When someone tells you there is a way to do it in O(logN) time, it’s best to squint and ask a few questions. When someone tells you you can gain a speed-up in computing primes, on a naive algorithm, it’s best to ask if that method also has correctness covered. Keep this thought please. There is a very obvious feeling that "now we have LLMs we don't need CS anymore", or don't need maths, science, knowledge... In fact, this goes rather further back than Transformers. I was just reading this old interview of Vladimir Vapnik [1], where he's taking the position that only the "easy" questions in science have explanatory answers, and that for the "hard questions" we can only hope to build predictive models. Vapnik is one of my herores of AI, but on that I think he's not just wrong, but pointing the way to a new dark ages, where we lose the little knowledge we have. Similar "intuitions" are expressed in the "Bitter Lesson" blogpost by Rich sutton [2]. Vapnik is right, of course, in that there are "hard" questions for which we don't know any good answers, and our current framework may not be enough to give any simple, explanatory answer anyway. Certainly, CS is full of bottomless pits of despair, undecidabilities, incompletenesses and intractabilities. I think that just says we're shit at creating formal frameworks. All the ones we've created have some flaw. Arithmetic is incomplete. Logic (even first-order logic) is undecidable. You can't divide by 0, there are tons of calculations that cannot be performed because they sink into uncountable infinities after two steps, etc etc and so on. We just suck at making universal abstractions, that are useful in the first instance, but don't end up painting ourselves into a corner, somewhere down the line. Well, so what? So far we've done pretty well in terms of science, in the sense of understanding the world and how it works. We're not going to give up now! If our tools are not good enough to take our knowledge to the next level, well, then, we just need to make better tools. Better maths. Better Computer Science. Better scientific methods. That has happened in the past at least once, if not many many times. Why can we not do it again? Why do we have to abandon our fate to unknowing, all-predicting machines? That doesn't make sense. ______________ [1] http://www.learningtheory.org/learning-has-just-started-an-interview-with-prof-vladimir-vapnik/ http://www.learningtheory.org/learning-has-just-started-an-i... [2] http://incompleteideas.net/IncIdeas/BitterLesson.html http://incompleteideas.net/IncIdeas/BitterLesson.html