You ever read something that just leaves you confused as to how two perspectives can be so different? I mean, from down in the trenches, AI is doing great. In t

You ever read something that just leaves you confused as to how two perspectives can be so different? I mean, from down in the trenches, AI is doing great. In the last couple of years, context window sizes have increased to reasonable limits, ring attention has vastly improved needle-in-a-haystack performance, prices have come down, open source models are close to GPT-3.5 performance making local inference viable. And that's just LLMs; embedding models and vector databases are better and more available, generative voice actually sounds human, generative images and video are coming along. It's never been cheaper or easier to rent GPUs in the cloud. Two years ago, if data was in unstructured free text or in an image it would be a massive challenge to get any value out it and often not worth the effort; now, it's nearly trivial. How could anyone look at this and conclude, "AI is losing steam?" I read it a couple of times and I think I figured it out. I'm looking at it as a technologist, while the WSJ is looking at it as an investment opportunity. When they say "losing steam" they mean "there's no alpha left." In other words, the smart money has already moved in, and the future value has already been priced in. This is a really telling quote: > AI could become a commodity That is a GOOD thing. It's called the ephemeralization of value[1]. Today, a HD color TV costs less than a black and white TV in 1950's. The WSJ would look at that and see a mere "commodity" because businesses are operating at lower margins and there's less money to be made, but billions of consumers benefit. Open source is the ultimate expression of ephemeralization, and it is inarguably a good thing when billions of people can benefit from state-of-the-art tech for free. So yeah, maybe it's too late to get rich by investing early. Maybe a lot of the vaporware startups and thin wrappers around OpenAI's API will crash and burn soon. And yes, it's definitely been overhyped in certain regards and pushed into use cases it can't really support by promoters who neither know nor care what about the real capabilities and limitations. All of that in normal adoption curve stuff. The technology is definitely not losing steam; it's barely getting started. If you're a CS student, you 100% should be learning about vectorization, automatic differentiation, and differentiable programming, because the GPU isn't some sort of niche topic you can ignore. We've taken the von Neumann architecture as far as it can go, right up to the quantum limits, and now we have go parallel. These techniques are pointing the way to a simple model of parallelization that can actually take advantage of modern hardware without breaking your brain or getting lost is a maze of locks and semaphores. The "transformer" isn't the end-all-be-all of this paradigm - it just happens to be one technique that worked well in practice on free text data. There's so much more out there waiting to be discovered. [1]: https://en.wikipedia.org/wiki/Ephemeralization https://en.wikipedia.org/wiki/Ephemeralization