His argument seems to be more about the limited utility of GenAI, specifically LLMs (vs image generation), than about more efficient AI techniques. We do seem t

His argument seems to be more about the limited utility of GenAI, specifically LLMs (vs image generation), than about more efficient AI techniques. We do seem to be at peak LLM hype right now, with C-suite execs thinking of "GenAI" as some sort of proto-AGI that can do magic, and OpenAI/etc still claiming that AGI is just around the corner if only investors will foot the bill for $100B data centers to train ever bigger models. It's not clear what level of datacenter/GPU investment is needed for inference vs training - all the talk of massive GPU clusters seems to be about training needs, not inference. As far as efficiency, presumably we'll eventually switch to smarter (brain-like) dataflow type designs with incremental learning, but for time being we're stuck synchronously pumping 100K contexts through hundreds of transformer layers, and trillion token pre-training runs. Brute force indeed!