The energy used by the human brain is highly variable because the brain has been optimized to reduce power use. This was a going concern in the ancestral enviro

The energy used by the human brain is highly variable because the brain has been optimized to reduce power use. This was a going concern in the ancestral environment. It has not been a great concern in the developmental environment of LLMs, so they are not optimized towards this. There are in fact many papers on how to optimize energy use of LLMs, and some amount of them may have already been deployed. You note this is offtopic, but I disagree. This is part of your fundamental misunderstanding of how to make good use of LLMs - and how LLMs make good use of context space. > The point is that if I ask you to do some complex task that requires a one-word answer, you will take a long time to think about it. If I ask an LLM, it will always take the same amount of time, regardless of the task. This is simply not the case. I will take a long time to think about it, because my brain is generating tokens in the conscious workspace. I will take roughly the same time for every token, same as a LLM. (I believe that's what brain rhythms are about.) However, as opposed to a LLM, I can do computation without echoing it to stdout. If you stuck a probe in my conscious workspace, I believe you would find it looking suspiciously similar to the output stream of a multimodal LLM! And there are several papers to the extent of - hey, LLMs do in fact give better answers if you let them insert filler words, or even dots or spaces or invisible tokens - because they learn to use the additional time you give them to -- think. Cough just like us cough. That's why all these tasks that tell the LLM to "not output any extraneous information but only give the answer" are so dumb and misleading. They're literally telling the LLM to blurt out the first thing that comes to mind. In other words, you think the context window is speech, which is why you are confused. The context window is the LLM's axis of time. That their time is conflated with speech is the primary reason why LLMs have a reputation for being glib, surface speakers - and why they continue to underperform. They literally cannot stop to think - not because of any fundamental inadequacy of the technology, but just because that's how we've set them up. I suspect the amount of boilerplate text given before an answer correlates directly with quality of the answer. That's why the energy is misleading - the computational work isn't just the token, but every token between the point where the answer could be computed and where the LLM was forced to commit to a final answer. And as one could expect from this, if you give the LLM time to think without speaking, their performance shoots up significantly. The moment somebody at OpenAI reads the QuietSTaR paper (GPT-2 scale network with GPT-3 quality!) and understands what it means, the timer to AGI begins.