Well, obviously this thread wasn't available when I asked since I was one of the first commenters. But: (1) You're right that it's a hard experiment to do now

Well, obviously this thread wasn't available when I asked since I was one of the first commenters. But: (1) You're right that it's a hard experiment to do now because LLMs can search the web, on the other hand... (2) LLMs can search the web, which cuts against the author's implicit premise that LLMs can only know what is in their training data. LLMs have access to tools and oracles. But also... (3) I checked gemma3:12b running in Ollama. It can not search the internet. But it also knew about fatty acid combustion in the brain and recapped the research history on it. Now obviously I expect gemma3 to be more prone to hallucination since it's a weaker model and doesn't have any safeguards a production model would have. Also it's working entirely from memory. But I feel comfortable concluding that the author overestimates the novelty of the study and underestimates LLMs. The study rewrites the Bayesian weights on various interpretations of fatty acid combustion results in the brain. It doesn't completely propose and prove an unheard of hypothesis. Even the offline Gemma told me that glucose-only metabolism used to be the dogma etc etc. But that's also true in general about science. Breakthroughs are always building on things that came before. And the LLM knows everything that came before.