Great writeup, thanks! That Kadinsky quote is what set off alarm bells for me, as it seems like a quintessential failure case for laypeople understanding LLMs -

Great writeup, thanks! That Kadinsky quote is what set off alarm bells for me, as it seems like a quintessential failure case for laypeople understanding LLMs -- they take some basic, vague insights produced by a chatbot as profound discoveries. It seems the reviewers may have agreed, to some extent; note that it was received by Machine Learning 24-03-26, but only accepted (after revisions) on 24-08-21. I wrote more below with a quote, but re: "who's the intended audience?" I think the answer is the same kind of people Gary Marcus writes for: other academic leaders, private investors, and general technologists. Definitely not engineers looking to apply their work immediately, nor the vast majority of scientists that are doing the long, boring legwork of establishing facts. In that context, I would defend the paper as evocative and creative, even though your criticisms all ring true. Like, take a look at their (his?) HuggingFace repo: https://huggingface.co/lamm-mit https://huggingface.co/lamm-mit It seems clear that they're doing serious work with real LLMs, even if it's scattershot. Honestly, if I was a prestigious department head with millions at my disposal in an engineering field, I'm not sure I would act any differently! ETA: Plus, I'll defend him purely on the basis of having a gorgeous, well-documented Git repo for the project: https://github.com/lamm-mit/GraphReasoning?tab=readme-ov-file https://github.com/lamm-mit/GraphReasoning?tab=readme-ov-fil... Does this constitute scientific value on its own? Not really. Does it immediately bias me in his favor? Absolutely!