3 ms·
If LLMs just regurgitate compressed text, they'd fail on any novel problem not in their training data. Yet, they routinely solve them, which means whatever's ha
by adamtaylor_13 8mo ago
If LLMs just regurgitate compressed text, they'd fail on any novel problem not in their training data. Yet, they routinely solve them, which means whatever's happening between input and output is more than retrieval, and calling it "not understanding" requires you to define understanding in a way that conveniently excludes everything except biological brains.
- varispeed 8mo agoThey don't solve novel problems. But if you have such strong belief, please give us examples.
- ainch 8mo agoDepends how precisely you define novel - I don't think LLMs are yet capable of posing and solving interesting problems, but they have been used to address known problems, and in doing so have contributed novel work. Examples include Erdos Problem #728[0] (Terence Tao said it was solved "more or less autonomously" by an LLM), IMO problems (Deepmind, OpenAI and Huang 2025), GPT-5.2 Pro contributing a conjecture in particle physics[1], systems like AlphaEvolve leveraging LLMs + evolutionary algorithms to generate new, faster algorithms for certain problems[2]. [0] https://mathstodon.xyz/@tao/115855840223258103 https://mathstodon.xyz/@tao/115855840223258103 [1] https://huggingface.co/blog/dlouapre/gpt-single-minus-gluons https://huggingface.co/blog/dlouapre/gpt-single-minus-gluons [2] https://deepmind.google/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/ https://deepmind.google/blog/alphaevolve-a-gemini-powered-co...
- sfn42 8mo agoYes there are some fascinating emergent properties at play, but when they fail it's blatantly obvious that there's no actual intelligence nor understanding. They are very cool and very useful tools, I use them on a daily basis now and the way I can just paste a vague screenshot with some vague text and they get it and give a useful response blows my mind every time. But it's very clear that it's all just smoke and mirrors, they're not intelligent and you can't trust them with anything.
- pennomi 8mo agoWhen humans fail a task, it’s obvious there is no actual intelligence nor understanding. Intelligence is not as cool as you think it is.
- atomicthumbs 8mo agoyou'd think with how often Opus builds two separate code paths without feature parity when you try to vibe code something complex, people wouldn't regard this whole thing so highly
- otabdeveloper4 8mo ago> they'd fail on any novel problem not in their training data Yes, and that's exactly what they do. No, none of the problems you gave to the LLM while toying around with them are in any way novel.
- adamtaylor_13 8mo agoNone of my codebases are in their training data, yet they routinely contribute to them in meaningful ways. They write code that I'm happy with that improves the codebases I work in. Do you not consider that novel problem solving?
- otabdeveloper4 8mo agoCorrect, you are not doing any novel problem solving.
- fennecbutt 8mo agoI somewhat agree with you but I also realise that there are very few "novel" problems in the world. I think it's really just more complex problem spaces is all. Same relative logic, just more of it/more steps or trials.