3 ms·
> A coding agent solving a software-engineering task spends dozens of steps reasoning No. That's simple PR hype. Parrotry is not reasoning.
by chrisjj 4mo ago
> A coding agent solving a software-engineering task spends dozens of steps reasoning
No. That's simple PR hype. Parrotry is not reasoning.
- brookst 4mo agoThere is a certain amount of irony in your comment that I hope you appreciate.
- astro1234 4mo agoWhy not? I think there’s fairly strong evidence that there is something that convincingly looks like reasoning. I think anthropic has done some nice circuit tracing and mechanistic interpretability work on this for instance.
- chrisjj 4mo ago> there is something that convincingly looks like reasoning Convincing ... people who believe chatbots are intelligent? Well sure.
- astro1234 4mo agoI don’t mean to say you are wrong, I just mean to ask what criteria you would consider necessary to satisfy in order for you to consider a system to be reasoning.
- Planktonne 4mo ago> I think there’s fairly strong evidence that there is something that convincingly looks like reasoning. No one is disputing that, but there is an enormous difference between evidence of 'something that looks like a thing' and evidence of the thing itself.
- astro1234 4mo agoI tend to agree but to adjudicate that someone has to define what they consider reasoning to be.
- Planktonne 4mo agoCrucially it's the responsibility of those who are making the claim here to bring evidence and definitions. The other approach, very popular on HN, has led to endless carping by people making outlandish claims without evidence and then demanding that every term possible be defined before they can be dismissed out of hand. The claim that any two things are equivalent just because they look similar is a strong one, and the onus is on the person advancing it to bring evidence, not to insist that they are considered correct until it is disproved. That's how it works with the flat earth and religion and everything else; we shouldn't accept an unsupported conclusion just because some people really, really like AI and don't want to have to justify their claims. The traditional response to such claims is ridicule [1]; we know trivially from all sorts of examples that presentation is not identity, and so 'but it looks similar' is not a convincing stance. 1 https://sites.psu.edu/sierraastle/2019/10/21/behold-a-man/ https://sites.psu.edu/sierraastle/2019/10/21/behold-a-man/
- rlt 4mo ago> Crucially it's the responsibility of those who are making the claim here to bring evidence and definitions Ok, I claim that if something draws reasonable conclusions to questions it hasn't previously seen by performing steps that look like reasoning, then it is reasoning. If you disagree then please define reasoning.
- Planktonne 4mo agoThis is actually a perfect example of what I was referring to in my earlier comment; the strident [1] and rather aggressive insistence that your unsupported belief be proved wrong, rather than you providing literally any evidence that two distinct things are identical other than the assertion itself. You're making a strong claim here; the burden of proof [2] lies with you, in much the same way that it would if you declared that horoscopes know the future. We have a sufficient explanation for LLM behaviour that we have no reason to discard based solely on your whim; if you want to be convincing, then you'll need more than assertion. Similar appearance does not mean identical nature, and your position does not warrant serious consideration until you provide support for it. [1] https://en.wikipedia.org/wiki/Sealioning https://en.wikipedia.org/wiki/Sealioning [2] https://en.wikipedia.org/wiki/Burden_of_proof_(philosophy) https://en.wikipedia.org/wiki/Burden_of_proof_(philosophy)
- vidarh 4mo agoWhat exactly do you consider reasoning?
- coldtea 4mo agoYou keep repeating this throughout the thread, one could even say just like a certain class of exotic birds :) Why couldn't parroting, after a certain level and complexity of the parroting infrastructure, be reasoning? What a priori restriction forbids it from being such? Flipping a NAND is not calculation either. Billions of them? Things change.
- HarHarVeryFunny 4mo agoCopying someone else's reasoning process (as best you understand it) is still a limited form of reasoning, even if it might be considered as cargo-cult reasoning (copying without understanding).
- chrisjj 4mo ago> Copying someone else's reasoning process (as best you understand it) is still a limited form of reasoning The LLM is not copying someone else's process. All it knows it someone else's product, and its only process is to compute the most likely next token within that.
- HarHarVeryFunny 4mo agoTrue, and this is why I said "as best you understand it". The visible (copyable) circumstances in which someone makes a reasoning decision aren't the same as the internal decision making process, so this is just cargo-cult copying - copy the surface form without understanding the reason for it. The perhaps surprising thing is that it works so well some of the time - specifically for math and coding.
- chrisjj 4mo agoIts surprises people just as did a web seach 30 years ago. "Wow, it found my page amongst millions - in seconds!" Just because raw dumb huge computer power is beyond the average user's compehension.