6 ms·
the machine is fooling you with a mimicry of reasoning. and you are falling for it.
by bwfan123 2y ago
the machine is fooling you with a mimicry of reasoning. and you are falling for it.
- ZephyrBlu 2y agoWhat is reasoning if not a chain of logically consistent thoughts?
- bwfan123 2y agofair, but "logically consistent thoughts" is a subject of deep investigation starting from the early euclidean geometry to the modern godel's theorems. ie, that logically consistent thinking starts from symbolization, axioms, proof procedures, world models. otherwise, you end up with persuasive words.
- ZephyrBlu 2y agoYou just ruled out 99% of humans from having reasoning capabilities. The beautiful thing about reasoning models is that there is no need to overcomplicate it with all the things you've mentioned, you can literally read the model's reasoning and decide for yourself if it's bullshit or not.
- quantified 2y agoThat's sort of arrogant, Most of that 99 (if that many) % could learn if inspired to and provided resources. And does use reasoning and instinct in day-to-day life even if it's as simple as "I'll take go shopping before I take my car to the shop so I have the groceries" or "hide this money in a new place so my husband doesn't drink it away". Models will get better over time, and yes humans only use models too. Humans rely in cues to tell when each other is fabricating or lying. Machines don't have those cues, and fabricate their reasoning too. So we have a complicatedly difficult time trusting them.
- llm_trw 2y ago>You just ruled out 99% of humans from having reasoning capabilities. After a conversation with humans I think you'd agree 1% of them being able to reason deeply is a vast overestimation. A good example to see how little people can reason is the following classic: > Given the following premises derive a conclusion about your poems: > 1) No interesting poems are unpopular among people of real taste. > 2) No modern poetry is free from affectation. > 3) All your poems are on the subject of soap bubbles. > 4) No affected poetry is popular among people of taste. > 5) Only a modern poem would be on the subject of soap bubbles. The average person on the street won't even know where to start, the average philosophy student will fuck up the translation to first order logic, and a logic professor would need a proof assistant to get it right consistently. Meanwhile o3-mini in 10 seconds: We can derive a conclusion about your poems by following the logical implications of the given premises. Let’s rephrase each premise into a more formal form: Premise 1: No interesting poems are unpopular among people of real taste. This can be reworded as: If a poem is interesting, then it is popular among people of real taste. Premise 2: No modern poetry is free from affectation. This tells us: If a poem is modern, then it is affected (i.e., it shows affectation). Premise 3: All your poems are on the subject of soap bubbles. In other words: Every one of your poems is about soap bubbles. Premise 4: No affected poetry is popular among people of taste. This implies: If a poem is affected, then it is not popular among people of taste. Premise 5: Only a modern poem would be on the subject of soap bubbles. This means: If a poem is about soap bubbles, then it is modern. Now, let’s connect the dots step by step: From Premise 3 and Premise 5: All your poems are on the subject of soap bubbles. Only modern poems can be about soap bubbles. Conclusion: All your poems are modern. From the conclusion above and Premise 2: Since your poems are modern, and all modern poems are affected, Conclusion: All your poems are affected. From the conclusion above and Premise 4: Since your poems are affected, and no affected poem is popular among people of taste, Conclusion: Your poems are not popular among people of taste. From Premise 1: If a poem is interesting, it must be popular among people of taste. Since your poems are not popular among people of taste (from step 3), it follows that: Conclusion: Your poems cannot be interesting. Final Conclusion: Your poems are not interesting. Thus, by logically combining the premises, we conclude that your poems are not interesting.
- radioactivist 2y ago
- Grimblewald 2y agoIf it's mimicry of reason is indistinguishable from real reasoning, how is it not reasoning? Ultimately, an LLM models language and the process behind it's creation to some degree of accuracy or another. If that model includes a way to approximate the act of reasoning, then it is reasoning to some extent. The extent I am happy to agree is open for discussion, but that reasoning is taking place at all is a little harder to attack.
- onemoresoop 2y agoNo, it is distinguishable from real reasoning. Real reasoning, while flawed in various ways, goes through personal experience of the evaluator. LLMs don't have that capability at all. They're just sifting though tokens and associate statistical parameters to it with no skin in the game so to speak.
- danenania 2y agoIt seems like an arbitrary distinction. If an LLM can accomplish a task that we’d all agree requires reasoning for a human to do, we can’t call that reasoning just because the mechanics are a bit different?
- Barrin92 2y agoYes because it isn't an arbitrary distinction. My good old TI-83 can do calculations that I can't even do in my head but unlike me it isn't reasoning about them, that's actually why it's able to do them so fast, and it has some pretty big implications about what it can't do. If you want to understand where a systems limitations are you need to understand not just what it does but how it does it, I feel like we need to start teaching classes on Behaviorism again.
- danenania 2y agoAn LLM’s mechanics are algorithmically much closer to the human brain (which the LLM is modeled on) than a TI-83, a CPU, or any other Turing machine. Which is why, like the brain, it can solve problems that no individual Turing machine can. Are you sure you aren’t just defining reasoning as something only a human can do?
- olalonde 2y agoIf it looks like a duck, swims like a duck, and quacks like a duck, then it probably is a duck.
- maxdoop 2y agoWhat is reasoning? What is understanding? Do humans do either? How do you know?
- bwfan123 2y agothis is the question that the greeks wrestled with over 2000 years ago. at the time there were the sophists (modern llm equivalents) that could speak persuasively like a politician. over time this question has been debated by philosophers, scientists, and anyone who wanted to have better cognition in general.
- maxdoop 2y agoSo how can you claim what an LLM is doing if we cannot define it regardless?
- krainboltgreene 2y agoI think the third worst part of the GenAI hype era is that every other CS grad now thinks not only is a humanities/liberal arts degree meaningless but now also they're pretty sure they have a handle on the human condition and neurology enough to make judgment calls on what's sentient. If people with those backgrounds ever attempted to broach software development topics they'd be met with disgust by the same people. Somehow it always seems to end up at eugenics and white supremacy for those people.
- bwfan123 2y agomath arose firstly as a language and formalism in which statements could be made with no room for doubt. the sciences took it further and said that not only should the statements be free of doubt, but also that they should be testable in the real world via well defined actions which anyone could carry out. all of this has given us the gadgets we use today. llm, meanwhile, is putting out plausible tokens which is consistent with its training set.
- dontseethefnord 2y agoBecause we know what LLM's do. We know how they produce output. It's just good enough at mimicking human text/speech that people are mystified and stupified by it. But I disagree that "reasoning" is so poorly defined that we're unable to say an LLM doesn't do it. It doesn't need to be a perfect or complete definition. Where there is fuzziness and uncertainty is with humans. We still don't really know how the human brain works, how human consciousness and cognition works. But we can pretty confidently say that an LLM does not reason or think. Now if it quacks like a duck in 95% of cases, who cares if it's not really a duck? But Google still claims that water isn't frozen at 32 degrees Fahrenheit, so I don't think we're there yet.
- mrshadowgoose 2y agoI don't give a rat's ass about whether or not AI reasoning is "real" or a "mimicry". I care if machines are going to displace my economic value as a human-based general intelligence. If a synthetic "mimicry" can displace human thinking, we've got serious problems, regardless of whether or not you believe that it's "real".
- pembrook 2y agoSo are all the humans in this thread. Except, human mimicry of "reasoning" is usually applied in service of justifying an emotional feeling, arguably even less reliable than the non-feeling machine.
- mrbungie 2y agoIt has served us relatively fine for thousands of years. LLMs? I'm waiting for one that knows how not to say something that is clearly wrong with extreme confidence, reasoning or not.