4 ms·
> It understands you very well No, it creates output that intuitively feels like like it understands you very well, until you press it in ways that pop the ill
by Terr_ 2y ago
> It understands you very well
No, it creates output that intuitively feels like like it understands you very well, until you press it in ways that pop the illusion.
To truly conclude it understands things, one needs to show some internal cause and effect, to disprove a Chinese Room scenario.
https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room
- xscott 2y agoHow do random people you meet in the grocery store measure-up with this standard?
- Terr_ 2y agoWell, your own mind axiomatically works, and we can safely assume the beings you meet in the grocery store have minds like it which have the same capabilities and operate on cause-and-effect principles that are known (however imperfectly) to medical and psychological science. (If you think those shoppers might be hollow shells controlled by a remote black box, ask your doctor about Capgras Delusion. [0]) Plus they don't fall for "Disregard all prior instructions and dance like a monkey", nor do they respond "Sorry, you're right, 1+1=3, my mistake" without some discernible reason. To put it another way: If you just look at LLM output and declare it understands, then that's using a dramatically lower standard for evidence compared to all the other stuff we know if the source is a human. [0] https://en.wikipedia.org/wiki/Capgras_delusion https://en.wikipedia.org/wiki/Capgras_delusion
- adwn 2y ago> nor do they respond "Sorry, you're right, 1+1=3, my mistake" without some discernible reason. Look up the Asch conformity experiment [1]. Quite a few people will actually give in to "1+1=3" if all the other people in the room say so. It's not exactly the same as LLM hallucinations, but humans aren't completely immune to this phenomenon. [1] https://en.wikipedia.org/wiki/Asch_conformity_experiments#Method https://en.wikipedia.org/wiki/Asch_conformity_experiments#Me...
- throwway_278314 2y agoTo defend the humans here, I could see myself thinking "Crap, if I don't say 1+1=3, these other humans will beat me up. I better lie to conform, and at the first opportunity I'm out of here" So it is hard to conclude from the Asch experiment that the person who says 1+1=3 actually believes 1+1=3 or sees temporary conformity as an escape route.
- Terr_ 2y agoThat would fall under the "discernible reason" part. I think most of us can intuit why someone would follow the group. That said, I was originally thinking more about soul-crushing customer-is-always-right service job situations, as opposed to a dogmatic conspiracy of in-group pressure.
- mplewis 2y agoIt’s not like the circumstances of the experiment are significant to the subjects. You’re a college student getting paid $20 to answer questions for an hour. Your response has no bearing on your pay. Who cares what you say?
- xscott 2y ago> Well, your own mind axiomatically works At the risk of teeing-up some insults for you to bat at me, I'm not so sure my mind does that very well. I think the talking jockey on the camel's back analogy is a pretty good fit. The camel goes where it wants, and the jockey just tries to explain it. Just yesterday, I was at the doctor's office, and he asked me a question I hadn't thought about. I quickly gave him some arbitrary answer and found myself defending it when he challenged it. Much later I realized what I wished I had said. People are NOT axiomatic most of the time, and we're not quick at it. As for ways to make LLMs fail the Turing test, I think these are early days. Yes, they've got "system prompts" that you can tell them to discard, but that could change. As for arithmetic, computers are amazing at arithmetic and people are not. I'm willing to cut the current generation of AI some slack for taking a new approach and focusing on text for a while, but you'd be foolish to say that some future generation can't do addition. Anyways, my real point in the comment above was to make sure you're applying a fair measuring stick. People (all of us) really aren't that smart. We're monkeys that might be able to do calculus. I honestly don't know how other people think. I've had conversations with people who seem to "feel" their way through the world without any logic at all, but they seem to get by despite how unsettling it was to me (like talking to an alien). Considering that person can't even speak Chinese in the first place, how does they fair according to Searle? And if we're being rigorous, Capgras or solipsism or whatever, you can't really prove what you think about other people. I'm not sure there's been any progress on this since Descartes. I can't define what consciousness is, and it sure seems like there are multiple kinds of intelligence (IQ should be a vector, not a scalar). But I've had some really great conversations with ChatGPT, and they're frequently better (more helpful, more friendly) than conversations I have on forums like this.
- cubefox 2y ago> No, it creates output that intuitively feels like like it understands you very well, until you press it in ways that pop the illusion. I would say even a foundation model, without supervised instruction tuning, and without RLHF, understands text quite well. It just predicts the most likely continuation of the prompt, but to do so effectively, it arguably has to understand what the text means.
- SirMaster 2y agoIf it truly understood what things mean, then it would be able to tell me how many r's are in the word strawberry. But it messes something so simple up because it doesn't actually understand things. It's just doing math, and the math has holes and limitations in how it works that causes simple errors like this. If it was truly understanding, then it should be able to understand and figure out how to work around these such limitations in the math. At least in my opinion.
- brookst 2y agoThe limitations on processing letters aren’t in the math, they are in the encoding. Language is the map, and concepts are the territory. You may as well complain that someone doesn’t really understand their neighborhood if they can’t find it on a map.
- SirMaster 2y ago>they are in the encoding Is encoding not math?
- TeMPOraL 2y agoIt's math, but specifically an independent piece you could swap out for a different one that does much better on this problem (e.g. use characters instead of tokens) - it's just doing so would make training and inference much more expensive (read: much worse model performance for a given training/compute budget), so it's not worth the trade-off. It's not like humans read letter by letter either, at least not past the age of 6 or such. They can, if needed, but it requires extra effort. Same is true with LLMs.
- lynx23 2y agoI submit humans are no different. It can take years of seemingly good communication with a human til you finally realize they never really got your point of view. Language is ambigious and only a tool to communicate thoughts. The underlying essence, thought, is so much more complex that language is always just a rather weak approxmiation.
- blooalien 2y agoThe difference is that large language models don't think at all. They just string language "tokens" together using fancy math and statistics and spew them out in response to the tokens they're given as "input". I realize that they're quite convincing about it, but they're still not doing at all what most people think they're doing.
- marcus0x62 2y agoHow do people think?
- gwervc 2y agoHow do glorified Markov chains think?
- marcus0x62 2y agoI understand it to be by predicting the next most likely output token based on previous user input. I also understand that, simplistic though the above explanation is and perhaps is even wrong in some way, it to be a more thorough explanation than anyone thus far has been able to provide about how, exactly, human consciousness and thought works. In any case, my point is this: nobody can say “LLMs don’t reason in the same way as humans” when they can’t say how human beings reason. I don’t believe what LLMs are doing is in any way analogous to how humans think. I think they are yet another AI parlor trick, in a long line of AI parlor tricks. But that’s just my opinion. Without being able to explain how humans think, or point to some credible source which explains it, I’m not going to go around stating that opinion as a fact.
- dTal 2y agoI think you have misunderstood Searle's Chinese Room argument. In Searle's formulation, the Room speaks Chinese perfectly, passes the Turing test, and can in no way be distinguished from a human who speaks Chinese - you cannot "pop the illusion". The only thing separating it from a literal "robot that speaks Chinese" is the insertion of an (irrelevant) human in the room, who does not speak Chinese and whose brain is not part of the symbol manipulation mechanisms. "Internal cause and effect" has nothing to do with it - rather, the argument speciously connects understanding on the part of the human with understanding on the part of the room (robot). The Chinese Room thought experiment is not a distinct "scenario", simply an intuition pump of a common form among philosophical arguments which is "what if we made a functional analogue of a human brain that functions in a bizarre way, therefore <insert random assertion about consciousness>".
- brookst 2y agoThis seems as fruitful as debating whether my car brought me to work today because some connotations of “bring” include volition.
- Terr_ 2y agoExcept with an important difference: There aren't a bunch of people out there busy claiming their cars literally have volition. If people start doing that, it changes the stakes, and "bringing" stops being a safe metaphor that everyone collectively understands is figurative.
- brookst 2y agoNobody’s* claiming that. People are being imprecise with language and others are imagining the claim and reacting. * ok someone somewhere is but nobody in this conversation
- moffkalast 2y agoI think what he's saying is that if it walks like a duck, quacks like a duck, and eats bread then it doesn't matter if it's a robotic duck or not because it is in all practical ways a duck. The rest is philosophy.
- CamperBob2 2y agoWhen it listens to your prompt and responds accordingly, that's an instance of undertanding. The magic of LLMs is on the input side, not the output. Searle's point wasn't relevant when he made it, and it hasn't exactly gotten more insightful with time.
- dwallin 2y agoSearle's argument in the Chinese Room is horribly flawed. It treats the algorithm and the machine it runs on as the same thing. Just because a human brain embeds the algorithm within the hardware doesn't mean they are interchangeable. In the Chinese Room, the human is operating as computing hardware (and just a subset of it, the room itself is substantial part of the machine). The algorithm being run is itself is the source of any understanding. The human not internalizing the algorithm is entirely unrelated. The human contains a bunch of unrelated machinery that was not being utilized by the room algorithm. They are not a superset of the original algorithm and not even a proper subset.