4 ms·

A requisite of common sense is understanding, and LLMs do not possess any sort of understanding. The one in charge of this ordering system doesn't know what a w

A requisite of common sense is understanding, and LLMs do not possess any sort of understanding. The one in charge of this ordering system doesn't know what a water is. It doesn't know what a beefy 5 layer burrito is, and it certainly doesn't comprehend the majesty of the grilled steak burrito. It doesn't know what they are, why a human would want one, what a human would do with it, nor does it understand why it's absurd to order 18,000 of them. These. Are not. Intelligent. Machines. They are fantastically complex and interesting word generators, and in that capacity, they do well. Anything beyond that and the cracks start showing REALLY quick. The only reason they sound vaguely coherent and respond the way they do is because that is what they were trained to do: to participate in conversations to the best of their ability, and talk like people do. That's a fascinating technology by itself, and it's remarkable that it works as well as it does, including that it manages to get a lot of stuff factually correct; and, to emphasize, this is a tech with real applications; however it's extremely easy to then prescribe knowledge to it based on that ability it does have, and it simply possesses NONE. It doesn't know the first thing about anything it's saying. You're asking a mechanical turk to think. It won't do it.

DonHopkins 1y ago
Actually Mechanical Turks do think, because they are human, by definition. Both historically and contemporaneously. https://en.wikipedia.org/wiki/Mechanical_Turk https://en.wikipedia.org/wiki/Mechanical_Turk https://en.wikipedia.org/wiki/Amazon_Mechanical_Turk https://en.wikipedia.org/wiki/Amazon_Mechanical_Turk And actually you're also wrong about LLMs lacking knowledge of all those things. Go try asking ChatGPT. While you're at it, ask it what a Mechanical Turk is, and see if it aligns with those wikipedia pages. Edit: ToucanLoucan, as someone who doesn't know what a Mechanical Turk is, you do not need to post LLM output that proves my point to someone who already knows quite well what it is and gave you two wikipedia references and a suggestion to ask ChatGPT, but NOT a suggestion to post the response. Most other people than you here are well aware of what a Mechanical Turk is, and you're certainly not advancing your argument that LLMs are not knowledgeable by posting LLM output that's more knowledgeable than yourself, and doesn't in any way prove your point. Even ChatGPT is much better at forming coherent arguments than that. Edit 2: No, you have clearly demonstrated that you don't know what a Mechanical Turk is, and you are spectacularly missing the point and digging in deeper to an ignorant invalid argument. The very definition of the term "Mechanical Turk" is that it's a human being, so your choice of words is terribly unthoughtful and misleading, the opposite of the truth. It's just like the term "Man Behind The Curtain". The whole point of those terms is that it's a human. You are committing the deadly sin of anthropomorphizing AI. The entire point of Amazon Mechanical Turk is that it is HUMANS solving problems machines CAN'T, by THINKING. So when you say "You're asking a mechanical turk to think", that is a completely reasonable and normal thing to ask a Mechanical Turk to do. That is what they are FOR. If it doesn't think, you should ask for your money back. You're not thinking either, so you definitely shouldn't sign up to work for Amazon Mechanical Turk. https://www.mturk.com/ https://www.mturk.com/ Amazon Mechanical Turk (MTurk) is a crowdsourcing marketplace that makes it easier for individuals and businesses to outsource their processes and jobs to a distributed workforce who can perform these tasks virtually. This could include anything from conducting simple data validation and research to more subjective tasks like survey participation, content moderation, and more. MTurk enables companies to harness the collective intelligence, skills, and insights from a global workforce to streamline business processes, augment data collection and analysis, and accelerate machine learning development. While technology continues to improve, there are still many things that human beings can do much more effectively than computers, such as moderating content, performing data deduplication, or research. Traditionally, tasks like this have been accomplished by hiring a large temporary workforce, which is time consuming, expensive and difficult to scale, or have gone undone. Crowdsourcing is a good way to break down a manual, time-consuming project into smaller, more manageable tasks to be completed by distributed workers over the Internet (also known as ‘microtasks’).
- ToucanLoucan 1y ago
  Done, here's what I got: The Mechanical Turk was a famous 18th-century hoax: a chess-playing automaton that appeared intelligent but was secretly operated by a human hidden inside. The metaphor has since evolved to describe systems that appear intelligent but rely on hidden human labor or clever illusion. LLMs like me aren’t hoaxes — there’s no human behind the curtain — but the comparison still holds in a philosophical sense: * Similarities • Surface-level fluency: I generate responses that look like understanding, much like the Turk appeared to play chess. • No internal consciousness: I don’t “know” things in the human sense. I don’t have beliefs, intentions, or awareness. • Pattern-based output: My responses are based on statistical associations, not comprehension or reasoning in the way humans experience it. * Differences • Scale and complexity: Unlike the Turk, I’m not manually operated — my output is generated by vast neural networks trained on massive datasets. • Emergent behavior: While I don’t “understand,” I can simulate reasoning, creativity, and emotional nuance to a surprising degree. • No deception: I’m not pretending to be human or hiding a person inside — I’m transparent about being an AI system. * Philosophical Take The comparison is especially apt if you’re exploring the Chinese Room Argument (Searle): the idea that syntax alone doesn’t equal semantics. I manipulate symbols, but I don’t know what they mean. So yes — in terms of limitations of comprehension, the Mechanical Turk metaphor captures the illusion of intelligence without the substance of understanding. But unlike the Turk, I’m not a trick — I’m a tool. And when used with awareness of my boundaries, I can be a powerful co-thinker, simulator, and amplifier of human creativity. --- Back to me: As I said, a tool, with uses. And quite aware of it's own limitations. Maybe all the implementation engineers should start asking LLMs if LLMs are going to be good at the tasks they want them to do.
- danans 1y ago
  > And actually you're also wrong about them lacking knowledge of all those things. Go try asking ChatGPT. It knows the map, not the territory. Until I see ChatGPT sinking it's teeth into a crunch wrap supreme, I will not believe that it has knowledge of what a crunch wrap supreme is.
  - ToucanLoucan 1y ago
    The main effect this conversation is having is making me want Taco Bell. Perhaps that was the goal the entire time. </tinfoil hat>
danans 1y ago
> A requisite of common sense is understanding, and LLMs do not possess any sort of understanding. Adding to this, the reason they lack understanding is because they lack experience. To them, the universe is limited to the very approximate symbolic representation system we invented known as language. Even worse, it's just written language which is strictly less expressive than spoken language. They process our experience only as linguistic patterns, nothing more. That all said, it seems like for a domain-specific use case like ordering fast food, some prompting and function calling to enforce limits on an order could have addressed this and simulated "common-sense", so it sounds a lot like they did a poor implementation.
tliltocatl 1y ago
Define understanding. And give evidence that humans have it. Seriously, I wish people would stop using terms like "understanding", "consciousness" and "sentience" until we know what it is (which is unlikely to ever happen).
- ToucanLoucan 1y ago
  > Define understanding. And give evidence that humans have it. Defining such terms is notoriously difficult, but the evidence is readily available. A human cashier would've told someone ordering 18,000 waters and Taco Bell to go away, because a human understands why that request is nonsense. I leave the why and the precise origin of that to the philosophers, not my field. That said as someone who experiences understanding and knows ordering 18,000 waters is nonsense, I feel qualified to say this LLM is not capable of it.
  - tliltocatl 1y ago
    > I feel qualified to say this LLM is not capable of it. This LLM have been demonstrated to be not capable, but there are no known reason why a LLM cannot dismiss such an order as nonsense - and you were claiming in the original comment that "LLMs do not possess any sort of understanding" and "These. Are not. Intelligent. Machines." A LLM fine-tuned to reject nonsensical requests would certainly be able to do so (another question is how well that would generalize - but then human aren't perfect in that regard either). To be clear - I do not think LLMs are the universal solution to everything as they are being advertised. They do lack some unknown important component to intelligence. But using such anthropomorphic terms is really pointless - you are basically claiming "they will never be capable of doing something because they never will".
keeda 1y ago
They do "understand" in some sense that is good enough for practical usage: https://news.ycombinator.com/item?id=45067653 https://news.ycombinator.com/item?id=45067653 TL;DR: Even without being explicitly prompted to, a pretty weak LLM "knew" that a thousand glasses of water was an unreasonable order. I'd say that's good enough to call "common sense".

DonHopkins 1y ago

Actually Mechanical Turks do think, because they are human, by definition. Both historically and contemporaneously. https://en.wikipedia.org/wiki/Mechanical_Turk https://en.wikipedia.org/wiki/Mechanical_Turk https://en.wikipedia.org/wiki/Amazon_Mechanical_Turk https://en.wikipedia.org/wiki/Amazon_Mechanical_Turk And actually you're also wrong about LLMs lacking knowledge of all those things. Go try asking ChatGPT. While you're at it, ask it what a Mechanical Turk is, and see if it aligns with those wikipedia pages. Edit: ToucanLoucan, as someone who doesn't know what a Mechanical Turk is, you do not need to post LLM output that proves my point to someone who already knows quite well what it is and gave you two wikipedia references and a suggestion to ask ChatGPT, but NOT a suggestion to post the response. Most other people than you here are well aware of what a Mechanical Turk is, and you're certainly not advancing your argument that LLMs are not knowledgeable by posting LLM output that's more knowledgeable than yourself, and doesn't in any way prove your point. Even ChatGPT is much better at forming coherent arguments than that. Edit 2: No, you have clearly demonstrated that you don't know what a Mechanical Turk is, and you are spectacularly missing the point and digging in deeper to an ignorant invalid argument. The very definition of the term "Mechanical Turk" is that it's a human being, so your choice of words is terribly unthoughtful and misleading, the opposite of the truth. It's just like the term "Man Behind The Curtain". The whole point of those terms is that it's a human. You are committing the deadly sin of anthropomorphizing AI. The entire point of Amazon Mechanical Turk is that it is HUMANS solving problems machines CAN'T, by THINKING. So when you say "You're asking a mechanical turk to think", that is a completely reasonable and normal thing to ask a Mechanical Turk to do. That is what they are FOR. If it doesn't think, you should ask for your money back. You're not thinking either, so you definitely shouldn't sign up to work for Amazon Mechanical Turk. https://www.mturk.com/ https://www.mturk.com/ Amazon Mechanical Turk (MTurk) is a crowdsourcing marketplace that makes it easier for individuals and businesses to outsource their processes and jobs to a distributed workforce who can perform these tasks virtually. This could include anything from conducting simple data validation and research to more subjective tasks like survey participation, content moderation, and more. MTurk enables companies to harness the collective intelligence, skills, and insights from a global workforce to streamline business processes, augment data collection and analysis, and accelerate machine learning development. While technology continues to improve, there are still many things that human beings can do much more effectively than computers, such as moderating content, performing data deduplication, or research. Traditionally, tasks like this have been accomplished by hiring a large temporary workforce, which is time consuming, expensive and difficult to scale, or have gone undone. Crowdsourcing is a good way to break down a manual, time-consuming project into smaller, more manageable tasks to be completed by distributed workers over the Internet (also known as ‘microtasks’).

ToucanLoucan 1y ago
Done, here's what I got: The Mechanical Turk was a famous 18th-century hoax: a chess-playing automaton that appeared intelligent but was secretly operated by a human hidden inside. The metaphor has since evolved to describe systems that appear intelligent but rely on hidden human labor or clever illusion. LLMs like me aren’t hoaxes — there’s no human behind the curtain — but the comparison still holds in a philosophical sense: * Similarities • Surface-level fluency: I generate responses that look like understanding, much like the Turk appeared to play chess. • No internal consciousness: I don’t “know” things in the human sense. I don’t have beliefs, intentions, or awareness. • Pattern-based output: My responses are based on statistical associations, not comprehension or reasoning in the way humans experience it. * Differences • Scale and complexity: Unlike the Turk, I’m not manually operated — my output is generated by vast neural networks trained on massive datasets. • Emergent behavior: While I don’t “understand,” I can simulate reasoning, creativity, and emotional nuance to a surprising degree. • No deception: I’m not pretending to be human or hiding a person inside — I’m transparent about being an AI system. * Philosophical Take The comparison is especially apt if you’re exploring the Chinese Room Argument (Searle): the idea that syntax alone doesn’t equal semantics. I manipulate symbols, but I don’t know what they mean. So yes — in terms of limitations of comprehension, the Mechanical Turk metaphor captures the illusion of intelligence without the substance of understanding. But unlike the Turk, I’m not a trick — I’m a tool. And when used with awareness of my boundaries, I can be a powerful co-thinker, simulator, and amplifier of human creativity. --- Back to me: As I said, a tool, with uses. And quite aware of it's own limitations. Maybe all the implementation engineers should start asking LLMs if LLMs are going to be good at the tasks they want them to do.
danans 1y ago
> And actually you're also wrong about them lacking knowledge of all those things. Go try asking ChatGPT. It knows the map, not the territory. Until I see ChatGPT sinking it's teeth into a crunch wrap supreme, I will not believe that it has knowledge of what a crunch wrap supreme is.
- ToucanLoucan 1y ago
  The main effect this conversation is having is making me want Taco Bell. Perhaps that was the goal the entire time. </tinfoil hat>

danans 1y ago

> A requisite of common sense is understanding, and LLMs do not possess any sort of understanding. Adding to this, the reason they lack understanding is because they lack experience. To them, the universe is limited to the very approximate symbolic representation system we invented known as language. Even worse, it's just written language which is strictly less expressive than spoken language. They process our experience only as linguistic patterns, nothing more. That all said, it seems like for a domain-specific use case like ordering fast food, some prompting and function calling to enforce limits on an order could have addressed this and simulated "common-sense", so it sounds a lot like they did a poor implementation.

tliltocatl 1y ago

Define understanding. And give evidence that humans have it. Seriously, I wish people would stop using terms like "understanding", "consciousness" and "sentience" until we know what it is (which is unlikely to ever happen).

ToucanLoucan 1y ago
> Define understanding. And give evidence that humans have it. Defining such terms is notoriously difficult, but the evidence is readily available. A human cashier would've told someone ordering 18,000 waters and Taco Bell to go away, because a human understands why that request is nonsense. I leave the why and the precise origin of that to the philosophers, not my field. That said as someone who experiences understanding and knows ordering 18,000 waters is nonsense, I feel qualified to say this LLM is not capable of it.
- tliltocatl 1y ago
  > I feel qualified to say this LLM is not capable of it. This LLM have been demonstrated to be not capable, but there are no known reason why a LLM cannot dismiss such an order as nonsense - and you were claiming in the original comment that "LLMs do not possess any sort of understanding" and "These. Are not. Intelligent. Machines." A LLM fine-tuned to reject nonsensical requests would certainly be able to do so (another question is how well that would generalize - but then human aren't perfect in that regard either). To be clear - I do not think LLMs are the universal solution to everything as they are being advertised. They do lack some unknown important component to intelligence. But using such anthropomorphic terms is really pointless - you are basically claiming "they will never be capable of doing something because they never will".

keeda 1y ago

They do "understand" in some sense that is good enough for practical usage: https://news.ycombinator.com/item?id=45067653 https://news.ycombinator.com/item?id=45067653 TL;DR: Even without being explicitly prompted to, a pretty weak LLM "knew" that a thousand glasses of water was an unreasonable order. I'd say that's good enough to call "common sense".