4 ms·

Whether LLMs are "intelligent" seems a wholly uninteresting distinction, resembling the internet ceremony surrounding whether a hotdog is a sandwich. There's p

by niam 1y ago

Whether LLMs are "intelligent" seems a wholly uninteresting distinction, resembling the internet ceremony surrounding whether a hotdog is a sandwich. There's probably very interesting discussion to be had about hotdogs and LLMs, but whether they're sandwiches or intelligent isn't a useful proxy to them.

deleted 1y ago
[deleted]
saulpw 1y ago
I disagree completely. Many people take for granted that the expression of intelligence/competence is the same as actual intelligence/competence, and many people are acting accordingly. But a simulacrum is definitively NOT the thing itself. When you trust fake intelligence, especially as a way to indulge mental laziness, your own faculties atrophy, and then in short order you can't even tell the difference between a real intelligence bomb and a dumb empty shell that has the word "intelligent" written on it.
- marcellus23 1y ago
  What are your thoughts on the Chinese room thought experiment?
  - saulpw 1y ago
    See my other comment above. Language manipulation is not sufficient for intelligence and understanding. There is no one in the Chinese Room who understands the questions and answers; there is no understanding in the system; there is no understanding at all.
- niam 1y ago
  I'm not even taking for granted what it means. Can you define it in a way that your neighbor will independently arrive at? It's an incredibly lossy container for whatever meaning that people will want to pack it with, moreso than for other words. Is a hotdog a simulacrum of a sandwich? Or a fake sandwich? I have no clue and don't care because it doesn't meaningfully inform me of the utility of the thing. An LLM might be "unintelligent" but I can't model what you think the consequences of that are. I'd skip the formalities and just talk about those instead.
  - saulpw 1y ago
    It sounds like you are [dis]interested in a philosophical discussion about epistemology. So it seems that you've skipped the inquiry yourself and have short-circuited to "don't care". Which is kind of "utilitarian". For other perspectives[0]: > The school of skepticism questions the human ability to attain knowledge, while fallibilism says that knowledge is never certain. Empiricists hold that all knowledge comes from sense experience, whereas rationalists believe that some knowledge does not depend on it. Coherentists argue that a belief is justified if it coheres with other beliefs. Foundationalists, by contrast, maintain that the justification of basic beliefs does not depend on other beliefs. Internalism and externalism debate whether justification is determined solely by mental states or also by external circumstances. For my part, I do believe that there is non-propositional knowledge. That a person can look at a set of facts/experiences/inputs and apply their mind towards discerning knowledge (or "truth"), or at least the relative probability of knowledge being true. That while this discernment and knowledge might be explained or justified verbally and logically, the actual discernment is non-verbal. And, for sure, correctness is not even essential--a person may discern that the truth is unknowable from the information at their disposal, and they may even discern incorrectly! But there is some mental process that can actually look behind the words to its "meaning" and then apply its own discernment to that meaning. (Notably this does not merely aggregating everyone else's discernment!) This is "intelligence", and it is something that humans can do, even if many of us often don't even apply this faculty ourselves. From discussions on HN and otherwise I gather this is what people refer to by "world-modeling". So my discernment is that language manipulation is neither necessary nor sufficient for intelligence--though it may be necessary to communicate more abstract intelligence. What LLM/AGI proponents are arguing is that language manipulation is sufficient for intelligence. This is a profound misunderstanding of intelligence, and one that should not be written off with a blithe and unexamined "but who knows what intelligence is anyway". [0] https://en.wikipedia.org/wiki/Epistemology https://en.wikipedia.org/wiki/Epistemology
    - niam 1y ago
      I'm not discounting the philosophy, just the language. I don't mean to sound blithe. If I do, it's not out of indifference but out of active determination that these kinds of terminological boundary disputes quickly veer into pointlessness. They seldom inform us of anything other than how we choose to use words.

by niam 1y ago

deleted 1y ago

[deleted]

saulpw 1y ago

I disagree completely. Many people take for granted that the expression of intelligence/competence is the same as actual intelligence/competence, and many people are acting accordingly. But a simulacrum is definitively NOT the thing itself. When you trust fake intelligence, especially as a way to indulge mental laziness, your own faculties atrophy, and then in short order you can't even tell the difference between a real intelligence bomb and a dumb empty shell that has the word "intelligent" written on it.

marcellus23 1y ago
What are your thoughts on the Chinese room thought experiment?
- saulpw 1y ago
  See my other comment above. Language manipulation is not sufficient for intelligence and understanding. There is no one in the Chinese Room who understands the questions and answers; there is no understanding in the system; there is no understanding at all.
niam 1y ago
I'm not even taking for granted what it means. Can you define it in a way that your neighbor will independently arrive at? It's an incredibly lossy container for whatever meaning that people will want to pack it with, moreso than for other words. Is a hotdog a simulacrum of a sandwich? Or a fake sandwich? I have no clue and don't care because it doesn't meaningfully inform me of the utility of the thing. An LLM might be "unintelligent" but I can't model what you think the consequences of that are. I'd skip the formalities and just talk about those instead.
- saulpw 1y ago
  It sounds like you are [dis]interested in a philosophical discussion about epistemology. So it seems that you've skipped the inquiry yourself and have short-circuited to "don't care". Which is kind of "utilitarian". For other perspectives[0]: > The school of skepticism questions the human ability to attain knowledge, while fallibilism says that knowledge is never certain. Empiricists hold that all knowledge comes from sense experience, whereas rationalists believe that some knowledge does not depend on it. Coherentists argue that a belief is justified if it coheres with other beliefs. Foundationalists, by contrast, maintain that the justification of basic beliefs does not depend on other beliefs. Internalism and externalism debate whether justification is determined solely by mental states or also by external circumstances. For my part, I do believe that there is non-propositional knowledge. That a person can look at a set of facts/experiences/inputs and apply their mind towards discerning knowledge (or "truth"), or at least the relative probability of knowledge being true. That while this discernment and knowledge might be explained or justified verbally and logically, the actual discernment is non-verbal. And, for sure, correctness is not even essential--a person may discern that the truth is unknowable from the information at their disposal, and they may even discern incorrectly! But there is some mental process that can actually look behind the words to its "meaning" and then apply its own discernment to that meaning. (Notably this does not merely aggregating everyone else's discernment!) This is "intelligence", and it is something that humans can do, even if many of us often don't even apply this faculty ourselves. From discussions on HN and otherwise I gather this is what people refer to by "world-modeling". So my discernment is that language manipulation is neither necessary nor sufficient for intelligence--though it may be necessary to communicate more abstract intelligence. What LLM/AGI proponents are arguing is that language manipulation is sufficient for intelligence. This is a profound misunderstanding of intelligence, and one that should not be written off with a blithe and unexamined "but who knows what intelligence is anyway". [0] https://en.wikipedia.org/wiki/Epistemology https://en.wikipedia.org/wiki/Epistemology
  - niam 1y ago
    I'm not discounting the philosophy, just the language. I don't mean to sound blithe. If I do, it's not out of indifference but out of active determination that these kinds of terminological boundary disputes quickly veer into pointlessness. They seldom inform us of anything other than how we choose to use words.