8 ms·

It works at the level of patterns in the training corpus. That those patterns might correlate somewhat with underlying real concepts should be no great surprise

It works at the level of patterns in the training corpus. That those patterns might correlate somewhat with underlying real concepts should be no great surprise - but it is not 'working' at that level.

diablozzq 3y ago
This describes humans in a nutshell. This is why people vary wildly in ability... But we consider humans intelligent.
- mahathu 3y ago
  >This describes humans in a nutshell. This is why people vary wildly in ability... Can you expand on that a little?
  - Sunhold 3y ago
    More than 80% of college students get the following question wrong.[1] I'd say this is an example of naive pattern matching with no real reasoning. Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations. Which is more probable? a) Linda is a bank teller. b) Linda is a bank teller and is active in the feminist movement. [1] https://en.wikipedia.org/wiki/Conjunction_fallacy https://en.wikipedia.org/wiki/Conjunction_fallacy
    - andsoitis 3y ago
      > I'd say this is an example of naive pattern matching with no real reasoning. It is an example of lack of mathematical (really probability) know how, not naïve pattern matching.
      - bsaul 3y ago
        I'd say it's an example of misphrased questions that people will intuitively make sense of, and thus produce strictly incorrect answer. Obviously, people rephrase the question into : is there more chance that she's a feminist or that she isn't ?
        SAI_Peregrinus 3y ago
        Well, it's multiple-choice. So the "a" answer excludes the "b" answer by convention, and thus implies she isn't a feminist. Rephrase the question to remove that implication and I expect far more people will pick the "a" answer.
    - qayxc 3y ago
      I'd say this is the very definition of "overthinking it" instead of pattern matching. Pattern matching would lead you straight to the correct answer - joint probability is always ≤ single probability. Meaning the information given in the question is just fluff. Pattern matching reduces to "Which is more probable: just A or else A ^ B?" at which point the correct answer becomes obvious.
    - matthewdgreen 3y ago
      Phrasing this in terms of probability seems very weird in the first place.
    - booleandilemma 3y ago
      Eh, the "correct" answer seems silly and overly mathematical to me. The fact is, you can infer things about people based on things you know about people. I can pick a random user on HN, and knowing they're a user of HN, I can say it's probable that they work in technology. We don't need to bring statistics into it and turn it into a math problem.
    - wwweston 3y ago
      Hank is 31 years old, single, outspoken, and very bright. Hank got a technical degree, and as a student was deeply interested in machine learning and the ethics & philosophy of potential artificial intelligence. He posts a problem on HN where he describes the personality, capabilities, interests, and background of a woman named Linda has and then asks people to reflect on her occupation, politics, and an intersection between the two. Which is more likely? a) Hank has written us a human interest problem b) Hank has written us a human interest and a probability problem. I don't think people are simply wrong about the Linda problem, I think they're imprecise about which question they're answering, and more or less think they're answering a question about what chances that Linda is a feminist vs what are the chances she's a bank teller not only using the givens+relevant priors about people but also their priors about what kind of question they're answering. It isn't "no real reasoning", it's just not high resolution enough to be technically correct by the standards of a constructed probability problem. You can argue LLMs are also not quite high resolution enough and I'd accept that. In my mind the question is what it would take to get some kind of ML software to a place where if you trained it on enough probability problems it would be able to evaluate the Hank problem above, including the issue of whether (a) and (b) are actually independent. ;)
    - riizade 3y ago
      This particular question doesn't seem like a fallacy to me. I, like most people intuitively answered b). Given the explanation on the Wikipedia page I went "oh, of course, yeah", but then I thought about why I'd answer b) given that I'm fairly familiar with basic probability. If you give me two options and ask me to pick between them, my brain is usually going to assume it's not a trivially true problem. Language needs context for any sense to be made of it. As a result of the above, reading the question, the intuitive reading makes the answer choices a) Linda is a bank teller (implicitly, a bank teller NOT active in the feminist movement) b) Linda is a bank teller and is active in the feminist movement. This question is one of language, context, and interpretation, not of people failing to understand basic probability. I suspect that if you prime people to excise interpretation of the question by presenting it as the following, the majority of people would guess correctly: --- "Consider the following two statements: 1) Linda is a Bank Teller 2) Linda is active in the feminist movement Which is more likely? a) 1 b) 1 ^ 2"
    - SAI_Peregrinus 3y ago
      Multiple choice questions imply that other choices are excluded (unless an answer such as "all of the above" is a choice. So the implied question is: a) Linda is a bank teller (and NOT active in the feminist movement) b) Linda is a bank teller and is active in the feminist movement What you want it to be asking here is: a) Linda is a bank teller, and may or may not be active in the feminist movement b) Linda is a bank teller, and is active in the feminist movement Most college students have taken quite a few multiple-choice tests (particularly in the US, high-schools train for standardized multiple-choice tests). The question isn't asking what the mathematicians seem to think it's asking, because it's format conveys extra restrictions.
- somewhereoutth 3y ago
  No, no it doesn't. Not even slightly. If you fed a human child only text from the internet you'd not produce a competent adult (by competent I mean able to feed themselves etc). That some tech people think that human intelligence can be reduced to such "textural mechanics" betrays a lack of depth of understanding and even appreciation of the deep and complex world within which we find ourselves. Our written corpus is but a particular reflection of this reality - the shadows on the wall in Plato's cave if you will.
  - darkerside 3y ago
    Feed themselves? You mean pick up a burger from the local McDonald's? There's got to be a better Turing test.
  - kqr 3y ago
    Are you making the argument that children do not come with an innate ability to read text and thus cannot learn from it, or are you making the argument that the internet does not contain, among other things, fairly detailed instructions on various ways to feed oneself?
    - somewhereoutth 3y ago
      I'm saying that without the context of experiencing the underlying reality, text of itself is meaningless. What is a 'spoon' anyway??
      - kqr 3y ago
        So you're saying one needs some sort of reference for which words refer to which real-world sensory experiences such as "the thing that looks like this is a spoon", and text models do not have sensory associations?
      - z7 3y ago
        Isn't "meaning" just how a word is used within a specific context? I think that was one of Wittgenstein's points. The word "dog" doesn't need to reference the underlying reality to have meaning. Its meaning emerges from its usage in relation to all other words. Language isn't a mirror of the external world, which is probably one of the reasons LLMs are so successful.
School-Cotton 3y ago
How do you know?
- somewhereoutth 3y ago
  Because that is all there is. By conservation of information you can't get out more than what you put in. If it were interacting with the real world in some way (i.e. had 'eyes' and 'hands'), with a learning reward system, then maybe deeper patterns could get encoded.
  - thumbuddy 3y ago
    Don't you dare start bringing reality and applying theory to these matters! - someone somewhere
  - School-Cotton 3y ago
    That sounds like nonsense. What is “conservation of information”; can you define it precisely?
- rifty 3y ago
  This might not be fully convincing or rigorous, but here is one of the interactions i've had which make me lean towards patterns not concepts... I have some unique synonyms I know which are not in dictionaries or commonly used but I know when I have used them in conversation they understand. I ask the LLM to provide definitions for WordA and WordB. -They come back essentially the same. I ask the LLM if the two definitions are synonymous. -The LLM responds with yes. I then ask if WordA is synonymous with WordB. -It replies no. To me through this and other interactions they don't actually conceptually hold things like the definition as the word. It responds DefinitionA is WordAs definition, it responds DefinitionB is WordBs definition. It follows that DefinitionA of WordA is synonymous with DefinitionB of WordB but not that WordA is synonymous with WordB... I expect the above is solvable through pattern recognition with tokens. I think given enough information it becomes effectively indistinguishable and practically the same to us. The above is just a testament to how it is getting there... Since it delivers intellect it will be called intelligent, but it doesn't appear to me to be approaching it like human intelligence. Though maybe given enough information, or through different modalities, like the real world as the other commenter says, it will model some deeper coherency.
namaria 3y ago
I find this debate fascinating, and I'm excited for more people realizing that a) We have a very, very blurry definition of intelligence, and b) As more people get convinced there is some intelligence in software, more people get convinced there is (or there might be) intelligence in animals. Whatever the bottom line on current LLMs is, the cultural change they might foster makes me smile.

diablozzq 3y ago

This describes humans in a nutshell. This is why people vary wildly in ability... But we consider humans intelligent.

mahathu 3y ago
>This describes humans in a nutshell. This is why people vary wildly in ability... Can you expand on that a little?
- Sunhold 3y ago
  More than 80% of college students get the following question wrong.[1] I'd say this is an example of naive pattern matching with no real reasoning. Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations. Which is more probable? a) Linda is a bank teller. b) Linda is a bank teller and is active in the feminist movement. [1] https://en.wikipedia.org/wiki/Conjunction_fallacy https://en.wikipedia.org/wiki/Conjunction_fallacy
  - andsoitis 3y ago
    > I'd say this is an example of naive pattern matching with no real reasoning. It is an example of lack of mathematical (really probability) know how, not naïve pattern matching.
    - bsaul 3y ago
      I'd say it's an example of misphrased questions that people will intuitively make sense of, and thus produce strictly incorrect answer. Obviously, people rephrase the question into : is there more chance that she's a feminist or that she isn't ?
      - SAI_Peregrinus 3y ago
        Well, it's multiple-choice. So the "a" answer excludes the "b" answer by convention, and thus implies she isn't a feminist. Rephrase the question to remove that implication and I expect far more people will pick the "a" answer.
  - qayxc 3y ago
    I'd say this is the very definition of "overthinking it" instead of pattern matching. Pattern matching would lead you straight to the correct answer - joint probability is always ≤ single probability. Meaning the information given in the question is just fluff. Pattern matching reduces to "Which is more probable: just A or else A ^ B?" at which point the correct answer becomes obvious.
  - matthewdgreen 3y ago
    Phrasing this in terms of probability seems very weird in the first place.
  - booleandilemma 3y ago
    Eh, the "correct" answer seems silly and overly mathematical to me. The fact is, you can infer things about people based on things you know about people. I can pick a random user on HN, and knowing they're a user of HN, I can say it's probable that they work in technology. We don't need to bring statistics into it and turn it into a math problem.
  - wwweston 3y ago
    Hank is 31 years old, single, outspoken, and very bright. Hank got a technical degree, and as a student was deeply interested in machine learning and the ethics & philosophy of potential artificial intelligence. He posts a problem on HN where he describes the personality, capabilities, interests, and background of a woman named Linda has and then asks people to reflect on her occupation, politics, and an intersection between the two. Which is more likely? a) Hank has written us a human interest problem b) Hank has written us a human interest and a probability problem. I don't think people are simply wrong about the Linda problem, I think they're imprecise about which question they're answering, and more or less think they're answering a question about what chances that Linda is a feminist vs what are the chances she's a bank teller not only using the givens+relevant priors about people but also their priors about what kind of question they're answering. It isn't "no real reasoning", it's just not high resolution enough to be technically correct by the standards of a constructed probability problem. You can argue LLMs are also not quite high resolution enough and I'd accept that. In my mind the question is what it would take to get some kind of ML software to a place where if you trained it on enough probability problems it would be able to evaluate the Hank problem above, including the issue of whether (a) and (b) are actually independent. ;)
  - riizade 3y ago
    This particular question doesn't seem like a fallacy to me. I, like most people intuitively answered b). Given the explanation on the Wikipedia page I went "oh, of course, yeah", but then I thought about why I'd answer b) given that I'm fairly familiar with basic probability. If you give me two options and ask me to pick between them, my brain is usually going to assume it's not a trivially true problem. Language needs context for any sense to be made of it. As a result of the above, reading the question, the intuitive reading makes the answer choices a) Linda is a bank teller (implicitly, a bank teller NOT active in the feminist movement) b) Linda is a bank teller and is active in the feminist movement. This question is one of language, context, and interpretation, not of people failing to understand basic probability. I suspect that if you prime people to excise interpretation of the question by presenting it as the following, the majority of people would guess correctly: --- "Consider the following two statements: 1) Linda is a Bank Teller 2) Linda is active in the feminist movement Which is more likely? a) 1 b) 1 ^ 2"
  - SAI_Peregrinus 3y ago
    Multiple choice questions imply that other choices are excluded (unless an answer such as "all of the above" is a choice. So the implied question is: a) Linda is a bank teller (and NOT active in the feminist movement) b) Linda is a bank teller and is active in the feminist movement What you want it to be asking here is: a) Linda is a bank teller, and may or may not be active in the feminist movement b) Linda is a bank teller, and is active in the feminist movement Most college students have taken quite a few multiple-choice tests (particularly in the US, high-schools train for standardized multiple-choice tests). The question isn't asking what the mathematicians seem to think it's asking, because it's format conveys extra restrictions.
somewhereoutth 3y ago
No, no it doesn't. Not even slightly. If you fed a human child only text from the internet you'd not produce a competent adult (by competent I mean able to feed themselves etc). That some tech people think that human intelligence can be reduced to such "textural mechanics" betrays a lack of depth of understanding and even appreciation of the deep and complex world within which we find ourselves. Our written corpus is but a particular reflection of this reality - the shadows on the wall in Plato's cave if you will.
- darkerside 3y ago
  Feed themselves? You mean pick up a burger from the local McDonald's? There's got to be a better Turing test.
- kqr 3y ago
  Are you making the argument that children do not come with an innate ability to read text and thus cannot learn from it, or are you making the argument that the internet does not contain, among other things, fairly detailed instructions on various ways to feed oneself?
  - somewhereoutth 3y ago
    I'm saying that without the context of experiencing the underlying reality, text of itself is meaningless. What is a 'spoon' anyway??
    - kqr 3y ago
      So you're saying one needs some sort of reference for which words refer to which real-world sensory experiences such as "the thing that looks like this is a spoon", and text models do not have sensory associations?
    - z7 3y ago
      Isn't "meaning" just how a word is used within a specific context? I think that was one of Wittgenstein's points. The word "dog" doesn't need to reference the underlying reality to have meaning. Its meaning emerges from its usage in relation to all other words. Language isn't a mirror of the external world, which is probably one of the reasons LLMs are so successful.

School-Cotton 3y ago

How do you know?

somewhereoutth 3y ago
Because that is all there is. By conservation of information you can't get out more than what you put in. If it were interacting with the real world in some way (i.e. had 'eyes' and 'hands'), with a learning reward system, then maybe deeper patterns could get encoded.
- thumbuddy 3y ago
  Don't you dare start bringing reality and applying theory to these matters! - someone somewhere
- School-Cotton 3y ago
  That sounds like nonsense. What is “conservation of information”; can you define it precisely?
rifty 3y ago
This might not be fully convincing or rigorous, but here is one of the interactions i've had which make me lean towards patterns not concepts... I have some unique synonyms I know which are not in dictionaries or commonly used but I know when I have used them in conversation they understand. I ask the LLM to provide definitions for WordA and WordB. -They come back essentially the same. I ask the LLM if the two definitions are synonymous. -The LLM responds with yes. I then ask if WordA is synonymous with WordB. -It replies no. To me through this and other interactions they don't actually conceptually hold things like the definition as the word. It responds DefinitionA is WordAs definition, it responds DefinitionB is WordBs definition. It follows that DefinitionA of WordA is synonymous with DefinitionB of WordB but not that WordA is synonymous with WordB... I expect the above is solvable through pattern recognition with tokens. I think given enough information it becomes effectively indistinguishable and practically the same to us. The above is just a testament to how it is getting there... Since it delivers intellect it will be called intelligent, but it doesn't appear to me to be approaching it like human intelligence. Though maybe given enough information, or through different modalities, like the real world as the other commenter says, it will model some deeper coherency.

namaria 3y ago

I find this debate fascinating, and I'm excited for more people realizing that a) We have a very, very blurry definition of intelligence, and b) As more people get convinced there is some intelligence in software, more people get convinced there is (or there might be) intelligence in animals. Whatever the bottom line on current LLMs is, the cultural change they might foster makes me smile.