5 ms·

I felt like that too, at first. I was really excited by what LLMs seem capable of. But when I started implementing one from scratch I realized that it's just li

I felt like that too, at first. I was really excited by what LLMs seem capable of. But when I started implementing one from scratch I realized that it's just linear algebra picking out the most probable match to a prompt. I'm afraid I had to come to the conclusion that LLMs, although being fun to play around with sometimes and seeming surprising at first, it's a similar kind of fun and surprise to sleight-of-hand. Like sleight-of-hand, and unlike real general intelligences (i.e. other people), it seems less and less fun and surprising the more exposed to it I become. I guess this won't change your mind though. Maybe you've understood more about them than I have, and maybe I'm just grumpy and cynical.

famouswaffles 3y ago
I used to think planes flew. I was really excited by what they seemed capable of when I heard about them. But when I saw one in person, I was disappointed. I saw no flapping. I saw no feathers. Sure I saw it soar through the clouds but that's just sleight of hand I'm sure. A trick. Planes surely do not fly.
- enasterosophes 3y ago
  Is this meant to be a refutation? I didn't say that I had only "heard about" LLMs. I said I played around with them. If you were being faithful to what I wrote, you would take in the full thing, including the part about how LLMs become less fun and surprising the more you play with them. Planes do not fly less effectively the more experienced you become at piloting them. So I can only conclude that you did not come here to have a discussion in good faith. So what are you really after? Do you think you'll change my mind with your poetry?
  - famouswaffles 3y ago
    It's an analogy for your flawed thinking. The situation is even worse than my story outlined because you understand neither the workings of the brain and how it is intelligent nor what the computation that preceeded a prediction of a language model represents. You don't know x and you don't y but you are convinced x=/y. >the part about how LLMs become less fun and surprising the more you play with them. Do they ? I don't think so. >Planes do not fly less effectively the more experienced you become at piloting them. How well a plane flies is an objective measure. How "fun" and "surprising" something is is largely subjective and can change on grounds that have nothing to do with what is being evaluated. Your refutation boils down to "It's just linear algebra!". Setting aside the fact that you can be reductionist about anything including biology, Okay? So what? Results matter not vague and meaningless philosophical ramblings.
    - alienicecream 3y ago
      But the results are indicative of what it is, and they fall short even if your only criteria is usefulness. It can't tell the difference between truth and falsity, let alone do more interesting things like reflecting on an experience, which our brains, however they might work, can. The more interesting question is why are people like you personally offended when someone is less than thrilled with a machine that has been programmed to do linear algebra over a large input? You seem to think that that mechanism in itself has a potential to do magic and that other people who reject that are a threat.
      - famouswaffles 3y ago
        >But the results are indicative of what it is, and they fall short even if your only criteria is usefulness There isn't a testable definition of general intelligence that GPT-4 fails that a chunk of humans also wouldn't. If they fall short then so do people. Guess those people aren't generally intelligent then according to you. >It can't tell the difference between truth and falsity Yes, it very much can. GPT-4 logits calibration pre RLHF - https://imgur.com/a/3gYel9r https://imgur.com/a/3gYel9r Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - https://arxiv.org/abs/2305.14975 https://arxiv.org/abs/2305.14975 Teaching Models to Express Their Uncertainty in Words - https://arxiv.org/abs/2205.14334 https://arxiv.org/abs/2205.14334 Language Models (Mostly) Know What They Know - https://arxiv.org/abs/2207.05221 https://arxiv.org/abs/2207.05221 The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - https://arxiv.org/abs/2310.06824 https://arxiv.org/abs/2310.06824 >The more interesting question is why are people like you personally offended I'm not offended about anything.
        alienicecream 3y ago
        > There isn't a testable definition of general intelligence that GPT-4 fails that a chunk of humans also wouldn't. Sure there is and it's a basic one: can it determine a fact?
wenc 3y ago
LLMs do not have general intelligence, but it does an awfully good job doing something human thought typically does -- synthesizing ideas. It's able to combine X and Y to produce Z. LLMs cannot do first-principles reasoning well (unless it has seen the pattern before), but they excel at reasoning through analogy. And guess what? Most humans reason analogically too. Analogical reasoning -- as opposed to first principles reasoning -- is actually more effective and useful in many situations, especially in ill-defined domains. Humans often think, "this situation looks like something that I've seen before, so I can apply my past knowledge but with a few tweaks."
HDThoreaun 3y ago
I don't see why the mechanism precludes LLM's from being intelligent. The reality is we have no idea how human intelligence works, and we certainly can't say that the human method is the only valid one. I think it's completely reasonable to conclude that linear algebra choosing the most probable output could eventually lead to a fully superinteligent system.

famouswaffles 3y ago

I used to think planes flew. I was really excited by what they seemed capable of when I heard about them. But when I saw one in person, I was disappointed. I saw no flapping. I saw no feathers. Sure I saw it soar through the clouds but that's just sleight of hand I'm sure. A trick. Planes surely do not fly.

enasterosophes 3y ago
Is this meant to be a refutation? I didn't say that I had only "heard about" LLMs. I said I played around with them. If you were being faithful to what I wrote, you would take in the full thing, including the part about how LLMs become less fun and surprising the more you play with them. Planes do not fly less effectively the more experienced you become at piloting them. So I can only conclude that you did not come here to have a discussion in good faith. So what are you really after? Do you think you'll change my mind with your poetry?
- famouswaffles 3y ago
  It's an analogy for your flawed thinking. The situation is even worse than my story outlined because you understand neither the workings of the brain and how it is intelligent nor what the computation that preceeded a prediction of a language model represents. You don't know x and you don't y but you are convinced x=/y. >the part about how LLMs become less fun and surprising the more you play with them. Do they ? I don't think so. >Planes do not fly less effectively the more experienced you become at piloting them. How well a plane flies is an objective measure. How "fun" and "surprising" something is is largely subjective and can change on grounds that have nothing to do with what is being evaluated. Your refutation boils down to "It's just linear algebra!". Setting aside the fact that you can be reductionist about anything including biology, Okay? So what? Results matter not vague and meaningless philosophical ramblings.
  - alienicecream 3y ago
    But the results are indicative of what it is, and they fall short even if your only criteria is usefulness. It can't tell the difference between truth and falsity, let alone do more interesting things like reflecting on an experience, which our brains, however they might work, can. The more interesting question is why are people like you personally offended when someone is less than thrilled with a machine that has been programmed to do linear algebra over a large input? You seem to think that that mechanism in itself has a potential to do magic and that other people who reject that are a threat.
    - famouswaffles 3y ago
      >But the results are indicative of what it is, and they fall short even if your only criteria is usefulness There isn't a testable definition of general intelligence that GPT-4 fails that a chunk of humans also wouldn't. If they fall short then so do people. Guess those people aren't generally intelligent then according to you. >It can't tell the difference between truth and falsity Yes, it very much can. GPT-4 logits calibration pre RLHF - https://imgur.com/a/3gYel9r https://imgur.com/a/3gYel9r Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - https://arxiv.org/abs/2305.14975 https://arxiv.org/abs/2305.14975 Teaching Models to Express Their Uncertainty in Words - https://arxiv.org/abs/2205.14334 https://arxiv.org/abs/2205.14334 Language Models (Mostly) Know What They Know - https://arxiv.org/abs/2207.05221 https://arxiv.org/abs/2207.05221 The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - https://arxiv.org/abs/2310.06824 https://arxiv.org/abs/2310.06824 >The more interesting question is why are people like you personally offended I'm not offended about anything.
      - alienicecream 3y ago
        > There isn't a testable definition of general intelligence that GPT-4 fails that a chunk of humans also wouldn't. Sure there is and it's a basic one: can it determine a fact?

wenc 3y ago

LLMs do not have general intelligence, but it does an awfully good job doing something human thought typically does -- synthesizing ideas. It's able to combine X and Y to produce Z. LLMs cannot do first-principles reasoning well (unless it has seen the pattern before), but they excel at reasoning through analogy. And guess what? Most humans reason analogically too. Analogical reasoning -- as opposed to first principles reasoning -- is actually more effective and useful in many situations, especially in ill-defined domains. Humans often think, "this situation looks like something that I've seen before, so I can apply my past knowledge but with a few tweaks."

HDThoreaun 3y ago

I don't see why the mechanism precludes LLM's from being intelligent. The reality is we have no idea how human intelligence works, and we certainly can't say that the human method is the only valid one. I think it's completely reasonable to conclude that linear algebra choosing the most probable output could eventually lead to a fully superinteligent system.