6 ms·

>We don't fully understand why current LLMs are bad at these tasks. In complete seriousness, can anyone can explain why LLMs are good at some tasks?

by _wire_ 2y ago

j16sdiz 2y ago
> why LLMs are good at some tasks? Like how we explain human doing tasks -- they are evolved to do that. I believe this is a non-answer, but if we are satisfied with that non answer for human, why not LLMs?
- layer8 2y ago
  I would argue that we are not satisfied with that answer for humans either.
zer00eyz 2y ago
Yes: An LLM isnt a model of human thinking. An LLM is an attempt to build a simulation of human communication. An LLM is to language what a forecast is to weather. No amount of weather data is actually going to turn that simulation into snow, no amount of LLM data is going to create AGI. That having been said, better models (smaller, more flexible ones) are going to result in a LOT of practical uses that have the potential to make our day to day lives easier (think digital personal assistant that has current knowledge).
- choeger 2y ago
  Great comment. Just one thought: Language, unlike weather, is meta-circular. All we know about specific words or sentences is again encoded in words and sentences. So the embedding encodes a subset of human knowledge. Hence, a LLM is predicting not only language but language with some sort of meaning.
  - zer00eyz 2y ago
    That re-embeding is also encoded in weather. It is why perfect forecasting is impossible, why we talk about the butterfly effect. The "hallucination problem" is simply the tyranny of Lorenz... one is not sure if a starting state will have a good outcome or swing wildly. Some good weather models are based on re-runing with tweaks to starting params, and then things that end up out of bounds can get tossed. Its harder to know when a result is out of bounds for an LLM, and we dont have the ability to run every request 100 times through various models to get an "average" output yet... However some of the reuse of layers does emulate this to an extent....
- red75prime 2y ago
  Ugh. Really? Those "simulated water isn't wet"(when applied to cognition) "arguments" were punched so many times it even hurts to look at them.
  - zer00eyz 2y ago
    No simulated water isnt wet. But an LLM isn't even trying to simulate cognition. It's a model that is predicting language. It has all the problems of a predictive model... the "hallucination" problem is just the tyranny of Lorenz.
    - adrianN 2y ago
      We don't really know what "cognition" is, so it's hard to tell whether a system is doing it.
    - lostmsu 2y ago
      This is plain wrong due to mixing of concepts. Language is technically something from Chomsky hierarchy. Predicting language is being able to tell if input is valid or invalid. LLMs do that, but they also build a statistical model across all valid inputs, and that is not just the language.
      - zer00eyz 2y ago
        >> Predicting language is being able to tell if input is valid or invalid. If this were the case then the hallucination problem would be solvable. That hallucination problem is not only going to be hard to detect in any meaningful way but it's going to be harder to eliminate. The very nature of LLM (mixing in noise aka temperature) means that they always risk going off the rails. This is the same thing Lorenz discovered in modeling weather...
        lostmsu 2y ago
        I don't think that "hallucination problem" is a problem at all worth addressing separately from just building bigger/better models that do the same thing. Because 1) it is present in humans, 2) it is clear bigger models have less of it than smaller models. If at scale nothing changes LLMs will eventually just hallucinate less than humans.
HarHarVeryFunny 2y ago
I'd guess because the Transformer architecture is (I assume) fairly close to the way that our brain learns and produces language - similar hierarchical approach and perhaps similar type of inter-embedding attention-based copying? Similar to how CNNs are so successful at image recognition, because they also roughly follow the way we do it too. Other seq-2-seq language approaches work too, but not as good as Transformers, which I'd guess is due to transformers better matching our own inductive biases, maybe due to the specific form of attention.
SomeCallMeTim 2y ago
LLMs are good at tasks that don't require actual understanding of the topic. They can come up with excellent (or excellent-looking-but-wrong) answers to any question that their training corpus covers. In a gross oversimplification, the "reasoning" they do is really just parroting a weighted average (with randomness injected) of the matching training data. What they're doing doesn't really match any definition of "understanding." An LLM (and any current AI) doesn't "understand" anything; it's effectively no more than a really big, really complicated spreadsheet. And no matter how complicated a spreadsheet gets, it's never going to understand anything. Not until we find the secret to actual learning. And increasingly it looks like actual learning probably relies on some of the quantum phenomena that are known to be present in the brain. We may not even have the science yet to understand how the brain learns. But I have become convinced that we're not going to find a way for digital-logic-based computers to bridge that gap.
- danenania 2y ago
  > the "reasoning" they do is really just parroting a weighted average (with randomness injected) of the matching training data Perhaps our brains are doing exactly the same, just with more sophistication?
  - xanderlewis 2y ago
    Every single discussion of ‘AGI’ has endless comments exactly like this. Whatever criticism is made of an attempt to produce a reasoning machine, there’s always inevitably someone who says ‘but that’s just what our brains do, duhhh… stop trying to feel special’. It’s boring, and it’s also completely content-free. This particular instance doesn’t even make sense: how can it be exactly the same, yet more sophisticated? Sorry.
    - danenania 2y ago
      As the comment I replied to very correctly said, we don’t know how the brain produces cognition. So you certainly cannot discard the hypothesis that it works through “parroting” a weighted average of training data just as LLMs are alleged to do. Considering that LLMs with a much smaller number of neurons than the brain are in many cases producing human-level output, there is some evidence, if circumstantial, that our brains may be doing something similar.
richardw 2y ago
LLM’s are a compressed and lossy form of our combined writing output, which it turns out is similarly structured enough to make new combinations of text seem reasonable, even enough to display simple reasoning. I find it useful to think “what can I expect from speaking with the dataset of combined writing of people”, rather than treating a basic LLM as a mind. That doesn’t mean we won’t end up approximating one eventually, but it’s going to take a lot of real human thinking first. For example, ChatGPT writes code to solve some questions rather than reasoning about it from text. The LLM is not doing the heavy lifting in that case. Give it (some) 3D questions or anything where there isn’t massive textual datasets and you often need to break out to specialised code. Another thought I find useful is that it considers its job done when it’s produced enough reasonable tokens, not when it’s actually solved a problem. You and I would continue to ponder the edge cases. It’s just happy if there are 1000 tokens that look approximately like its dataset. Agents make that a bit smarter but they’re still limited by the goal of being happy when each has produced the required token quota, missing eg implications that we’d see instantly. Obviously we’re smart enough to keep filling those gaps.
- tobiasSoftware 2y ago
  "I find it useful to think “what can I expect from speaking with the dataset of combined writing of people”, rather than treating a basic LLM as a mind." I've been doing this as well, mentally I think of LLMs as the librarians of the internet.
  - pbhjpbhj 2y ago
    They're bad librarians. They're not bad, they do a bad job of being librarians, which is a good thing! They can't quite tell you the exact quote, but they do recall the gist, they're not sure it was Gandhi who said that thing but they think he did, it might be in this post or perhaps one of these. They'll point you to the right section of the library to find what you're after, but make sure you verify it!
    - marquisdepolis 2y ago
      They are librarians, just that it happens to be the library of Babel.
pbhjpbhj 2y ago
If you look at transfer learning, I think that is a useful point at which to understand task-specific application and hence why LLMs excel at some tasks and not others. Tasks are specialised for using the training corpus, the attention mechanisms, the loss functions, and such. I'll leave it to others to expand on actual answers, but IMO focusing on transfer learning helps to understand how an LLM does inferences.

>We don't fully understand why current LLMs are bad at these tasks. In complete seriousness, can anyone can explain why LLMs are good at some tasks?

by _wire_ 2y ago

>We don't fully understand why current LLMs are bad at these tasks. In complete seriousness, can anyone can explain why LLMs are good at some tasks?

j16sdiz 2y ago

> why LLMs are good at some tasks? Like how we explain human doing tasks -- they are evolved to do that. I believe this is a non-answer, but if we are satisfied with that non answer for human, why not LLMs?

layer8 2y ago
I would argue that we are not satisfied with that answer for humans either.

zer00eyz 2y ago

Yes: An LLM isnt a model of human thinking. An LLM is an attempt to build a simulation of human communication. An LLM is to language what a forecast is to weather. No amount of weather data is actually going to turn that simulation into snow, no amount of LLM data is going to create AGI. That having been said, better models (smaller, more flexible ones) are going to result in a LOT of practical uses that have the potential to make our day to day lives easier (think digital personal assistant that has current knowledge).

choeger 2y ago
Great comment. Just one thought: Language, unlike weather, is meta-circular. All we know about specific words or sentences is again encoded in words and sentences. So the embedding encodes a subset of human knowledge. Hence, a LLM is predicting not only language but language with some sort of meaning.
- zer00eyz 2y ago
  That re-embeding is also encoded in weather. It is why perfect forecasting is impossible, why we talk about the butterfly effect. The "hallucination problem" is simply the tyranny of Lorenz... one is not sure if a starting state will have a good outcome or swing wildly. Some good weather models are based on re-runing with tweaks to starting params, and then things that end up out of bounds can get tossed. Its harder to know when a result is out of bounds for an LLM, and we dont have the ability to run every request 100 times through various models to get an "average" output yet... However some of the reuse of layers does emulate this to an extent....
red75prime 2y ago
Ugh. Really? Those "simulated water isn't wet"(when applied to cognition) "arguments" were punched so many times it even hurts to look at them.
- zer00eyz 2y ago
  No simulated water isnt wet. But an LLM isn't even trying to simulate cognition. It's a model that is predicting language. It has all the problems of a predictive model... the "hallucination" problem is just the tyranny of Lorenz.
  - adrianN 2y ago
    We don't really know what "cognition" is, so it's hard to tell whether a system is doing it.
  - lostmsu 2y ago
    This is plain wrong due to mixing of concepts. Language is technically something from Chomsky hierarchy. Predicting language is being able to tell if input is valid or invalid. LLMs do that, but they also build a statistical model across all valid inputs, and that is not just the language.
    - zer00eyz 2y ago
      >> Predicting language is being able to tell if input is valid or invalid. If this were the case then the hallucination problem would be solvable. That hallucination problem is not only going to be hard to detect in any meaningful way but it's going to be harder to eliminate. The very nature of LLM (mixing in noise aka temperature) means that they always risk going off the rails. This is the same thing Lorenz discovered in modeling weather...
      - lostmsu 2y ago
        I don't think that "hallucination problem" is a problem at all worth addressing separately from just building bigger/better models that do the same thing. Because 1) it is present in humans, 2) it is clear bigger models have less of it than smaller models. If at scale nothing changes LLMs will eventually just hallucinate less than humans.

HarHarVeryFunny 2y ago

I'd guess because the Transformer architecture is (I assume) fairly close to the way that our brain learns and produces language - similar hierarchical approach and perhaps similar type of inter-embedding attention-based copying? Similar to how CNNs are so successful at image recognition, because they also roughly follow the way we do it too. Other seq-2-seq language approaches work too, but not as good as Transformers, which I'd guess is due to transformers better matching our own inductive biases, maybe due to the specific form of attention.

SomeCallMeTim 2y ago

LLMs are good at tasks that don't require actual understanding of the topic. They can come up with excellent (or excellent-looking-but-wrong) answers to any question that their training corpus covers. In a gross oversimplification, the "reasoning" they do is really just parroting a weighted average (with randomness injected) of the matching training data. What they're doing doesn't really match any definition of "understanding." An LLM (and any current AI) doesn't "understand" anything; it's effectively no more than a really big, really complicated spreadsheet. And no matter how complicated a spreadsheet gets, it's never going to understand anything. Not until we find the secret to actual learning. And increasingly it looks like actual learning probably relies on some of the quantum phenomena that are known to be present in the brain. We may not even have the science yet to understand how the brain learns. But I have become convinced that we're not going to find a way for digital-logic-based computers to bridge that gap.

danenania 2y ago
> the "reasoning" they do is really just parroting a weighted average (with randomness injected) of the matching training data Perhaps our brains are doing exactly the same, just with more sophistication?
- xanderlewis 2y ago
  Every single discussion of ‘AGI’ has endless comments exactly like this. Whatever criticism is made of an attempt to produce a reasoning machine, there’s always inevitably someone who says ‘but that’s just what our brains do, duhhh… stop trying to feel special’. It’s boring, and it’s also completely content-free. This particular instance doesn’t even make sense: how can it be exactly the same, yet more sophisticated? Sorry.
  - danenania 2y ago
    As the comment I replied to very correctly said, we don’t know how the brain produces cognition. So you certainly cannot discard the hypothesis that it works through “parroting” a weighted average of training data just as LLMs are alleged to do. Considering that LLMs with a much smaller number of neurons than the brain are in many cases producing human-level output, there is some evidence, if circumstantial, that our brains may be doing something similar.

richardw 2y ago

LLM’s are a compressed and lossy form of our combined writing output, which it turns out is similarly structured enough to make new combinations of text seem reasonable, even enough to display simple reasoning. I find it useful to think “what can I expect from speaking with the dataset of combined writing of people”, rather than treating a basic LLM as a mind. That doesn’t mean we won’t end up approximating one eventually, but it’s going to take a lot of real human thinking first. For example, ChatGPT writes code to solve some questions rather than reasoning about it from text. The LLM is not doing the heavy lifting in that case. Give it (some) 3D questions or anything where there isn’t massive textual datasets and you often need to break out to specialised code. Another thought I find useful is that it considers its job done when it’s produced enough reasonable tokens, not when it’s actually solved a problem. You and I would continue to ponder the edge cases. It’s just happy if there are 1000 tokens that look approximately like its dataset. Agents make that a bit smarter but they’re still limited by the goal of being happy when each has produced the required token quota, missing eg implications that we’d see instantly. Obviously we’re smart enough to keep filling those gaps.

tobiasSoftware 2y ago
"I find it useful to think “what can I expect from speaking with the dataset of combined writing of people”, rather than treating a basic LLM as a mind." I've been doing this as well, mentally I think of LLMs as the librarians of the internet.
- pbhjpbhj 2y ago
  They're bad librarians. They're not bad, they do a bad job of being librarians, which is a good thing! They can't quite tell you the exact quote, but they do recall the gist, they're not sure it was Gandhi who said that thing but they think he did, it might be in this post or perhaps one of these. They'll point you to the right section of the library to find what you're after, but make sure you verify it!
  - marquisdepolis 2y ago
    They are librarians, just that it happens to be the library of Babel.

pbhjpbhj 2y ago

If you look at transfer learning, I think that is a useful point at which to understand task-specific application and hence why LLMs excel at some tasks and not others. Tasks are specialised for using the training corpus, the attention mechanisms, the loss functions, and such. I'll leave it to others to expand on actual answers, but IMO focusing on transfer learning helps to understand how an LLM does inferences.