4 ms·

Even a stopped clock gives the right time, twice a day. LLMs are plausibility engines. The fundamental hypothesis on trial here is that increasing plausibility

Even a stopped clock gives the right time, twice a day. LLMs are plausibility engines. The fundamental hypothesis on trial here is that increasing plausibility corresponds to increasing correctness. This is easily rejected for the human-sourced content that trains them, implying an upper bound on any dependent phenomenon. It follows that simply scaling LLMs will not produce AGI.

FeepingCreature 3y ago
You're assuming that humans reliably pursue correctness rather than plausibility. The whole scientific system specifically being designed to suppress plausible-sounding-but-incorrect claims says otherwise.
- happytiger 3y ago
  That’s an astute observation.
- Jensson 3y ago
  > The whole scientific system specifically being designed to suppress plausible-sounding-but-incorrect claims says otherwise. But that we have came up with a scientific system is proof that humans actually can tell the difference, we might be bad at it but we still manage to do it. Humans that can't tell the difference are very unreliable and usually shunned in engineering and other STEM fields.
  - layer8 3y ago
    Doesn’t a claim become instantly implausible when we see how it is incorrect? It's only plausible as long as it might still turn out to be correct. Furthermore, sometimes mistakes are made when assessing correctness, even in mathematical proofs. A correctness judgement thus still has a degree of plausibility associated with it. I'm thus not sure if the distinction being made here isn't just one of degree.
    - Jensson 3y ago
      > I'm thus not sure if the distinction being made here isn't just one of degree. I think humans has two levels here, "that sounds plausible" and "I can't imagine how this could be wrong so I am willing to risk my life on it". The second can still be wrong, but usually it isn't. For example when you walk on a bridge, you assume the bridge is there and confidently put down your foot even though you would die if that part of the bridge happened to be an hallucination. So science is a set of statements and experiments that sums up to facts that humans can't imagine are wrong. Sometimes they are still a bit wrong, but then only slightly like newtons equations or theories that aren't backed up by experiments. But in general science is full of facts that humans stake their life on every day. LLMs doesn't seem to have that certainty layer, something so certain that the LLM would stake its life on it. I'd for sure never stake my life on anything an LLM says as it is today, but I happily do it for humans every time I take the elevator etc.
      - layer8 3y ago
        I would count your bridge example firmly in the plausibility category, in that I would find the bridge implausible, but not impossible, to collapse. In Bayesian speak, it’s a matter of credence. We usually don’t consciously think about it when our brain decides it's good enough with sufficient certainty, but in the end it's still based on a likelihood judgement.
  - FeepingCreature 3y ago
    Right, but the point is the scientific system is a cultural accomplishment, it didn't get hardcoded into our brains. We should hence not expect "only believe things that are actually true" as a standard for AGI either, hence LLMs are not necessarily disqualified by being bad at it.
    - Jensson 3y ago
      For living beings there is a class of errors that kills you, there is no coming back from that. Our brains were trained to figure out what facts it is willing to stake its life on, I am sure all bigger living beings has such a part of their brain, so I'd say that yes we have that programmed into us. The scientific method is just a way to make humans use that reliable part of their brain instead of the plausible part, even in situations that aren't life or death. I don't think it is possible to come up with something like the scientific method without having a built-in sense of what facts are.
      - pixl97 3y ago
        >without having a built-in sense of what facts are. There is no such thing in 'higher intelligence' beyond protoception. Pretty much every mammal has the same built in sense that you do, and yet none of them are carting around human intelligence. More so, when human children are left feral and not raised around other humans we typically consider them mentally disabled. Again, they learn the protoception stuff just fine, but all the stuff that "puts us above" the mammals is missing, it is not inane. Your counting, reading, and tools use are all learned behaviors after birth and form a continuous chain of learned behaviors taught by people (and more recently books and computers). There is a reason that after development of the scientific method that most human endeavors experienced an exponential growth curve. It's because we don't recognise facts from fiction at a much higher rate than other animals, it's that we teach each other fact from fiction better than other animals.
        Jensson 3y ago
        > Pretty much every mammal has the same built in sense that you do, and yet none of them are carting around human intelligence Having a sense of what are facts and what are assumptions doesn't mean you are smart, but not having it means you are pretty dumb. LLMs doesn't have that. Natural language is more powerful than programming languages since it lets us communicate assumptions, guesses, uncertainty etc, to other intelligent beings. So a good LLM needs to be able to consider certainty level of statements and act accordingly, todays LLMs doesn't do that so generate dumber responses than a human would since they lack a sanity checker.
  - deleted 3y ago
    [deleted]
hnben 3y ago
> implying an upper bound on any dependent phenomenon the implication sounds good, but is easily rejected with a counter example: a good student can surpass his master. A mediorcre student with multiple masters can surpass all of them (tangent: this is why the medival master-journeyman system was so efficient, I think). Or to make the argument more abstract: Your implication seems to assume, that transferlearning does not exist.
- inopinatus 3y ago
  LLMs do not study. I’ll repeat an assertion I’ve made in this forum before: that the benchmark for AGI will be when two AIs play one another at an incomprehensible game of their own devising. However, there is a hidden corollary in my earlier statement. By implying an upper bound on the correctness of their answers, we can assess that LLMs will not themselves achieve sentience and form a civilisational threat. That actually increases their potential utility in many applications. Much as you wouldn’t, for safety’s sake, put a jet engine in a conventional car (occasional mad-scientist projects notwithstanding).
fl7305 3y ago
> LLMs are plausibility engines. This is one way to look at LLMs. But it doesn't automatically impose an upper bound on their capabilities. Humans are reproductive organisms. This is also true. At a glance, it would seem that humans could not evolve to be intelligent. They are only selected on their reproductive abilities. But obviously the categorization, while true, doesn't impose an upper bound on human abilities. LLMs are evolved to be as efficient as possible at retaining knowledge. A simple strategy for retaining knowledge is memorization. Neural networks definitely can do that. A different strategy for retaining knowledge is using algorithms. Neural networks can evolve to retain knowledge using algorithms too. For instance, it has been shown that a small neural network evolved FFT-like structures to perform addition. It started out with memorization, which was far from perfect, but later in the training it switched to an algorithm using FFT for addition which produced perfect results. I think the better LLMs retain knowledge through sophisticated compression. It involves building a world model and ways to relate the input text to that model. I see it as the building blocks of a reasoning machine. It is incomplete and buggy, and the current architectures might hit a ceiling soon, but nonetheless it is a completely different thing than pure memorization.
- nyrikki 3y ago
  FFT is easy fairly easy for transformers to find as a pattern. Note that OpenAI admits that pre-training (pattern finding and matching) is where the abilities on professional tests comes from. Even proofs in presburger arithmetic have an upper bound on the second level of the polynomial hierarchy, which is way beyond the computational power of any LLM https://arxiv.org/abs/2207.00729 https://arxiv.org/abs/2207.00729 >The model’s capabilities on exams appear to stem primarily from the pre-training process and are not significantly affected by RLHF. On multiple choice questions, both the base GPT-4 model and the RLHF model perform equally well on average across the exams we tested https://arxiv.org/abs/2303.08774 https://arxiv.org/abs/2303.08774 To practically scale, parallelism is required, and the ability to find algorithms within the L or TC0 complexity classes is limited. Remember that individual ANNs as used in transformers are just binary linear classifiers. Which you can do a lot with, but probably limited to P assuming we don't find out L or TC0 are larger than we think now. Presburger arithmetic, or first order logic with (+,=), or (*,=) is the strongest FoL that is both complete and consistent. move up to Peano arithmetic and you start to hit the limits from Gödel, Church, and Turing. There will be instances that an LLM can learn with CoT, zero shot etc... But most of those will be due to luck or parallel patterns in the training set. LLMs simplicity bias is great for plausible answers to our of distribution questions, but puts limits on what can be learned as far as 'algorithms' go. They will work for 'there exists' problems far better than for any 'for all' or 'for most' problems unless they have learnable patterns in the training set.
gitfan86 3y ago
That is true, but only in a philosophical sense. AI is on a path to outperform humans at a large number of tasks and jobs that previously would have been described as requiring intelligence.

FeepingCreature 3y ago

You're assuming that humans reliably pursue correctness rather than plausibility. The whole scientific system specifically being designed to suppress plausible-sounding-but-incorrect claims says otherwise.

happytiger 3y ago
That’s an astute observation.
Jensson 3y ago
> The whole scientific system specifically being designed to suppress plausible-sounding-but-incorrect claims says otherwise. But that we have came up with a scientific system is proof that humans actually can tell the difference, we might be bad at it but we still manage to do it. Humans that can't tell the difference are very unreliable and usually shunned in engineering and other STEM fields.
- layer8 3y ago
  Doesn’t a claim become instantly implausible when we see how it is incorrect? It's only plausible as long as it might still turn out to be correct. Furthermore, sometimes mistakes are made when assessing correctness, even in mathematical proofs. A correctness judgement thus still has a degree of plausibility associated with it. I'm thus not sure if the distinction being made here isn't just one of degree.
  - Jensson 3y ago
    > I'm thus not sure if the distinction being made here isn't just one of degree. I think humans has two levels here, "that sounds plausible" and "I can't imagine how this could be wrong so I am willing to risk my life on it". The second can still be wrong, but usually it isn't. For example when you walk on a bridge, you assume the bridge is there and confidently put down your foot even though you would die if that part of the bridge happened to be an hallucination. So science is a set of statements and experiments that sums up to facts that humans can't imagine are wrong. Sometimes they are still a bit wrong, but then only slightly like newtons equations or theories that aren't backed up by experiments. But in general science is full of facts that humans stake their life on every day. LLMs doesn't seem to have that certainty layer, something so certain that the LLM would stake its life on it. I'd for sure never stake my life on anything an LLM says as it is today, but I happily do it for humans every time I take the elevator etc.
    - layer8 3y ago
      I would count your bridge example firmly in the plausibility category, in that I would find the bridge implausible, but not impossible, to collapse. In Bayesian speak, it’s a matter of credence. We usually don’t consciously think about it when our brain decides it's good enough with sufficient certainty, but in the end it's still based on a likelihood judgement.
- FeepingCreature 3y ago
  Right, but the point is the scientific system is a cultural accomplishment, it didn't get hardcoded into our brains. We should hence not expect "only believe things that are actually true" as a standard for AGI either, hence LLMs are not necessarily disqualified by being bad at it.
  - Jensson 3y ago
    For living beings there is a class of errors that kills you, there is no coming back from that. Our brains were trained to figure out what facts it is willing to stake its life on, I am sure all bigger living beings has such a part of their brain, so I'd say that yes we have that programmed into us. The scientific method is just a way to make humans use that reliable part of their brain instead of the plausible part, even in situations that aren't life or death. I don't think it is possible to come up with something like the scientific method without having a built-in sense of what facts are.
    - pixl97 3y ago
      >without having a built-in sense of what facts are. There is no such thing in 'higher intelligence' beyond protoception. Pretty much every mammal has the same built in sense that you do, and yet none of them are carting around human intelligence. More so, when human children are left feral and not raised around other humans we typically consider them mentally disabled. Again, they learn the protoception stuff just fine, but all the stuff that "puts us above" the mammals is missing, it is not inane. Your counting, reading, and tools use are all learned behaviors after birth and form a continuous chain of learned behaviors taught by people (and more recently books and computers). There is a reason that after development of the scientific method that most human endeavors experienced an exponential growth curve. It's because we don't recognise facts from fiction at a much higher rate than other animals, it's that we teach each other fact from fiction better than other animals.
      - Jensson 3y ago
        > Pretty much every mammal has the same built in sense that you do, and yet none of them are carting around human intelligence Having a sense of what are facts and what are assumptions doesn't mean you are smart, but not having it means you are pretty dumb. LLMs doesn't have that. Natural language is more powerful than programming languages since it lets us communicate assumptions, guesses, uncertainty etc, to other intelligent beings. So a good LLM needs to be able to consider certainty level of statements and act accordingly, todays LLMs doesn't do that so generate dumber responses than a human would since they lack a sanity checker.
- deleted 3y ago
  [deleted]

hnben 3y ago

> implying an upper bound on any dependent phenomenon the implication sounds good, but is easily rejected with a counter example: a good student can surpass his master. A mediorcre student with multiple masters can surpass all of them (tangent: this is why the medival master-journeyman system was so efficient, I think). Or to make the argument more abstract: Your implication seems to assume, that transferlearning does not exist.

inopinatus 3y ago
LLMs do not study. I’ll repeat an assertion I’ve made in this forum before: that the benchmark for AGI will be when two AIs play one another at an incomprehensible game of their own devising. However, there is a hidden corollary in my earlier statement. By implying an upper bound on the correctness of their answers, we can assess that LLMs will not themselves achieve sentience and form a civilisational threat. That actually increases their potential utility in many applications. Much as you wouldn’t, for safety’s sake, put a jet engine in a conventional car (occasional mad-scientist projects notwithstanding).

fl7305 3y ago

> LLMs are plausibility engines. This is one way to look at LLMs. But it doesn't automatically impose an upper bound on their capabilities. Humans are reproductive organisms. This is also true. At a glance, it would seem that humans could not evolve to be intelligent. They are only selected on their reproductive abilities. But obviously the categorization, while true, doesn't impose an upper bound on human abilities. LLMs are evolved to be as efficient as possible at retaining knowledge. A simple strategy for retaining knowledge is memorization. Neural networks definitely can do that. A different strategy for retaining knowledge is using algorithms. Neural networks can evolve to retain knowledge using algorithms too. For instance, it has been shown that a small neural network evolved FFT-like structures to perform addition. It started out with memorization, which was far from perfect, but later in the training it switched to an algorithm using FFT for addition which produced perfect results. I think the better LLMs retain knowledge through sophisticated compression. It involves building a world model and ways to relate the input text to that model. I see it as the building blocks of a reasoning machine. It is incomplete and buggy, and the current architectures might hit a ceiling soon, but nonetheless it is a completely different thing than pure memorization.

nyrikki 3y ago
FFT is easy fairly easy for transformers to find as a pattern. Note that OpenAI admits that pre-training (pattern finding and matching) is where the abilities on professional tests comes from. Even proofs in presburger arithmetic have an upper bound on the second level of the polynomial hierarchy, which is way beyond the computational power of any LLM https://arxiv.org/abs/2207.00729 https://arxiv.org/abs/2207.00729 >The model’s capabilities on exams appear to stem primarily from the pre-training process and are not significantly affected by RLHF. On multiple choice questions, both the base GPT-4 model and the RLHF model perform equally well on average across the exams we tested https://arxiv.org/abs/2303.08774 https://arxiv.org/abs/2303.08774 To practically scale, parallelism is required, and the ability to find algorithms within the L or TC0 complexity classes is limited. Remember that individual ANNs as used in transformers are just binary linear classifiers. Which you can do a lot with, but probably limited to P assuming we don't find out L or TC0 are larger than we think now. Presburger arithmetic, or first order logic with (+,=), or (*,=) is the strongest FoL that is both complete and consistent. move up to Peano arithmetic and you start to hit the limits from Gödel, Church, and Turing. There will be instances that an LLM can learn with CoT, zero shot etc... But most of those will be due to luck or parallel patterns in the training set. LLMs simplicity bias is great for plausible answers to our of distribution questions, but puts limits on what can be learned as far as 'algorithms' go. They will work for 'there exists' problems far better than for any 'for all' or 'for most' problems unless they have learnable patterns in the training set.

gitfan86 3y ago

That is true, but only in a philosophical sense. AI is on a path to outperform humans at a large number of tasks and jobs that previously would have been described as requiring intelligence.