3 ms·

I agree that's an argument. I would contend that argument is obviously false. If it were true, LLMs could multiply scalar numbers together trivially. It shou

I agree that's an argument. I would contend that argument is obviously false. If it were true, LLMs could multiply scalar numbers together trivially. It should be the easiest thing in the world for them. The network required to do that well is extremely small, the parameter sizes of these models are gigantic, and the textual expression is highly regular: multiplication is the simplest concept imaginable. That they cannot do that basic task implies to me that they have almost no conceptual understanding unless the fit is almost memorizable or the space is highly regular. That LLMs can't multiply numbers properly isn't surprising if they don't really understand concepts prior to emitting text. Where they do logical tasks, that can be done with minimal or no understanding, because syllogisms and logical formalisms are highly structured in text arguments.

GaggiX 2y ago
Multiplication requires O(n^2) complexity with the usual algorithm used by humans, LLMs have a constant amount of computation available and they are not really efficient machines for math evaluation. They can definitely evaluate unseen expressions and you train a neural network to learn how to do sums and multiplications, I have trained models on sums and they are able to do sums never seen during training, the model learns the algorithm just by giving it inputs and outputs.
jdietrich 2y ago
LLMs do contain conceptual representations and LLMs are capable of abstract reasoning. This is trivially provable by asking them to reason about something that is a) purely abstract and b) not in the training data, e.g. "All floots are gronks. Some gronks are klorps. Are any floots klorps?" Any of the leading LLMs will correctly answer questions of this type much more often than chance.
- LetsGetTechnicl 2y ago
  That is not an example of a LLM being capable of abstract reasoning. Changing the question from "What is the capital of United States?" which is easily answerable to something completely abstract and "not in the training model" doesn't change that LLM's are just very advanced text prediction, and always will be. The nature of their design means they are incapable of AGI.
  - jdietrich 2y ago
    The question I gave is a literal textbook example of abstract reasoning. LLMs are just very advanced text prediction, but they are also provably capable of abstract reasoning. If you think that those statements are contradictory, I would encourage you to read up on the Bayesian hypotheses in cognitive science - it is highly plausible that our brains are also just very advanced prediction models.
    - nsagent 2y ago
      You're quite right that LLMs can seemingly do some abstract reasoning problems, but I would not say they aren't in the training data. Sure, the exact form using the made up word gronk might not be in the training data, but the general form of that reasoning problem definitely exists, quite frequently in fact.
      - jdietrich 2y ago
        Yes, but the general form of the problem tells you nothing about the answer to any specific case. To perform any better than chance, the model has to actually reason through the problem.
      - cgag 2y ago
        Have you seen this? ``` You will be given a name of an object (such as Car, Chair, Elephant) and a letter in the alphabet. Your goal is to first produce a 1-line description of how that object can be combined with the letter in an image (for example, for an elephant and the letter J, the trunk of the elephant can have a J shape, and for the letter A and a house, the house can have an A shape with the upper triangle of the A being the roof). Following the short description, please create SVG code to produce this (in the SVG use shapes like ellipses, triangles etc and polygons but try to defer from using quadratic curves). ``` ``` Round 5: A car and the letter E. Description: The car has an E shape on its front bumper, with the horizontal lines of the E being lights and the vertical line being the license plate. ``` Image generated here: https://imgur.com/a/Ia4Q2h3 https://imgur.com/a/Ia4Q2h3 How does it "just" predict the letter E could be used in such a way to draw a car? How does it just text predict working SVG code that draws the car made out of basic shapes and the letter E? I don't know how anyone could suggest there are no conceptual models embedded in there.
- stirfish 2y ago
  I just asked chatgpt "All floots are gronks. Some gronks are klorps. Are any floots klorps?" ------ To determine if any floots are klorps, let's analyze the given statements: 1. All floots are gronks. This means every floot falls into the category of gronks. 2. Some gronks are klorps. This means there is an overlap between the set of gronks and the set of klorps. Since all floots are included in the set of gronks and some gronks are klorps, it is possible that some floots are klorps. However, we cannot conclusively say that any floots are klorps without additional information. It is only certain that if there is any overlap between floots and klorps, it is possible, but not guaranteed, that some floots are klorps.
  - card_zero 2y ago
    Huh, almost right. ("possible, but not guaranteed?" it's necessarily true. That whole sentence was a waste of space, and wrong.) Edit: I mean "if there is any overlap", it's necessarily true. I should have quoted the whole thing.
    - jdietrich 2y ago
      Nope, ChatGPT was right, the answer is indeterminable. The klorps that are gronks could be a wholly distinct subset to the klorps that are floots. It also correctly evaluates "All gronks are floots. Some gronks are klorps. Are any floots klorps?", to which the answer is definitively yes.
      - card_zero 2y ago
        > The klorps that are gronks could be a wholly distinct subset to the klorps that are floots. So? It's still the case that "if there is any overlap between floots and klorps," it is "guaranteed, that some floots are klorps." It's tautological. Unless there's a way to read "overlap" so that it doesn't mean "some of one category are also in the other category, and vice versa"? Oh, when I said "it's necessarily true" I was refering to this last sentence of the output, not the question posed in the input. Hence we are at cross purposes I think.
- wonnage 2y ago
  Or maybe they're just pattern matching on the very particular sentence structure you've chosen. This isn't a convincing example at all
  - jdietrich 2y ago
    This isn't something I should convince you of. Just open up ChatGPT or Claude and try it for yourself. Think up a batch of your own questions and see how a modern LLM fares. I assure you that it'll do much better than chance. If you're so inclined, you can run enough tests to achieve statistical significance in the course of your lunch break. It depresses me that we seem to be spending more time arguing and hypothesising about LLMs than empirically testing them. The question of whether LLMs can think is completely settled, as their performance at zero-shot problems is simply impossible through pure memorisation or pattern-matching. The question that remains is far more interesting - how do they think? https://arxiv.org/pdf/2205.11916 https://arxiv.org/pdf/2205.11916
    - nickpsecurity 2y ago
      Given their training set, our hypothesis so far should be that they're just tweaking things they've already seen by applying a series of simple rules. They're still not doing what human beings do. We have introspection, creativity operating outside what we've seen, modeling others' thoughts, planning in new domains, and so on. We also operate without hallucination most of the time. I've yet to see an A.I. do all of this reliably and consistently. Then, that it did that without training input similar to the output. So, they don't just pattern match or purely memorize. They do more than that. They do way less than humans. Unlike humans, they also try to do everything with one or a few components vs our (100-200?) brain components. Crossing that gap might be achievable. It will not be done by current architectures, though.
- abernard1 2y ago
  This does not indicate abstract reasoning. I said: > Where they do logical tasks, that can be done with minimal or no understanding, because syllogisms and logical formalisms are highly structured in text arguments. There is an enormous amount of text in the training set that is structured in the way you said such that syntactic replacement would be effective. That is also unsurprising and does not represent abstract reasoning any more than "King - Man + Woman = Queen" in word2vec. It's showing that there's high degrees of structure in syllogisms, and that it need know nothing about what a gronk, floot, or klorp is at all because the structure of the syllogism is repeated all over the internet.
deleted 2y ago
[deleted]
Zambyte 2y ago
> If it were true, LLMs could multiply scalar numbers together trivially. FWIW most large models can do it better than I can in my head.
famouswaffles 2y ago
>If it were true, LLMs could multiply scalar numbers together trivially. I mean, it's not like GPT-4 can't do this with more accuracy than a human without a calculator.

GaggiX 2y ago

Multiplication requires O(n^2) complexity with the usual algorithm used by humans, LLMs have a constant amount of computation available and they are not really efficient machines for math evaluation. They can definitely evaluate unseen expressions and you train a neural network to learn how to do sums and multiplications, I have trained models on sums and they are able to do sums never seen during training, the model learns the algorithm just by giving it inputs and outputs.

jdietrich 2y ago

LLMs do contain conceptual representations and LLMs are capable of abstract reasoning. This is trivially provable by asking them to reason about something that is a) purely abstract and b) not in the training data, e.g. "All floots are gronks. Some gronks are klorps. Are any floots klorps?" Any of the leading LLMs will correctly answer questions of this type much more often than chance.

LetsGetTechnicl 2y ago
That is not an example of a LLM being capable of abstract reasoning. Changing the question from "What is the capital of United States?" which is easily answerable to something completely abstract and "not in the training model" doesn't change that LLM's are just very advanced text prediction, and always will be. The nature of their design means they are incapable of AGI.
- jdietrich 2y ago
  The question I gave is a literal textbook example of abstract reasoning. LLMs are just very advanced text prediction, but they are also provably capable of abstract reasoning. If you think that those statements are contradictory, I would encourage you to read up on the Bayesian hypotheses in cognitive science - it is highly plausible that our brains are also just very advanced prediction models.
  - nsagent 2y ago
    You're quite right that LLMs can seemingly do some abstract reasoning problems, but I would not say they aren't in the training data. Sure, the exact form using the made up word gronk might not be in the training data, but the general form of that reasoning problem definitely exists, quite frequently in fact.
    - jdietrich 2y ago
      Yes, but the general form of the problem tells you nothing about the answer to any specific case. To perform any better than chance, the model has to actually reason through the problem.
    - cgag 2y ago
      Have you seen this? ``` You will be given a name of an object (such as Car, Chair, Elephant) and a letter in the alphabet. Your goal is to first produce a 1-line description of how that object can be combined with the letter in an image (for example, for an elephant and the letter J, the trunk of the elephant can have a J shape, and for the letter A and a house, the house can have an A shape with the upper triangle of the A being the roof). Following the short description, please create SVG code to produce this (in the SVG use shapes like ellipses, triangles etc and polygons but try to defer from using quadratic curves). ``` ``` Round 5: A car and the letter E. Description: The car has an E shape on its front bumper, with the horizontal lines of the E being lights and the vertical line being the license plate. ``` Image generated here: https://imgur.com/a/Ia4Q2h3 https://imgur.com/a/Ia4Q2h3 How does it "just" predict the letter E could be used in such a way to draw a car? How does it just text predict working SVG code that draws the car made out of basic shapes and the letter E? I don't know how anyone could suggest there are no conceptual models embedded in there.
stirfish 2y ago
I just asked chatgpt "All floots are gronks. Some gronks are klorps. Are any floots klorps?" ------ To determine if any floots are klorps, let's analyze the given statements: 1. All floots are gronks. This means every floot falls into the category of gronks. 2. Some gronks are klorps. This means there is an overlap between the set of gronks and the set of klorps. Since all floots are included in the set of gronks and some gronks are klorps, it is possible that some floots are klorps. However, we cannot conclusively say that any floots are klorps without additional information. It is only certain that if there is any overlap between floots and klorps, it is possible, but not guaranteed, that some floots are klorps.
- card_zero 2y ago
  Huh, almost right. ("possible, but not guaranteed?" it's necessarily true. That whole sentence was a waste of space, and wrong.) Edit: I mean "if there is any overlap", it's necessarily true. I should have quoted the whole thing.
  - jdietrich 2y ago
    Nope, ChatGPT was right, the answer is indeterminable. The klorps that are gronks could be a wholly distinct subset to the klorps that are floots. It also correctly evaluates "All gronks are floots. Some gronks are klorps. Are any floots klorps?", to which the answer is definitively yes.
    - card_zero 2y ago
      > The klorps that are gronks could be a wholly distinct subset to the klorps that are floots. So? It's still the case that "if there is any overlap between floots and klorps," it is "guaranteed, that some floots are klorps." It's tautological. Unless there's a way to read "overlap" so that it doesn't mean "some of one category are also in the other category, and vice versa"? Oh, when I said "it's necessarily true" I was refering to this last sentence of the output, not the question posed in the input. Hence we are at cross purposes I think.
wonnage 2y ago
Or maybe they're just pattern matching on the very particular sentence structure you've chosen. This isn't a convincing example at all
- jdietrich 2y ago
  This isn't something I should convince you of. Just open up ChatGPT or Claude and try it for yourself. Think up a batch of your own questions and see how a modern LLM fares. I assure you that it'll do much better than chance. If you're so inclined, you can run enough tests to achieve statistical significance in the course of your lunch break. It depresses me that we seem to be spending more time arguing and hypothesising about LLMs than empirically testing them. The question of whether LLMs can think is completely settled, as their performance at zero-shot problems is simply impossible through pure memorisation or pattern-matching. The question that remains is far more interesting - how do they think? https://arxiv.org/pdf/2205.11916 https://arxiv.org/pdf/2205.11916
  - nickpsecurity 2y ago
    Given their training set, our hypothesis so far should be that they're just tweaking things they've already seen by applying a series of simple rules. They're still not doing what human beings do. We have introspection, creativity operating outside what we've seen, modeling others' thoughts, planning in new domains, and so on. We also operate without hallucination most of the time. I've yet to see an A.I. do all of this reliably and consistently. Then, that it did that without training input similar to the output. So, they don't just pattern match or purely memorize. They do more than that. They do way less than humans. Unlike humans, they also try to do everything with one or a few components vs our (100-200?) brain components. Crossing that gap might be achievable. It will not be done by current architectures, though.
abernard1 2y ago
This does not indicate abstract reasoning. I said: > Where they do logical tasks, that can be done with minimal or no understanding, because syllogisms and logical formalisms are highly structured in text arguments. There is an enormous amount of text in the training set that is structured in the way you said such that syntactic replacement would be effective. That is also unsurprising and does not represent abstract reasoning any more than "King - Man + Woman = Queen" in word2vec. It's showing that there's high degrees of structure in syllogisms, and that it need know nothing about what a gronk, floot, or klorp is at all because the structure of the syllogism is repeated all over the internet.

deleted 2y ago

[deleted]

Zambyte 2y ago

> If it were true, LLMs could multiply scalar numbers together trivially. FWIW most large models can do it better than I can in my head.

famouswaffles 2y ago

>If it were true, LLMs could multiply scalar numbers together trivially. I mean, it's not like GPT-4 can't do this with more accuracy than a human without a calculator.