4 ms·

We can prove that transformers can do computation beyond memorization, and that they can learn it from examples, unless you think learning an algorithm is memor

We can prove that transformers can do computation beyond memorization, and that they can learn it from examples, unless you think learning an algorithm is memorization. https://arxiv.org/abs/2301.05217 https://arxiv.org/abs/2301.05217 https://arxiv.org/abs/2310.16028 https://arxiv.org/abs/2310.16028

bamboozled 3y ago
Maybe there doing something we don’t really have a name for yet ? This is why it causes so much controversy. Not really generalizing, not memorizing, maybe approximating ?
- dataflow 3y ago
  "Vibing"?
  - mirekrusin 3y ago
    No need to come up with any new name, we already have it: 1. autocompletion or 2. next token prediction and 3. (reverse) diffusion
- ein0p 3y ago
  There is a name for this already: associative memory. That’s how you catch a ball: you condition your memory of catching a ball with proprioceptive and visual input, much like a multimodal transformer. There’s no thinking involved - you wouldn’t be able to catch it if you had to think
  - vidarh 3y ago
    An associative memory can model truth tables of logic circuits, and so the steps to go from associative memory to arbitrary computation pretty much requires adding state and a loop. And we do provide state and a loop. As such, while that does not prove that any given model is capable of reasoning, describing what goes on at any step as associative memory says nothing meaningful about the computational power of the wider system. It's very possible the way we train current models will turn out to place fundamental limits on the abilities the models will have, but that will not be because the models act as associative memories.
    - ein0p 3y ago
      I think the current models will be a fundamental part of larger, more comprehensive models, and they will handle the “easy” parts, much like we, humans, use our associative memory to be able to do things in realtime, “without thinking”. Yann LeCun wrote a paper on this which more people should read: https://openreview.net/pdf?id=BZ5a1r-kVsf https://openreview.net/pdf?id=BZ5a1r-kVsf. Moreover, Meta’s research is starting to move in that direction https://ai.meta.com/blog/v-jepa-yann-lecun-ai-model-video-joint-embedding-predictive-architecture/ https://ai.meta.com/blog/v-jepa-yann-lecun-ai-model-video-jo...
- crotchfire 3y ago
  > Not really generalizing, not memorizing, maybe approximating ? The term you are searching for is confabulating.
- spuz 3y ago
  There is certainly a lot of discussion around semantics to be had before we can agree on what exactly an LLM is capable of. We know that LLMs tend not to be good at math. Some people will say that the fact that an LLM cannot sum two numbers demonstrates they cannot generalise. Yet others would say the way LLMs calculate sums is generalisation because it mirrors our own process of addition which works mostly by memorisation and a lot of double checking. My perspective is that the "thinking" that LLMs do is a lot closer to the kind that humans do which is to say a lot of pattern matching but with no fundamentally precise logic underlying it. If LLMs are flawed in some way then humans are also flawed in the same way.
- deleted 3y ago
  [deleted]
dataflow 3y ago
I guess what constitutes memorization depends on what you consider "learning an algorithm". Memorizing to humans doesn't really mean learning the exact input/output pairs per se. Like a student might "learn" an algorithm for differentiation (d/dx x^n = n x^(n-1)) and then differentiate 732638 x^2 just fine despite never seeing it before, but then tell you the derivative of yx^2 with respect to y is 2yx, or something. Did they really learn how to differentiate or did they just learn a common vibe around it? When teachers see that sort of regurgitation, they call it memorization, despite the input being unique from what the student had seen in the past.
saberience 3y ago
Surely learning an algorithm is memorization? I still remember the physics equations from when I was at school (a long time ago) because it was drilled into us repeatedly. Stuff like v = u + at, or s=tu + 1/2 (at)^2, etc etc. I don't really recall how they were derived etc but I did memorize them. It definitely doesn't mean I'm good at physics or understand the deeper meaning of these equations. I also had to memorize bubble sort, merge, heap, quicksort etc when I was in university, but I don't think I could invent these sorts from first principles without looking it up. Memorization doesn't really equal understanding, it's just memorization.
- sushibowl 3y ago
  > Surely learning an algorithm is memorization? Not in this context, no. An LLM is never given any algorithm to memorise. It is given only input->output sets, and it "learns" what the algorithm is from those sets. We know that it does this because it is able to generalize that algorithm to inputs and outputs outside of its set of training examples. So we know that it doesn't only memorise which input connects to which output, and regurgitate that information. It has come to "understand" the formula that connects the input to the output, without ever being given that formula.
  - quantum_state 3y ago
    It’s fitting a function to data and guessing the value of the function for a new input. It does not know what is under the hood of the function, etc., and see the implications when the function produces a wrong value … etc.
    - astrange 3y ago
      It can do that. Verifying an answer is just another algorithm it can learn. LLMs mostly can't do math but that, like most of their other flaws, is because of the tokenizer.
- retrac 3y ago
  > Surely learning an algorithm is memorization? Yes. In a sense, every algorithm can be reduced to memorization, where you precalculate all possible inputs to all possible outputs and store them in a giant lookup table.
rurban 3y ago
They might do "computation" and seem to be "intelligent", however they do lack logic and rigor, and cannot explain their reasoning. In contrast to old-school AI. Such "intelligent" monkeys might be able to write reports, lead governments or lead wars, but in engineering you need more skills than that. Which leads us to the lack of a proper definition of AGI. (https://en.wikipedia.org/wiki/Artificial_general_intelligence https://en.wikipedia.org/wiki/Artificial_general_intelligenc...)

bamboozled 3y ago

Maybe there doing something we don’t really have a name for yet ? This is why it causes so much controversy. Not really generalizing, not memorizing, maybe approximating ?

dataflow 3y ago
"Vibing"?
- mirekrusin 3y ago
  No need to come up with any new name, we already have it: 1. autocompletion or 2. next token prediction and 3. (reverse) diffusion
ein0p 3y ago
There is a name for this already: associative memory. That’s how you catch a ball: you condition your memory of catching a ball with proprioceptive and visual input, much like a multimodal transformer. There’s no thinking involved - you wouldn’t be able to catch it if you had to think
- vidarh 3y ago
  An associative memory can model truth tables of logic circuits, and so the steps to go from associative memory to arbitrary computation pretty much requires adding state and a loop. And we do provide state and a loop. As such, while that does not prove that any given model is capable of reasoning, describing what goes on at any step as associative memory says nothing meaningful about the computational power of the wider system. It's very possible the way we train current models will turn out to place fundamental limits on the abilities the models will have, but that will not be because the models act as associative memories.
  - ein0p 3y ago
    I think the current models will be a fundamental part of larger, more comprehensive models, and they will handle the “easy” parts, much like we, humans, use our associative memory to be able to do things in realtime, “without thinking”. Yann LeCun wrote a paper on this which more people should read: https://openreview.net/pdf?id=BZ5a1r-kVsf https://openreview.net/pdf?id=BZ5a1r-kVsf. Moreover, Meta’s research is starting to move in that direction https://ai.meta.com/blog/v-jepa-yann-lecun-ai-model-video-joint-embedding-predictive-architecture/ https://ai.meta.com/blog/v-jepa-yann-lecun-ai-model-video-jo...
crotchfire 3y ago
> Not really generalizing, not memorizing, maybe approximating ? The term you are searching for is confabulating.
spuz 3y ago
There is certainly a lot of discussion around semantics to be had before we can agree on what exactly an LLM is capable of. We know that LLMs tend not to be good at math. Some people will say that the fact that an LLM cannot sum two numbers demonstrates they cannot generalise. Yet others would say the way LLMs calculate sums is generalisation because it mirrors our own process of addition which works mostly by memorisation and a lot of double checking. My perspective is that the "thinking" that LLMs do is a lot closer to the kind that humans do which is to say a lot of pattern matching but with no fundamentally precise logic underlying it. If LLMs are flawed in some way then humans are also flawed in the same way.
deleted 3y ago
[deleted]

dataflow 3y ago

I guess what constitutes memorization depends on what you consider "learning an algorithm". Memorizing to humans doesn't really mean learning the exact input/output pairs per se. Like a student might "learn" an algorithm for differentiation (d/dx x^n = n x^(n-1)) and then differentiate 732638 x^2 just fine despite never seeing it before, but then tell you the derivative of yx^2 with respect to y is 2yx, or something. Did they really learn how to differentiate or did they just learn a common vibe around it? When teachers see that sort of regurgitation, they call it memorization, despite the input being unique from what the student had seen in the past.

saberience 3y ago

Surely learning an algorithm is memorization? I still remember the physics equations from when I was at school (a long time ago) because it was drilled into us repeatedly. Stuff like v = u + at, or s=tu + 1/2 (at)^2, etc etc. I don't really recall how they were derived etc but I did memorize them. It definitely doesn't mean I'm good at physics or understand the deeper meaning of these equations. I also had to memorize bubble sort, merge, heap, quicksort etc when I was in university, but I don't think I could invent these sorts from first principles without looking it up. Memorization doesn't really equal understanding, it's just memorization.

sushibowl 3y ago
> Surely learning an algorithm is memorization? Not in this context, no. An LLM is never given any algorithm to memorise. It is given only input->output sets, and it "learns" what the algorithm is from those sets. We know that it does this because it is able to generalize that algorithm to inputs and outputs outside of its set of training examples. So we know that it doesn't only memorise which input connects to which output, and regurgitate that information. It has come to "understand" the formula that connects the input to the output, without ever being given that formula.
- quantum_state 3y ago
  It’s fitting a function to data and guessing the value of the function for a new input. It does not know what is under the hood of the function, etc., and see the implications when the function produces a wrong value … etc.
  - astrange 3y ago
    It can do that. Verifying an answer is just another algorithm it can learn. LLMs mostly can't do math but that, like most of their other flaws, is because of the tokenizer.
retrac 3y ago
> Surely learning an algorithm is memorization? Yes. In a sense, every algorithm can be reduced to memorization, where you precalculate all possible inputs to all possible outputs and store them in a giant lookup table.

rurban 3y ago

They might do "computation" and seem to be "intelligent", however they do lack logic and rigor, and cannot explain their reasoning. In contrast to old-school AI. Such "intelligent" monkeys might be able to write reports, lead governments or lead wars, but in engineering you need more skills than that. Which leads us to the lack of a proper definition of AGI. (https://en.wikipedia.org/wiki/Artificial_general_intelligence https://en.wikipedia.org/wiki/Artificial_general_intelligenc...)