6 ms·

I don't see how the latter follows from the former. Here's how I think about it: the fact that it can interpret the same words differently in different context

I don't see how the latter follows from the former. Here's how I think about it: the fact that it can interpret the same words differently in different contexts alone shows that even on a temperature of 0 (i.e., lowest randomness possible) there could be something that possibly resembles reasoning happening. It might be a mimicry of reasoning, but I don't think that having adjustable parameters on how random they are makes it any less of one. I also don't see how that idea would fit in with the o1 models, which explicitly have "reasoning" tokens. Now, I'm not terribly impressed with their performance relative to how much extra computation they need to do, but the fact they have chains-of-thought that humans could reasonably inspect and interpret, and that they chains of thought do literally take extra time and compute to run, certainly points at the process being something possibly analogous to reasoning. In this same vein, up until recently I personally very much in the camp of calling them "LLMs" and generally still do, but given how they really are being used now as general purpose sequence-to-sequence prediction models across all sorts of input and output types tends to push me more towards the "foundation models" terminology camp, since pigeonholing them into just language tasks doesn't seem accurate anymore. o1 was the turning point for me on this personally, since it is explicitly predicting and being optimized for correctness in the "reasoning tokens" (in scare quotes again since that's what openai calls it). All that said, I personally think that calling what they do reasoning, and meaning it in the exact same way as how humans reason, is anthropomorphizing the models in a way that's not really useful. They clearly operate in ways that are quite different from humans in many ways. Sometimes that might imitate human reasoning, other times it doesn't. But, the fact they have that randomness parameter seems to be to be totally unrelated to any of the above thoughts or merits about the models having reasoning abilities.

tananan 2y ago
The notion is AFAIS that a deterministic algorithm is obviously not reasoning, and a deterministic algorithm interspersed with dice rolls is obviously not reasoning either. Of course, some would beg to differ. It's quite common nowadays to believe that we are something like the latter.
- pishpash 2y ago
  Why is a deterministic algorithm not reasoning? Reasoning is very deterministic.
  - tananan 2y ago
    It's not about (in-)determinism really, it's about the algorithm part. An algorithm that does something can in principle be ran by someone who doesn't know what the algorithm does. You could have a kid calculate an integral by giving it a sequence of directions whose purpose it doesn't understand (e.g. cut out some cardboard that matches the shape, put it on one side of the scale, place enough unit cardboard pieces on the other side until they are even, then tell me how many pieces you put). Reasoning has more to do with how the problem came about. A person had to come against a certain problem, figure out a way in which they can solve it, then apply the (perhaps algorithmic) solution. The algorithmic part is only an artifact.
    - pishpash 2y ago
      I think you overlook how algorithms come about. How does GPT write novel code, which are algorithms?
      - tananan 2y ago
        Not sure I track. It would help to know where you're coming from. Given a long enough life-span, a lot of pencil and paper, and some dice, I could do the forward passes of GPT and "write novel code", without there having been any reasoning about the code I'm writing down - I wouldn't even need to know what the code is about.
        XenophileJKO 2y ago
        I mean in theory I could sit and calculate electrical and chemical potentials and interactions and figure out what your next comment will be.
ActorNightly 2y ago
>he fact that it can interpret the same words differently in different contexts alone shows that even on a temperature of 0 ( This is the problem with using loaded language like "reason" and "interpret". The model is not interpreting anything. All that is being done is a multdimentional map lookup with statistics. > also don't see how that idea would fit in with the o1 models, which explicitly have "reasoning" tokens. An LLM on top of an LLM (i.e using context to generate inputs to an LLM) is just a fancier LLM. To really understand all of this, all you need to do is look at how Transformer works, namely the attention block. There is no such thing as Query, Key, and Value in the sense of how they are implied to be used. The may as well be called A,B,C, as they are all learned in training, and can be freely interchanged in naming. All you do for inference is multiply the output vector by A,B,C to get 3 matrices, then multiply them together (technically with a scaling factor for 2 of them, but again, doesn't matter for which 2, and the scaling factor can be built into the matrix itself) And because you can unroll matrix multiplication into a 2 layer neural network, that means that any LLM in its current form today can be represented as a set of linear layers. And we know that a set of linear layers is simply a function. And every function has a finite range for a finite domain. And the inability to expand that range given a finite domain means its not reasoning. So we have to rely on hacks like temperature to make it appear like reasoning, when its really not even close.
- Eisenstein 2y ago
  > The model is not interpreting anything. All that is being done is a multdimentional map lookup with statistics. So what? Can you propose another method to make a computing device understand language? The method of the creation of the output does not stipulate anything about the nature of the thing creating it. If someone could map out a human brain and tell you how thoughts are made and added a 'all that is being done is' in front of it, does that make your thought creation trivial? > An LLM on top of an LLM (i.e using context to generate inputs to an LLM) is just a fancier LLM. This is called a tautology. You have not given any compelling reasons why an LLM cannot do anything, so calling something another LLM is not compelling either. > To really understand all of this, all you need to do is look at how Transformer works, namely the attention block. There is no such thing as Query, Key, and Value in the sense of how they are implied to be used. The may as well be called A,B,C, as they are all learned in training, and can be freely interchanged in naming. All you do for inference is multiply the output vector by A,B,C to get 3 matrices, then multiply them together (technically with a scaling factor for 2 of them, but again, doesn't matter for which 2, and the scaling factor can be built into the matrix itself) Here is how it works, so therefore it must meet some criteria I have imposed arbitrarily. > So we have to rely on hacks like temperature to make it appear like reasoning, when its really not even close. You still haven't produced any valid argument at all, for why one thing would be evidence of the other.

tananan 2y ago

The notion is AFAIS that a deterministic algorithm is obviously not reasoning, and a deterministic algorithm interspersed with dice rolls is obviously not reasoning either. Of course, some would beg to differ. It's quite common nowadays to believe that we are something like the latter.

pishpash 2y ago
Why is a deterministic algorithm not reasoning? Reasoning is very deterministic.
- tananan 2y ago
  It's not about (in-)determinism really, it's about the algorithm part. An algorithm that does something can in principle be ran by someone who doesn't know what the algorithm does. You could have a kid calculate an integral by giving it a sequence of directions whose purpose it doesn't understand (e.g. cut out some cardboard that matches the shape, put it on one side of the scale, place enough unit cardboard pieces on the other side until they are even, then tell me how many pieces you put). Reasoning has more to do with how the problem came about. A person had to come against a certain problem, figure out a way in which they can solve it, then apply the (perhaps algorithmic) solution. The algorithmic part is only an artifact.
  - pishpash 2y ago
    I think you overlook how algorithms come about. How does GPT write novel code, which are algorithms?
    - tananan 2y ago
      Not sure I track. It would help to know where you're coming from. Given a long enough life-span, a lot of pencil and paper, and some dice, I could do the forward passes of GPT and "write novel code", without there having been any reasoning about the code I'm writing down - I wouldn't even need to know what the code is about.
      - XenophileJKO 2y ago
        I mean in theory I could sit and calculate electrical and chemical potentials and interactions and figure out what your next comment will be.

ActorNightly 2y ago

>he fact that it can interpret the same words differently in different contexts alone shows that even on a temperature of 0 ( This is the problem with using loaded language like "reason" and "interpret". The model is not interpreting anything. All that is being done is a multdimentional map lookup with statistics. > also don't see how that idea would fit in with the o1 models, which explicitly have "reasoning" tokens. An LLM on top of an LLM (i.e using context to generate inputs to an LLM) is just a fancier LLM. To really understand all of this, all you need to do is look at how Transformer works, namely the attention block. There is no such thing as Query, Key, and Value in the sense of how they are implied to be used. The may as well be called A,B,C, as they are all learned in training, and can be freely interchanged in naming. All you do for inference is multiply the output vector by A,B,C to get 3 matrices, then multiply them together (technically with a scaling factor for 2 of them, but again, doesn't matter for which 2, and the scaling factor can be built into the matrix itself) And because you can unroll matrix multiplication into a 2 layer neural network, that means that any LLM in its current form today can be represented as a set of linear layers. And we know that a set of linear layers is simply a function. And every function has a finite range for a finite domain. And the inability to expand that range given a finite domain means its not reasoning. So we have to rely on hacks like temperature to make it appear like reasoning, when its really not even close.

Eisenstein 2y ago
> The model is not interpreting anything. All that is being done is a multdimentional map lookup with statistics. So what? Can you propose another method to make a computing device understand language? The method of the creation of the output does not stipulate anything about the nature of the thing creating it. If someone could map out a human brain and tell you how thoughts are made and added a 'all that is being done is' in front of it, does that make your thought creation trivial? > An LLM on top of an LLM (i.e using context to generate inputs to an LLM) is just a fancier LLM. This is called a tautology. You have not given any compelling reasons why an LLM cannot do anything, so calling something another LLM is not compelling either. > To really understand all of this, all you need to do is look at how Transformer works, namely the attention block. There is no such thing as Query, Key, and Value in the sense of how they are implied to be used. The may as well be called A,B,C, as they are all learned in training, and can be freely interchanged in naming. All you do for inference is multiply the output vector by A,B,C to get 3 matrices, then multiply them together (technically with a scaling factor for 2 of them, but again, doesn't matter for which 2, and the scaling factor can be built into the matrix itself) Here is how it works, so therefore it must meet some criteria I have imposed arbitrarily. > So we have to rely on hacks like temperature to make it appear like reasoning, when its really not even close. You still haven't produced any valid argument at all, for why one thing would be evidence of the other.