5 ms·

>That's amazing and all It earnestly is amazing. >Why do you think LLM's can't "perform this kind of translation by merely comparing tokens statistically tend

by knome 4y ago

>That's amazing and all It earnestly is amazing. >Why do you think LLM's can't "perform this kind of translation by merely comparing tokens statistically tending to be near one another". The implication I gather from those dismissing this technology as "merely comparing tokens statistically tending to be near one another" is that they think the whole of the LLM is nothing but a trick, a sleight of hand where it generates output token by token Markov style after consuming enough text, just using some percent chance of tokens following on after another. In my example, it generated the metaphor "sweeping the ice" as a replacement for "spinning my tires". Both are using in context to imply a level of busywork associated with the respective contexts. To generate such a transformation requires understanding this inner meaning and associating it. It requires a level of creativity to determine that "sweeping the ice" in curling is an appropriate replacement. You cannot rearrange "spinning my tires" into "sweeping the ice" without having derived and translated the underlying meaning. There is nothing to connect the two sets of tokens except the metaphoric intent. >Why should we assume that the well-known and understood design and functionality of LLMs is not enough to explain their behaviour? Obviously LLMs are capable of this as it is an LLM that is producing it. I am not denying that the inputs and outputs are tokens marked up with distance information, nor that the process of generating tokens is just a bunch of vector multiplication to generate neuron values from weights across the layers of the network. I am suggesting that with sufficient training, layers and parameters, that the neural net must be forced into a configuration that allows it to perform this meaning/intent based processing as gradient descent pushes and pulls its the network along the twelve thousand or so dimensions GPT-3 uses. Undoubtedly GPT-4 and soon 5 use even more. LLMs have repeatedly been found to have developed novel and unintended capabilities as their complexity has increased. Why do you think that LLMs shouldn't be capable of manipulating on the level of meanings and intents rather than just rearranging tokens?

YeGoblynQueenne 4y ago
>> You cannot rearrange "spinning my tires" into "sweeping the ice" without having derived and translated the underlying meaning. There is nothing to connect the two sets of tokens except the metaphoric intent. Here, you are again explaining what, but you are not explaining why. You are saying what you think, but I still don't know why you think that. Why is it that you >> cannot rearrange "spinning my tires" into "sweeping the ice" << etc? Why do you say this? For example, I can see that both are verb phrases with a similar structure. Language models excel in reproducing structure and we know they can translate between structures, also, as they do when translating between languages. So why can't they do it in this case without "underlying meaning"? >> Why do you think that LLMs shouldn't be capable of manipulating on the level of meanings and intents rather than just rearranging tokens? I didn't say I think that. I asked you: >> Why do you think LLM's can't "perform this kind of translation by merely comparing tokens statistically tending to be near one another". And I still don't know why you say that. For example, what you say here doesn't help me understand: >> I am suggesting that with sufficient training, layers and parameters, that the neural net must be forced into a configuration that allows it to perform this meaning/intent based processing as gradient descent pushes and pulls its the network along the twelve thousand or so dimensions GPT-3 uses. That's a description of the what, not the how. But it's not a description of any "whats" we know anyone coded into a language model in the first place. It's a description of a mechanism that is claimed to have appeared spontaneously. That is a very big claim and I don't see why I have to accept it just because someone said that's what's going on. I want to know how things work. You're not explaining how things work. You're explaining how you think they might work. But I don't even know why you think they work that way.
- knome 4y ago
  >You're not explaining how things work It maps a vector of eight thousand tokens to some number, likely thousands, of "neurons", multiplying each by a weight associated by each connection. All of these are then combined to form the next layer of values by multiplying and combining all of the values to all of the neurons in the next layer. Since all neurons connect to each other at each layer, many complex interactions can be created between them. These multiplications will go through about a hundred layers until being mapped back to a single token. That token will be inserted at the end of the prompt and it will do this again and again until some stop token is reached, or the eight thousand token prompt is full. --- As for what I think is happening, I think that information about meaning and methods for extracting it are being encoded across those billions of weights and thousand and thousands of neurons such that these multiplications will transform an initial vector into an abstract numeric form that across the whole span of any set of neurons might represent various abstract concepts and their interrelationships, such that they can then be used to select the next token in the series. --- As for why, I expect this happens because abstract representation is how the model compresses data as gradient descent drives it twelve thousand different directions just a bit at a time as it trains on each document, meandering through the abstract space of potential neural net configurations to arrive at a local maximum that is sufficiently powerful enough to manipulate human language.
  - YeGoblynQueenne 4y ago
    First of all, thanks for your patience. I'm still not convinced, but I appreciate your effort to explain how you think. Now. For brevity, I'll refer to the three sections (separated by "---" in your comment) as A, B, and C. Section A is a description of how a neural net trains a language model. I have some quibbles ("neurons"...) but it's talking about known and observable facts. So I will call it a "factual" description. I note you did not include the word "meaning" in that factual description and that's because you restricted yourself to facts. You could not give a factual description with the word "meaning" in it (in the sense of "the meaning of..."). Section B and section C are again speculation and interpretation that are not based on known or observed facts, or even observable facts. So I still do not know why you think all this is going on inside a language model. To clarify, in sections B and C you are at the very least putting an interpretation on the facts in section A. But I have no idea why you even need an interpretation, and why you choose that particular interpretation. What do language models have to do with meaning, and why do we absolutely need to explain their behaviour by talking about meaning? Where is this information in your comment? I cannot see it.

by knome 4y ago

YeGoblynQueenne 4y ago

>> You cannot rearrange "spinning my tires" into "sweeping the ice" without having derived and translated the underlying meaning. There is nothing to connect the two sets of tokens except the metaphoric intent. Here, you are again explaining what, but you are not explaining why. You are saying what you think, but I still don't know why you think that. Why is it that you >> cannot rearrange "spinning my tires" into "sweeping the ice" << etc? Why do you say this? For example, I can see that both are verb phrases with a similar structure. Language models excel in reproducing structure and we know they can translate between structures, also, as they do when translating between languages. So why can't they do it in this case without "underlying meaning"? >> Why do you think that LLMs shouldn't be capable of manipulating on the level of meanings and intents rather than just rearranging tokens? I didn't say I think that. I asked you: >> Why do you think LLM's can't "perform this kind of translation by merely comparing tokens statistically tending to be near one another". And I still don't know why you say that. For example, what you say here doesn't help me understand: >> I am suggesting that with sufficient training, layers and parameters, that the neural net must be forced into a configuration that allows it to perform this meaning/intent based processing as gradient descent pushes and pulls its the network along the twelve thousand or so dimensions GPT-3 uses. That's a description of the what, not the how. But it's not a description of any "whats" we know anyone coded into a language model in the first place. It's a description of a mechanism that is claimed to have appeared spontaneously. That is a very big claim and I don't see why I have to accept it just because someone said that's what's going on. I want to know how things work. You're not explaining how things work. You're explaining how you think they might work. But I don't even know why you think they work that way.

knome 4y ago
>You're not explaining how things work It maps a vector of eight thousand tokens to some number, likely thousands, of "neurons", multiplying each by a weight associated by each connection. All of these are then combined to form the next layer of values by multiplying and combining all of the values to all of the neurons in the next layer. Since all neurons connect to each other at each layer, many complex interactions can be created between them. These multiplications will go through about a hundred layers until being mapped back to a single token. That token will be inserted at the end of the prompt and it will do this again and again until some stop token is reached, or the eight thousand token prompt is full. --- As for what I think is happening, I think that information about meaning and methods for extracting it are being encoded across those billions of weights and thousand and thousands of neurons such that these multiplications will transform an initial vector into an abstract numeric form that across the whole span of any set of neurons might represent various abstract concepts and their interrelationships, such that they can then be used to select the next token in the series. --- As for why, I expect this happens because abstract representation is how the model compresses data as gradient descent drives it twelve thousand different directions just a bit at a time as it trains on each document, meandering through the abstract space of potential neural net configurations to arrive at a local maximum that is sufficiently powerful enough to manipulate human language.
- YeGoblynQueenne 4y ago
  First of all, thanks for your patience. I'm still not convinced, but I appreciate your effort to explain how you think. Now. For brevity, I'll refer to the three sections (separated by "---" in your comment) as A, B, and C. Section A is a description of how a neural net trains a language model. I have some quibbles ("neurons"...) but it's talking about known and observable facts. So I will call it a "factual" description. I note you did not include the word "meaning" in that factual description and that's because you restricted yourself to facts. You could not give a factual description with the word "meaning" in it (in the sense of "the meaning of..."). Section B and section C are again speculation and interpretation that are not based on known or observed facts, or even observable facts. So I still do not know why you think all this is going on inside a language model. To clarify, in sections B and C you are at the very least putting an interpretation on the facts in section A. But I have no idea why you even need an interpretation, and why you choose that particular interpretation. What do language models have to do with meaning, and why do we absolutely need to explain their behaviour by talking about meaning? Where is this information in your comment? I cannot see it.