8 ms·

That's amazing and all, but it's also speculation, yes? It's how you imagine, or guess, that things are happening. It's not like you, or anyone else, knows that

That's amazing and all, but it's also speculation, yes? It's how you imagine, or guess, that things are happening. It's not like you, or anyone else, knows that this kind of thing is really happening. Moreover, there isn't any clear reason why you need to speculate all that, in order to explain the observed behaviour of Large Language Models (LLMs). You say you don't think LLMs "can perform this kind of translation by merely comparing tokens statistically tending to be near one another". And yet, that's what we know that LLMs do, because that's what they are made to do [edit: they do what you describe after the "by merely"]. Why do we need to imagine some other, so far unobserved, process at play? What is the motivation? What is the justification? Why do you think LLM's can't "perform this kind of translation by merely comparing tokens statistically tending to be near one another". It's clear what you are saying, but, why? Why are you saying that? Why should we assume that the well-known and understood design and functionality of LLMs is not enough to explain their behaviour? Is it because we haven't observed such behaviour before and some people are surprised by it? Well, we didn't have as large language models before, so obviously we didn't have a chance to observe their behaviour first-hand, but we also didn't have cars that could go 100 miles an hour, until we did, and making a faster car is no reason to be surprised, or wonder "how does it do it if it's just a car?".

knome 4y ago
>That's amazing and all It earnestly is amazing. >Why do you think LLM's can't "perform this kind of translation by merely comparing tokens statistically tending to be near one another". The implication I gather from those dismissing this technology as "merely comparing tokens statistically tending to be near one another" is that they think the whole of the LLM is nothing but a trick, a sleight of hand where it generates output token by token Markov style after consuming enough text, just using some percent chance of tokens following on after another. In my example, it generated the metaphor "sweeping the ice" as a replacement for "spinning my tires". Both are using in context to imply a level of busywork associated with the respective contexts. To generate such a transformation requires understanding this inner meaning and associating it. It requires a level of creativity to determine that "sweeping the ice" in curling is an appropriate replacement. You cannot rearrange "spinning my tires" into "sweeping the ice" without having derived and translated the underlying meaning. There is nothing to connect the two sets of tokens except the metaphoric intent. >Why should we assume that the well-known and understood design and functionality of LLMs is not enough to explain their behaviour? Obviously LLMs are capable of this as it is an LLM that is producing it. I am not denying that the inputs and outputs are tokens marked up with distance information, nor that the process of generating tokens is just a bunch of vector multiplication to generate neuron values from weights across the layers of the network. I am suggesting that with sufficient training, layers and parameters, that the neural net must be forced into a configuration that allows it to perform this meaning/intent based processing as gradient descent pushes and pulls its the network along the twelve thousand or so dimensions GPT-3 uses. Undoubtedly GPT-4 and soon 5 use even more. LLMs have repeatedly been found to have developed novel and unintended capabilities as their complexity has increased. Why do you think that LLMs shouldn't be capable of manipulating on the level of meanings and intents rather than just rearranging tokens?
- YeGoblynQueenne 4y ago
  >> You cannot rearrange "spinning my tires" into "sweeping the ice" without having derived and translated the underlying meaning. There is nothing to connect the two sets of tokens except the metaphoric intent. Here, you are again explaining what, but you are not explaining why. You are saying what you think, but I still don't know why you think that. Why is it that you >> cannot rearrange "spinning my tires" into "sweeping the ice" << etc? Why do you say this? For example, I can see that both are verb phrases with a similar structure. Language models excel in reproducing structure and we know they can translate between structures, also, as they do when translating between languages. So why can't they do it in this case without "underlying meaning"? >> Why do you think that LLMs shouldn't be capable of manipulating on the level of meanings and intents rather than just rearranging tokens? I didn't say I think that. I asked you: >> Why do you think LLM's can't "perform this kind of translation by merely comparing tokens statistically tending to be near one another". And I still don't know why you say that. For example, what you say here doesn't help me understand: >> I am suggesting that with sufficient training, layers and parameters, that the neural net must be forced into a configuration that allows it to perform this meaning/intent based processing as gradient descent pushes and pulls its the network along the twelve thousand or so dimensions GPT-3 uses. That's a description of the what, not the how. But it's not a description of any "whats" we know anyone coded into a language model in the first place. It's a description of a mechanism that is claimed to have appeared spontaneously. That is a very big claim and I don't see why I have to accept it just because someone said that's what's going on. I want to know how things work. You're not explaining how things work. You're explaining how you think they might work. But I don't even know why you think they work that way.
  - knome 4y ago
    >You're not explaining how things work It maps a vector of eight thousand tokens to some number, likely thousands, of "neurons", multiplying each by a weight associated by each connection. All of these are then combined to form the next layer of values by multiplying and combining all of the values to all of the neurons in the next layer. Since all neurons connect to each other at each layer, many complex interactions can be created between them. These multiplications will go through about a hundred layers until being mapped back to a single token. That token will be inserted at the end of the prompt and it will do this again and again until some stop token is reached, or the eight thousand token prompt is full. --- As for what I think is happening, I think that information about meaning and methods for extracting it are being encoded across those billions of weights and thousand and thousands of neurons such that these multiplications will transform an initial vector into an abstract numeric form that across the whole span of any set of neurons might represent various abstract concepts and their interrelationships, such that they can then be used to select the next token in the series. --- As for why, I expect this happens because abstract representation is how the model compresses data as gradient descent drives it twelve thousand different directions just a bit at a time as it trains on each document, meandering through the abstract space of potential neural net configurations to arrive at a local maximum that is sufficiently powerful enough to manipulate human language.
Workaccount2 4y ago
> And yet, that's what we know that LLMs do, because that's what they are made to do [edit: they do what you describe after the "by merely"]. Why do we need to imagine some other, so far unobserved, process at play? What is the motivation? What is the justification? We also know how neurons work, interconnect, and communicate. There is very clearly emergent behavior going on, and while neuroscientists have yielded this point for decades about the brain (any brain, even tadpoles), the AI guys still are stubbornly holding on to "we know exactly how the system works, because we know how the individual bits work".
- YeGoblynQueenne 4y ago
  >> There is very clearly emergent behavior going on, and while neuroscientists have yielded this point for decades about the brain (any brain, even tadpoles), the AI guys still are stubbornly holding on to "we know exactly how the system works, because we know how the individual bits work". That's because the "AI guys" are working with computers, and not with tadpoles, or even neurons. And certainly not brains.
  - Workaccount2 4y ago
    There is no difference unless you believe in some kind of supernatural influence in biological organisms. The "brains are not computers" point has been moot for decades since it would require brains to be able to perform actions that cannot be mathematically described i.e. break the laws of nature. Which is to say "If a brain can do it, a Turing machine can simulate it" which pretty much everyone in the field agrees with. You can stand on a dualistic platform and hold that the brain has supernatural abilities, but in mainstream discussions of the idea has been dead for a long time. You can argue that computers aren't close to simulating a brain, sure, but that doesn't really get you anywhere, since the difference just becomes a matter of practical scale rather than something that escapes theory. Or even just a factor of efficiency, a synthetic consciousness might be able to run on current hardware if the right program is found. The brain might be grossly inefficient for consciousness, we don't know.
    - 4y ago

knome 4y ago

>That's amazing and all It earnestly is amazing. >Why do you think LLM's can't "perform this kind of translation by merely comparing tokens statistically tending to be near one another". The implication I gather from those dismissing this technology as "merely comparing tokens statistically tending to be near one another" is that they think the whole of the LLM is nothing but a trick, a sleight of hand where it generates output token by token Markov style after consuming enough text, just using some percent chance of tokens following on after another. In my example, it generated the metaphor "sweeping the ice" as a replacement for "spinning my tires". Both are using in context to imply a level of busywork associated with the respective contexts. To generate such a transformation requires understanding this inner meaning and associating it. It requires a level of creativity to determine that "sweeping the ice" in curling is an appropriate replacement. You cannot rearrange "spinning my tires" into "sweeping the ice" without having derived and translated the underlying meaning. There is nothing to connect the two sets of tokens except the metaphoric intent. >Why should we assume that the well-known and understood design and functionality of LLMs is not enough to explain their behaviour? Obviously LLMs are capable of this as it is an LLM that is producing it. I am not denying that the inputs and outputs are tokens marked up with distance information, nor that the process of generating tokens is just a bunch of vector multiplication to generate neuron values from weights across the layers of the network. I am suggesting that with sufficient training, layers and parameters, that the neural net must be forced into a configuration that allows it to perform this meaning/intent based processing as gradient descent pushes and pulls its the network along the twelve thousand or so dimensions GPT-3 uses. Undoubtedly GPT-4 and soon 5 use even more. LLMs have repeatedly been found to have developed novel and unintended capabilities as their complexity has increased. Why do you think that LLMs shouldn't be capable of manipulating on the level of meanings and intents rather than just rearranging tokens?

YeGoblynQueenne 4y ago
>> You cannot rearrange "spinning my tires" into "sweeping the ice" without having derived and translated the underlying meaning. There is nothing to connect the two sets of tokens except the metaphoric intent. Here, you are again explaining what, but you are not explaining why. You are saying what you think, but I still don't know why you think that. Why is it that you >> cannot rearrange "spinning my tires" into "sweeping the ice" << etc? Why do you say this? For example, I can see that both are verb phrases with a similar structure. Language models excel in reproducing structure and we know they can translate between structures, also, as they do when translating between languages. So why can't they do it in this case without "underlying meaning"? >> Why do you think that LLMs shouldn't be capable of manipulating on the level of meanings and intents rather than just rearranging tokens? I didn't say I think that. I asked you: >> Why do you think LLM's can't "perform this kind of translation by merely comparing tokens statistically tending to be near one another". And I still don't know why you say that. For example, what you say here doesn't help me understand: >> I am suggesting that with sufficient training, layers and parameters, that the neural net must be forced into a configuration that allows it to perform this meaning/intent based processing as gradient descent pushes and pulls its the network along the twelve thousand or so dimensions GPT-3 uses. That's a description of the what, not the how. But it's not a description of any "whats" we know anyone coded into a language model in the first place. It's a description of a mechanism that is claimed to have appeared spontaneously. That is a very big claim and I don't see why I have to accept it just because someone said that's what's going on. I want to know how things work. You're not explaining how things work. You're explaining how you think they might work. But I don't even know why you think they work that way.
- knome 4y ago
  >You're not explaining how things work It maps a vector of eight thousand tokens to some number, likely thousands, of "neurons", multiplying each by a weight associated by each connection. All of these are then combined to form the next layer of values by multiplying and combining all of the values to all of the neurons in the next layer. Since all neurons connect to each other at each layer, many complex interactions can be created between them. These multiplications will go through about a hundred layers until being mapped back to a single token. That token will be inserted at the end of the prompt and it will do this again and again until some stop token is reached, or the eight thousand token prompt is full. --- As for what I think is happening, I think that information about meaning and methods for extracting it are being encoded across those billions of weights and thousand and thousands of neurons such that these multiplications will transform an initial vector into an abstract numeric form that across the whole span of any set of neurons might represent various abstract concepts and their interrelationships, such that they can then be used to select the next token in the series. --- As for why, I expect this happens because abstract representation is how the model compresses data as gradient descent drives it twelve thousand different directions just a bit at a time as it trains on each document, meandering through the abstract space of potential neural net configurations to arrive at a local maximum that is sufficiently powerful enough to manipulate human language.

Workaccount2 4y ago

> And yet, that's what we know that LLMs do, because that's what they are made to do [edit: they do what you describe after the "by merely"]. Why do we need to imagine some other, so far unobserved, process at play? What is the motivation? What is the justification? We also know how neurons work, interconnect, and communicate. There is very clearly emergent behavior going on, and while neuroscientists have yielded this point for decades about the brain (any brain, even tadpoles), the AI guys still are stubbornly holding on to "we know exactly how the system works, because we know how the individual bits work".

YeGoblynQueenne 4y ago
>> There is very clearly emergent behavior going on, and while neuroscientists have yielded this point for decades about the brain (any brain, even tadpoles), the AI guys still are stubbornly holding on to "we know exactly how the system works, because we know how the individual bits work". That's because the "AI guys" are working with computers, and not with tadpoles, or even neurons. And certainly not brains.
- Workaccount2 4y ago
  There is no difference unless you believe in some kind of supernatural influence in biological organisms. The "brains are not computers" point has been moot for decades since it would require brains to be able to perform actions that cannot be mathematically described i.e. break the laws of nature. Which is to say "If a brain can do it, a Turing machine can simulate it" which pretty much everyone in the field agrees with. You can stand on a dualistic platform and hold that the brain has supernatural abilities, but in mainstream discussions of the idea has been dead for a long time. You can argue that computers aren't close to simulating a brain, sure, but that doesn't really get you anywhere, since the difference just becomes a matter of practical scale rather than something that escapes theory. Or even just a factor of efficiency, a synthetic consciousness might be able to run on current hardware if the right program is found. The brain might be grossly inefficient for consciousness, we don't know.
  - 4y ago