9 ms·

The problem with using a regex as you describe is that GPT isn't working at the level of words, its working at a level of meaning and intent. You can have it tr

by knome 4y ago

The problem with using a regex as you describe is that GPT isn't working at the level of words, its working at a level of meaning and intent. You can have it translate rather meaningfully between metaphors that exist in completely different contexts. Somewhere in the 90+ levels of interconnected neurons, the literal meanings are abstracted and re-encoded. A regex isn't going to accomplish that. edit: you might be able to match this little tiny GPT, but not one of the billions-of-parameters varieties.

dvt 4y ago
> its working at a level of meaning and intent LLMs work on tokens, which, for the most part, are words (longer words are generally split up into multiple tokens). Using concepts like "meaning" and "intent" muddies the waters unnecessarily.
- Tarq0n 4y ago
  Those are the in / outputs, but the model works on the embeddings which is a high dimensional representation of meaning.
  - nerdponx 4y ago
    Meaning only insofar as can be observed from the context of the word. I don't think it's reasonable to call that "meaning" in and of itself.
    - SanderNL 4y ago
      It’s a good philosophical exercise to imagine what’s left of the meaning of a word if you strip all context from it. What is a word if it doesn’t reference anything, if it has no context? Isn’t this like a pointer without referencing an address? I’ll give you that words can reference tokens of other modalities like hearing and such, but that is still the same problem.
      - nerdponx 4y ago
        I'm thinking in the other direction. Of course context reveals (or as in your philosophical exercise, defines) some of the meaning of the word. But how much of the meaning is context-dependent depends on the word and the context itself where it's being used. For example, consider a word that means two different things in two different contexts that would otherwise be very far apart in whatever latent "meaning space" they might exist in. A naive model might end up averaging across both contexts, resulting in a representation of that word that represents neither meaning of the word, being located vaguely in between both regions of space. Alternatively, this could pull "groups" of words that otherwise are not related into adjacent regions of space, creating the appearance of semantic similarity when there shouldn't be any. In this example, the physical surrounding word context is actually misleading, and addition information is needed in order to make the learned embeddings more realistic. In any case, it should be obvious to people trained in statistical practice that word models do not encode "meaning" per se, but rather an estimate of a model of meaning, which is not a complete model, but it does a pretty good job most of the time. We as the experts should be careful to stick to language along these lines, so as not to promulgate fanciful ideas about AI. I am not even going to bother with the "it's not really intelligent!" angle here. For all I care, the model is fully sentient. But the fact is that the token embeddings do not encode "meaning", it encodes "an estimate of model of meaning". They are our best effort attempt at encoding meaning parsimoniously, but that is not the same thing as meaning itself. Map is not the territory, all models are wrong, etc.
        SanderNL 4y ago
        > Map is not the territory I think this one will get fuzzy real fast when you talk about cognition itself. Otherwise, thanks for the thoughtful response. Food for thought.
  - dvt 4y ago
    > high dimensional representation of meaning This is word salad. I don't even know what you mean by "meaning," but an embedding is simply an array filled with a bunch of floating point numbers[1] and, importantly, these embeddings are trained, so "doctor" will have an embedding similar to "hospital" because it's seen in close proximity in the training data. [1] https://www.tensorflow.org/text/guide/word_embeddings https://www.tensorflow.org/text/guide/word_embeddings
    - Dylan16807 4y ago
      How is it word salad? The length of the number array is referred to as dimensions for math reasons. That's half the statement right there. > importantly, these embeddings are trained, so "doctor" will have an embedding similar to "hospital" because it's seen in close proximity in the training data It is trained to put together words with similar... meaning. So the embedding is a representation of... meaning.
      - dvt 4y ago
        > It is trained to put together words with similar... meaning. So the embedding is a representation of... meaning. This might be waxing philosophical, but my position is that the meaning of a word is intrinsic, not extrinsic. The meaning of the word "ball" has nothing to do with the fact that it's next to the word "net" on a page (which is what LLMs do). But rather, "ball" is the round rubbery thing I kick around in my back yard.
        airstrike 4y ago
        Except "ball" can also mean a party, so you do need "net" or other words for context
        vidarh 4y ago
        More fundamentally, until someone has told you that "ball" can mean "a round thing" or pointed to a ball and said that "ball" can mean that, "ball" does not have that meaning to you. The meaning is purely extrinsic. There's nothing about a ball that makes it named that, as shown by the fact that a ball is called other things in other languages.
  - beepbooptheory 4y ago
    I feel like if meaning itself was simply contained or even encoded in the words we write, the world would be a very different place to us. How could we ever find ourselves, as we do, arguing about the meaning of a text? Even then driest prose can sometimes lead to different interpretations. Is there "meaning" somehow inside the novel on my desk? Can you extract it and manipulate it? If I had no body, environment, temporality, what even could the book mean to me? Even more, if nobody could teach me the meanings of the words themselves in the book, what could it ever be to me but a collection of certain patterns?
    - gabereiser 4y ago
      but it is exactly human-kinds ability to put meaning to words that make us unique. That give us language and communication, artistry and music. Words mean nothing without meaning. It's not encoded, it's inferred. There's nouns, adjectives, verbs, etc and some of them are tied to the same word. That's very important. The chain of words derives it's meaning in context so having a system that operates on meaning is, dare I say, meaningful. It's the whole point.
    - knome 4y ago
      >if meaning itself was simply contained or even encoded in the words we write I feel you might enjoy Chapter VI of Douglas Hofstadter's "Godel, Escher, Back", being "The Location of Meaning".
- knome 4y ago
  We read words. To translate some words into other words of different literal meanings while maintaining non-literal metaphorical meaning implies the capacity to "understand" the words at that level. That it represents data as tokens is irrelevant to the kinds of processing it obviously performs. Pretending GPT is just using a percent chance that some token should show up next is absurd. It's almost a hundred layers of neural pathways.
  - dvt 4y ago
    > Pretending GPT is just using a percent chance that some token should show up next is absurd. It's almost a hundred layers of neural pathways. That is literally what it's doing though. "Pretending a CPU is just adding 0s and 1s is absurd. It's made out of billions and billions of transistors."
    - knome 4y ago
      A multi-layer neural net is not limited to vague statistical combinations. You can represent XOR in a neural network. A NAND. If you can represent NAND. You can represent arbitrary logic. Across billions of weights and across nearly a hundred layers of possible decision points. Meaning can be spread across the neurons in ways that allow each to be part of many many different representations. This is only in GPT-3. GPT-4 and soon 5 no doubt exceed these considerably. https://towardsdatascience.com/how-neural-networks-solve-the-xor-problem-59763136bdd7 https://towardsdatascience.com/how-neural-networks-solve-the...
      - dvt 4y ago
        > A multi-layer neural net is not limited to vague statistical combinations. It, absolutely, unequivocally, 100% is only limited to statistical inference. That's how it works. The problem of linear separability (which is like 60 years old, by the way) has nothing to do with logic, but it's actually a byproduct of (a) the fact that neural networks use linear activation functions, and (b) XOR is not separable by a single hyperplane. Hence, you need multiple layers. No idea how you get to "arbitrary logic" from this.
- famouswaffles 4y ago
  It's obviously what's happening lol. It's why: 1. GPT-3 can speak native-level french despite the language only being 1.8% of it's training corpus. 2. GPT-3 can follow instruction in other languages just fine despite the instruct tuning data being almost entirely in English. 3. LLMs trained on code demonstrably reason better. It's fairly obvious pre-trained LLMs aren't really operating at just the word/token level.
  - Donckele 4y ago
    1) 1.8% is still massive 2) 1.8% is still massive 3) what?
    - Sharlin 4y ago
      It’s also fluent (though not at a native level) in, say, Finnish, and can write reasonable poems and whatever, even though Finnish text probably comprises something like 0.01% of the corpus.
      - deleted 4y ago
        [deleted]
      - vidarh 4y ago
        Norwegian as well. Though given Norwegian is also a Germanic language like English, Finnish is a lot more impressive. I managed to get it trip up on one single gendered article in Norwegian, while asking about Sweden, in a way very similar to how a human switching between Norwegian and Swedish might (it would be valid Swedish, and was followed by a Swedish word that's also valid in Norwegian but with another gender), but for the most part it writes Norwegian better than a lot of native speakers I've known would write (but not better than how they'd speak). EDIT: I got it to "translate" a test sentence from Norwegian to "Riksmål", an old-fashioned artificial Norwegian dialect that seeks to stay closer to very conservative old Norwegian (basically a lot closer to Danish) and it managed that just fine too, and there's very little material that you'd be able to tell explicitly is Riksmål as most of the time it'd just be described as Norwegian unless the writer is trying to make a point. Since that worked, I tried Nynorsk ("New Norwegian") which is used by something like 20% of the population, and it did fine. So I tried a local dialect from Eastern Norway, that is predominantly spoken and only occasionally written and it got even that pretty much how I'd have said it myself, after a lecture on how it varies greatly within the region (which is true) The differences there are subtle, and frankly a lot of native speakers would get one or more of these wrong. EDIT2: Last try at stumping it with Norwegian: "AKP-dialect". AKP was a Maoist political party of a few hundred members whose members were largely intellectuals. Many of them adopted a "faux workers dialect". This had no formal name, so it takes some understanding of Norwegian 1970's and 80's politics to even guess what it is. It understood what I meant, but didn't understood what spesifikk language-features I was referring to. Then again, if I asked random Norwegians about that most would need an explanation too. When I mentioned a couple of people who wrote like that it produced a plausible version (but clearly more inspired by the writing style of one of them - Jon Michelet)
- mistercow 4y ago
  I propose a ban on all arguments involving LLMs that boil down to “your usage of this word doesn’t match my personal connotations for that word”. It’s fine to ask someone to be clarify their definitions, but every other discussion of these models becomes a pointless debate about semantics. See, for example, below.
- woadwarrior01 4y ago
  > LLMs work on tokens, which, for the most part, are words (longer words are generally split up into multiple tokens). Although this is mostly true, there are exceptions to the rule. Look at the ByT5 family[1] of models for instance. These models operate on byte level representations of text and are inherently token free. I read a recent paper[2] that found that replacing token based text encoders in text to image models with these byte level models improves the spelling abilities of the txt2img models (which are otherwise notoriously bad at spelling in all human languages). [1]: https://arxiv.org/abs/2105.13626 https://arxiv.org/abs/2105.13626 [2]: https://arxiv.org/abs/2212.10562 https://arxiv.org/abs/2212.10562
alexahn 4y ago
Meaning and intent can only be applied if structure can be captured. I'm not proposing building some god-machine that has a library of all possible regular expressions. I'm merely saying that it could be helpful to nudge something like a neural network towards looking for these types of structures, such that they can be a basis for probabilistic encoding. It may be possible that the meaning that gets abstracted thereafter might actually be more concise and generalized.
- famouswaffles 4y ago
  Part of the breakthrough with LLMs has precisely been about encoding nothing at all about language and letting the transformer figure it out from scratch. We don't know what is necessary for learning language.
bordercases 4y ago
Has this been rigorously considered or is it just vibes? Seems preemptive to presume that unveiling more structure wouldn't give understanding meaning and intent in AI systems a leg up.
- famouswaffles 4y ago
  It's obviously what's happening. It's why: 1. GPT-3 can speak native-level french despite the language only being 1.8% of it's training corpus. 2. GPT-3 can follow instruction in other languages just fine despite the instruct tuning data being almost entirely in English. 3. LLMs trained on code demonstrably reason better. It's fairly obvious pre-trained LLMs aren't really operating at just the word/token level. Part of the breakthrough with LLMs has precisely been about encoding nothing at all about language and letting the transformer figure it out from scratch. We don't know what is necessary for learning language. Whatever structure we would add at our current level of understanding would far more likely hamper progress than anything. It would also be extremely surface level which LLMs go far beyond.

by knome 4y ago

dvt 4y ago

> its working at a level of meaning and intent LLMs work on tokens, which, for the most part, are words (longer words are generally split up into multiple tokens). Using concepts like "meaning" and "intent" muddies the waters unnecessarily.

Tarq0n 4y ago
Those are the in / outputs, but the model works on the embeddings which is a high dimensional representation of meaning.
- nerdponx 4y ago
  Meaning only insofar as can be observed from the context of the word. I don't think it's reasonable to call that "meaning" in and of itself.
  - SanderNL 4y ago
    It’s a good philosophical exercise to imagine what’s left of the meaning of a word if you strip all context from it. What is a word if it doesn’t reference anything, if it has no context? Isn’t this like a pointer without referencing an address? I’ll give you that words can reference tokens of other modalities like hearing and such, but that is still the same problem.
    - nerdponx 4y ago
      I'm thinking in the other direction. Of course context reveals (or as in your philosophical exercise, defines) some of the meaning of the word. But how much of the meaning is context-dependent depends on the word and the context itself where it's being used. For example, consider a word that means two different things in two different contexts that would otherwise be very far apart in whatever latent "meaning space" they might exist in. A naive model might end up averaging across both contexts, resulting in a representation of that word that represents neither meaning of the word, being located vaguely in between both regions of space. Alternatively, this could pull "groups" of words that otherwise are not related into adjacent regions of space, creating the appearance of semantic similarity when there shouldn't be any. In this example, the physical surrounding word context is actually misleading, and addition information is needed in order to make the learned embeddings more realistic. In any case, it should be obvious to people trained in statistical practice that word models do not encode "meaning" per se, but rather an estimate of a model of meaning, which is not a complete model, but it does a pretty good job most of the time. We as the experts should be careful to stick to language along these lines, so as not to promulgate fanciful ideas about AI. I am not even going to bother with the "it's not really intelligent!" angle here. For all I care, the model is fully sentient. But the fact is that the token embeddings do not encode "meaning", it encodes "an estimate of model of meaning". They are our best effort attempt at encoding meaning parsimoniously, but that is not the same thing as meaning itself. Map is not the territory, all models are wrong, etc.
      - SanderNL 4y ago
        > Map is not the territory I think this one will get fuzzy real fast when you talk about cognition itself. Otherwise, thanks for the thoughtful response. Food for thought.
- dvt 4y ago
  > high dimensional representation of meaning This is word salad. I don't even know what you mean by "meaning," but an embedding is simply an array filled with a bunch of floating point numbers[1] and, importantly, these embeddings are trained, so "doctor" will have an embedding similar to "hospital" because it's seen in close proximity in the training data. [1] https://www.tensorflow.org/text/guide/word_embeddings https://www.tensorflow.org/text/guide/word_embeddings
  - Dylan16807 4y ago
    How is it word salad? The length of the number array is referred to as dimensions for math reasons. That's half the statement right there. > importantly, these embeddings are trained, so "doctor" will have an embedding similar to "hospital" because it's seen in close proximity in the training data It is trained to put together words with similar... meaning. So the embedding is a representation of... meaning.
    - dvt 4y ago
      > It is trained to put together words with similar... meaning. So the embedding is a representation of... meaning. This might be waxing philosophical, but my position is that the meaning of a word is intrinsic, not extrinsic. The meaning of the word "ball" has nothing to do with the fact that it's next to the word "net" on a page (which is what LLMs do). But rather, "ball" is the round rubbery thing I kick around in my back yard.
      - airstrike 4y ago
        Except "ball" can also mean a party, so you do need "net" or other words for context
        vidarh 4y ago
        More fundamentally, until someone has told you that "ball" can mean "a round thing" or pointed to a ball and said that "ball" can mean that, "ball" does not have that meaning to you. The meaning is purely extrinsic. There's nothing about a ball that makes it named that, as shown by the fact that a ball is called other things in other languages.
- beepbooptheory 4y ago
  I feel like if meaning itself was simply contained or even encoded in the words we write, the world would be a very different place to us. How could we ever find ourselves, as we do, arguing about the meaning of a text? Even then driest prose can sometimes lead to different interpretations. Is there "meaning" somehow inside the novel on my desk? Can you extract it and manipulate it? If I had no body, environment, temporality, what even could the book mean to me? Even more, if nobody could teach me the meanings of the words themselves in the book, what could it ever be to me but a collection of certain patterns?
  - gabereiser 4y ago
    but it is exactly human-kinds ability to put meaning to words that make us unique. That give us language and communication, artistry and music. Words mean nothing without meaning. It's not encoded, it's inferred. There's nouns, adjectives, verbs, etc and some of them are tied to the same word. That's very important. The chain of words derives it's meaning in context so having a system that operates on meaning is, dare I say, meaningful. It's the whole point.
  - knome 4y ago
    >if meaning itself was simply contained or even encoded in the words we write I feel you might enjoy Chapter VI of Douglas Hofstadter's "Godel, Escher, Back", being "The Location of Meaning".
knome 4y ago
We read words. To translate some words into other words of different literal meanings while maintaining non-literal metaphorical meaning implies the capacity to "understand" the words at that level. That it represents data as tokens is irrelevant to the kinds of processing it obviously performs. Pretending GPT is just using a percent chance that some token should show up next is absurd. It's almost a hundred layers of neural pathways.
- dvt 4y ago
  > Pretending GPT is just using a percent chance that some token should show up next is absurd. It's almost a hundred layers of neural pathways. That is literally what it's doing though. "Pretending a CPU is just adding 0s and 1s is absurd. It's made out of billions and billions of transistors."
  - knome 4y ago
    A multi-layer neural net is not limited to vague statistical combinations. You can represent XOR in a neural network. A NAND. If you can represent NAND. You can represent arbitrary logic. Across billions of weights and across nearly a hundred layers of possible decision points. Meaning can be spread across the neurons in ways that allow each to be part of many many different representations. This is only in GPT-3. GPT-4 and soon 5 no doubt exceed these considerably. https://towardsdatascience.com/how-neural-networks-solve-the-xor-problem-59763136bdd7 https://towardsdatascience.com/how-neural-networks-solve-the...
    - dvt 4y ago
      > A multi-layer neural net is not limited to vague statistical combinations. It, absolutely, unequivocally, 100% is only limited to statistical inference. That's how it works. The problem of linear separability (which is like 60 years old, by the way) has nothing to do with logic, but it's actually a byproduct of (a) the fact that neural networks use linear activation functions, and (b) XOR is not separable by a single hyperplane. Hence, you need multiple layers. No idea how you get to "arbitrary logic" from this.
famouswaffles 4y ago
It's obviously what's happening lol. It's why: 1. GPT-3 can speak native-level french despite the language only being 1.8% of it's training corpus. 2. GPT-3 can follow instruction in other languages just fine despite the instruct tuning data being almost entirely in English. 3. LLMs trained on code demonstrably reason better. It's fairly obvious pre-trained LLMs aren't really operating at just the word/token level.
- Donckele 4y ago
  1) 1.8% is still massive 2) 1.8% is still massive 3) what?
  - Sharlin 4y ago
    It’s also fluent (though not at a native level) in, say, Finnish, and can write reasonable poems and whatever, even though Finnish text probably comprises something like 0.01% of the corpus.
    - deleted 4y ago
      [deleted]
    - vidarh 4y ago
      Norwegian as well. Though given Norwegian is also a Germanic language like English, Finnish is a lot more impressive. I managed to get it trip up on one single gendered article in Norwegian, while asking about Sweden, in a way very similar to how a human switching between Norwegian and Swedish might (it would be valid Swedish, and was followed by a Swedish word that's also valid in Norwegian but with another gender), but for the most part it writes Norwegian better than a lot of native speakers I've known would write (but not better than how they'd speak). EDIT: I got it to "translate" a test sentence from Norwegian to "Riksmål", an old-fashioned artificial Norwegian dialect that seeks to stay closer to very conservative old Norwegian (basically a lot closer to Danish) and it managed that just fine too, and there's very little material that you'd be able to tell explicitly is Riksmål as most of the time it'd just be described as Norwegian unless the writer is trying to make a point. Since that worked, I tried Nynorsk ("New Norwegian") which is used by something like 20% of the population, and it did fine. So I tried a local dialect from Eastern Norway, that is predominantly spoken and only occasionally written and it got even that pretty much how I'd have said it myself, after a lecture on how it varies greatly within the region (which is true) The differences there are subtle, and frankly a lot of native speakers would get one or more of these wrong. EDIT2: Last try at stumping it with Norwegian: "AKP-dialect". AKP was a Maoist political party of a few hundred members whose members were largely intellectuals. Many of them adopted a "faux workers dialect". This had no formal name, so it takes some understanding of Norwegian 1970's and 80's politics to even guess what it is. It understood what I meant, but didn't understood what spesifikk language-features I was referring to. Then again, if I asked random Norwegians about that most would need an explanation too. When I mentioned a couple of people who wrote like that it produced a plausible version (but clearly more inspired by the writing style of one of them - Jon Michelet)
mistercow 4y ago
I propose a ban on all arguments involving LLMs that boil down to “your usage of this word doesn’t match my personal connotations for that word”. It’s fine to ask someone to be clarify their definitions, but every other discussion of these models becomes a pointless debate about semantics. See, for example, below.
woadwarrior01 4y ago
> LLMs work on tokens, which, for the most part, are words (longer words are generally split up into multiple tokens). Although this is mostly true, there are exceptions to the rule. Look at the ByT5 family[1] of models for instance. These models operate on byte level representations of text and are inherently token free. I read a recent paper[2] that found that replacing token based text encoders in text to image models with these byte level models improves the spelling abilities of the txt2img models (which are otherwise notoriously bad at spelling in all human languages). [1]: https://arxiv.org/abs/2105.13626 https://arxiv.org/abs/2105.13626 [2]: https://arxiv.org/abs/2212.10562 https://arxiv.org/abs/2212.10562

alexahn 4y ago

Meaning and intent can only be applied if structure can be captured. I'm not proposing building some god-machine that has a library of all possible regular expressions. I'm merely saying that it could be helpful to nudge something like a neural network towards looking for these types of structures, such that they can be a basis for probabilistic encoding. It may be possible that the meaning that gets abstracted thereafter might actually be more concise and generalized.

famouswaffles 4y ago
Part of the breakthrough with LLMs has precisely been about encoding nothing at all about language and letting the transformer figure it out from scratch. We don't know what is necessary for learning language.

bordercases 4y ago

Has this been rigorously considered or is it just vibes? Seems preemptive to presume that unveiling more structure wouldn't give understanding meaning and intent in AI systems a leg up.

famouswaffles 4y ago
It's obviously what's happening. It's why: 1. GPT-3 can speak native-level french despite the language only being 1.8% of it's training corpus. 2. GPT-3 can follow instruction in other languages just fine despite the instruct tuning data being almost entirely in English. 3. LLMs trained on code demonstrably reason better. It's fairly obvious pre-trained LLMs aren't really operating at just the word/token level. Part of the breakthrough with LLMs has precisely been about encoding nothing at all about language and letting the transformer figure it out from scratch. We don't know what is necessary for learning language. Whatever structure we would add at our current level of understanding would far more likely hamper progress than anything. It would also be extremely surface level which LLMs go far beyond.