4 ms·

Apologies in advance for the super naive question; but assuming that we can create vectors that encode for the meaning of entire sentences, what prevents us fro

Apologies in advance for the super naive question; but assuming that we can create vectors that encode for the meaning of entire sentences, what prevents us from training llms to predict those vectors instead of single words?

deleted 3y ago
[deleted]
yorwba 3y ago
You still need to convert between words and sentence vectors somehow. You could try using a faster model for that, but I suspect that the output quality will suffer.
- magicalhippo 3y ago
  LLMs somehow need to do this anyway, implicitly or not. Has anyone tried to do it more explicitly? That is to break out the idea that characters are formed into words, and words into sentences, and a sentence is a sequence of "concepts" for the lack of a better description. So have one NN which takes a sequence of tokens and predicts an moderately-dimensional "word vector", which is fed into another which predicts a high-dimensional "concept vector". Then the "thinking layer" would map a sequence of "concept vectors" to "concept vectors", and then you'd have some layers which does the reverse of the input layers to output tokens which can be printed. Thought being that by splitting it up like this you could swap out the decode and encode layers independently to translate, for example, and so on. Just a shower thought.
  - flawsofar 3y ago
    A sentence is both a sequence and a tree of phrases. Phrases have a head and one or more valents; they’re relations. If you wanted to create an embedding algorithm for phrases, you could and you could throw a transformer at it. I don’t know how you get the output of higher levels to diffuse to phrase and word levels.
marcyb5st 3y ago
Not a stupid question in my opinion. The problem is that once you have the vectors representing the answers you need something like another model that goes back to a word representation of said answers. Something like a diffusion model but for text. Additionally, the function that this diffusion model will approximate won't be injective, but at best surjective and at worst not even a function (in the mathematical meaning) since many textual represantions are possible given an embedding, and most of those won't be valid (not grammatically valid, no sense sentences, ...). Finally remember that the embeddings are a "lossy" representation of some datum and so the inverse function will lose a lot of the nuances/context/... . LLMs avoid the problems above by predicting the next (now next n tokens) in a way that is self consistent with the query and the previous n tokens, so the function they approximate should be mostly surjective.
- throw310822 3y ago
  > The problem is that once you have the vectors representing the answers you need something like another model that goes back to a word representation of said answers. Something like a diffusion model but for text. Could it be just a smaller llm that takes as input both the semantic vector and the prompt, and is trained to predict the output tokens based on those? A model with high linguistic abilities and very little reasoning skills.
  - marcyb5st 3y ago
    I think what you suggest would be very similar to a encoder-decoder architecture, which has been abandoned in favor of decoder-only architectures (https://cameronrwolfe.substack.com/p/decoder-only-transformers-the-workhorse https://cameronrwolfe.substack.com/p/decoder-only-transforme...). So I am guessing that what you suggest has already been tried and didn't work out, ut not sure why (the problems I mentioned above or something else). Sorry, that's where the limit of my knowledge is. I work on ML stuff, but mostly on "traditional" deep learning and so I am not up to speed with the genAI field (also, the sheer amount of papers coming out makes it basically impossible stay up to date of you're not in the field).
- HanClinto 3y ago
faabian 3y ago
Author here -- that's a very good point and as I understand work in progress in different teams. Training autoencoders for language is actually super easy given the small amount of information contained in text (compared to vision/video), the hard part is making the model focus on the semantic part if all signal we have comes from exact match in token space. Hence Yann LeCun's ideas on joint embedding predictive architectures. Note also that there is always a trade-off between auxiliary tasks giving more signal but shifting the focus. In our case, we noticed degradation if the number of predicted tokens is too high. So latent prediction methods need to sort out what is useful.
- mike_hearn 3y ago
  Aren't the models already doing this, in a way? We know they can do things like write rhyming poems and song lyrics that do make perfect sense, so at some point the activations must be encoding some sort of overall plan for the upcoming sentences, even if maybe every word isn't predicted yet.
  - faabian 3y ago
    Yes. Otherwise next-token models wouldn't be nearly as good as they are. But the question is how to train these capabilities most efficiently! We had some interesting findings on how with increasing model/dataset scale/data quality, capabilities can move from "only learnable with multi-token prediction" to "indifferent" and "multi-token prediction actually hurts". This depends on the capability itself, induction e.g. matures way earlier in this sense than code generation capabilities.
    - mike_hearn 3y ago
      Is it possible that anti-scaling effect occurs because you are removing some middle layers to free up space for the extra output heads? I only scanned the paper quickly but what happens if you treat the technique as strictly additive and don't keep parameter sizes fixed?
  - mjburgess 3y ago
jerpint 3y ago
My understanding is that tokenization is part of the bottleneck. When you break up a sentence to tokens, each token gets a vector representation. The dictionary of all tokens would be infinite if it was at the sentence level
- faabian 3y ago
  Vectors can do what one-hot vectors cannot do -- no one said inputs need to be rows from an token_id -> vector embeddings map. Basically, we are doing this already by moving from one-hot vectors to n-tuples of one-hot vectors, increasing the effective vocabulary size from V to V^n.
bjourne 3y ago
That's hierarchical prediction. On one level you predict the style of the paragraph, on another the tone and form of the sentence, and on the third the next word. However, this form of prediction is quite difficult since predictions from the layers affect each other.
wangii 3y ago
the problem is then the total number of computation drops dramatically therefore leads to much less "thinking" power. i think the idea originated from an understanding that when we write/speak, we have an overall idea. my current hypothesis is it's probably an illusion. you may want to search for "filler" papers to read.
everforward 3y ago
Also a noob here, if we encoded, trained on and synthesized sentence vectors, wouldn’t that move the AIs ability to create novel things up from sentences to words? I.e. we currently operate on words (roughly) so the AI can only use words it knows but can synthesize unique sentences from words. If the AI operates on sentences, wouldn’t it only be able to regurgitate sentences it has seen before? So it could synthesize novel paragraphs, but not sentences? I’m not convinced that sentences are a useful abstraction for AI (in English, anyways). They’re barely useful to humans. Check out your average chat conversation, email, YouTube comment, etc. There’s a very good chance the sentences aren’t actually sentences, or that they haven’t even bothered to use punctuation. I just don’t think sentences map to a semantic device. A sentence could be two words or half an English paper depending on the writer. It could traverse a half dozen ideas or a single one. Where a sentence ends generally is more about the writer than the semantics.

deleted 3y ago

[deleted]

yorwba 3y ago

You still need to convert between words and sentence vectors somehow. You could try using a faster model for that, but I suspect that the output quality will suffer.

magicalhippo 3y ago
LLMs somehow need to do this anyway, implicitly or not. Has anyone tried to do it more explicitly? That is to break out the idea that characters are formed into words, and words into sentences, and a sentence is a sequence of "concepts" for the lack of a better description. So have one NN which takes a sequence of tokens and predicts an moderately-dimensional "word vector", which is fed into another which predicts a high-dimensional "concept vector". Then the "thinking layer" would map a sequence of "concept vectors" to "concept vectors", and then you'd have some layers which does the reverse of the input layers to output tokens which can be printed. Thought being that by splitting it up like this you could swap out the decode and encode layers independently to translate, for example, and so on. Just a shower thought.
- flawsofar 3y ago
  A sentence is both a sequence and a tree of phrases. Phrases have a head and one or more valents; they’re relations. If you wanted to create an embedding algorithm for phrases, you could and you could throw a transformer at it. I don’t know how you get the output of higher levels to diffuse to phrase and word levels.

marcyb5st 3y ago

Not a stupid question in my opinion. The problem is that once you have the vectors representing the answers you need something like another model that goes back to a word representation of said answers. Something like a diffusion model but for text. Additionally, the function that this diffusion model will approximate won't be injective, but at best surjective and at worst not even a function (in the mathematical meaning) since many textual represantions are possible given an embedding, and most of those won't be valid (not grammatically valid, no sense sentences, ...). Finally remember that the embeddings are a "lossy" representation of some datum and so the inverse function will lose a lot of the nuances/context/... . LLMs avoid the problems above by predicting the next (now next n tokens) in a way that is self consistent with the query and the previous n tokens, so the function they approximate should be mostly surjective.

throw310822 3y ago
> The problem is that once you have the vectors representing the answers you need something like another model that goes back to a word representation of said answers. Something like a diffusion model but for text. Could it be just a smaller llm that takes as input both the semantic vector and the prompt, and is trained to predict the output tokens based on those? A model with high linguistic abilities and very little reasoning skills.
- marcyb5st 3y ago
  I think what you suggest would be very similar to a encoder-decoder architecture, which has been abandoned in favor of decoder-only architectures (https://cameronrwolfe.substack.com/p/decoder-only-transformers-the-workhorse https://cameronrwolfe.substack.com/p/decoder-only-transforme...). So I am guessing that what you suggest has already been tried and didn't work out, ut not sure why (the problems I mentioned above or something else). Sorry, that's where the limit of my knowledge is. I work on ML stuff, but mostly on "traditional" deep learning and so I am not up to speed with the genAI field (also, the sheer amount of papers coming out makes it basically impossible stay up to date of you're not in the field).
HanClinto 3y ago

faabian 3y ago

Author here -- that's a very good point and as I understand work in progress in different teams. Training autoencoders for language is actually super easy given the small amount of information contained in text (compared to vision/video), the hard part is making the model focus on the semantic part if all signal we have comes from exact match in token space. Hence Yann LeCun's ideas on joint embedding predictive architectures. Note also that there is always a trade-off between auxiliary tasks giving more signal but shifting the focus. In our case, we noticed degradation if the number of predicted tokens is too high. So latent prediction methods need to sort out what is useful.

mike_hearn 3y ago
Aren't the models already doing this, in a way? We know they can do things like write rhyming poems and song lyrics that do make perfect sense, so at some point the activations must be encoding some sort of overall plan for the upcoming sentences, even if maybe every word isn't predicted yet.
- faabian 3y ago
  Yes. Otherwise next-token models wouldn't be nearly as good as they are. But the question is how to train these capabilities most efficiently! We had some interesting findings on how with increasing model/dataset scale/data quality, capabilities can move from "only learnable with multi-token prediction" to "indifferent" and "multi-token prediction actually hurts". This depends on the capability itself, induction e.g. matures way earlier in this sense than code generation capabilities.
  - mike_hearn 3y ago
    Is it possible that anti-scaling effect occurs because you are removing some middle layers to free up space for the extra output heads? I only scanned the paper quickly but what happens if you treat the technique as strictly additive and don't keep parameter sizes fixed?
- mjburgess 3y ago

jerpint 3y ago

My understanding is that tokenization is part of the bottleneck. When you break up a sentence to tokens, each token gets a vector representation. The dictionary of all tokens would be infinite if it was at the sentence level

faabian 3y ago
Vectors can do what one-hot vectors cannot do -- no one said inputs need to be rows from an token_id -> vector embeddings map. Basically, we are doing this already by moving from one-hot vectors to n-tuples of one-hot vectors, increasing the effective vocabulary size from V to V^n.

bjourne 3y ago

That's hierarchical prediction. On one level you predict the style of the paragraph, on another the tone and form of the sentence, and on the third the next word. However, this form of prediction is quite difficult since predictions from the layers affect each other.

wangii 3y ago

the problem is then the total number of computation drops dramatically therefore leads to much less "thinking" power. i think the idea originated from an understanding that when we write/speak, we have an overall idea. my current hypothesis is it's probably an illusion. you may want to search for "filler" papers to read.

everforward 3y ago

Also a noob here, if we encoded, trained on and synthesized sentence vectors, wouldn’t that move the AIs ability to create novel things up from sentences to words? I.e. we currently operate on words (roughly) so the AI can only use words it knows but can synthesize unique sentences from words. If the AI operates on sentences, wouldn’t it only be able to regurgitate sentences it has seen before? So it could synthesize novel paragraphs, but not sentences? I’m not convinced that sentences are a useful abstraction for AI (in English, anyways). They’re barely useful to humans. Check out your average chat conversation, email, YouTube comment, etc. There’s a very good chance the sentences aren’t actually sentences, or that they haven’t even bothered to use punctuation. I just don’t think sentences map to a semantic device. A sentence could be two words or half an English paper depending on the writer. It could traverse a half dozen ideas or a single one. Where a sentence ends generally is more about the writer than the semantics.