3 ms·

We already know how to do "short term memory". We can mathematically condense/average embedding representations of text and use these as input (i.e. combine a p

We already know how to do "short term memory". We can mathematically condense/average embedding representations of text and use these as input (i.e. combine a page of text into the representation space of one or several tokens, textual inversion style). Once embeddings get good enough, this kind of memory will be ubiquitous. Not sure why we are collectively having amnesia about sentence transformers and related ideas It's worth noting that "soft-prompts" were a thing that the coomer crowd was already using with openassistant. Not sure why the mainstream NLP crowd is being so damn slow in implementing stuff like this NLP is shocking undertooled, and I wrote a whole gist complaining about this that got to the front page of HN awhile ago: https://gist.github.com/Hellisotherpeople/45c619ee22aac6865ca4bb328eb58faf https://gist.github.com/Hellisotherpeople/45c619ee22aac6865c...

brucethemoose2 3y ago
Soft prompts and textual inversion, as I understand them, required considerable hardware resources and were effectively made obsolete by low precision LORA training. But the text token merging people do for negative prompts is fast iirc. Not that you are wrong. LLM prompting does seem quite primitive compared to zoo of encoding/prompting methods we have for SD... And some of it is directly applicable to LLMs. I never even thought about that before.
- Der_Einzige 3y ago
  All of it is directly applicable to LLMs. Textual Inversion/Soft Prompts are not made "obsolete" at all by Loras. Textual Inversion is far faster to train, extremely effective on newer models like SDXL which have better clip encoding, and has neat properties of not messing with the general weights of the rest of the model, which is sometimes desirable.
- ijk 3y ago
  Soft prompts for language models are still viable, but the most accessible version of open source soft prompt training was a TPU-based implementation that depended on features of Google Colab that were no longer available. You can modify it to run locally on a GPU, though it is a bit slow to train. It provides different advantages when compared to a LoRA.
TeMPOraL 3y ago
I occasionally remark or ask, how is it that LLM APIs are always either text -> text, or at best (for smaller models) text -> embeddings, with the idea being of you stuffing the latter in some database for future lookups - where the most interesting thing to expose should be embeddings -> text. As in, I want to be able to feed the LLM arbitrary embedding vectors - whether they come from averaging embeddings of different prompts, some general optimization pattern (textual inversion style, indeed), or just iteratively shifting the input vector in some direction, walking down the latent space and generating output every step, just to see what happens. I ask that, and I get blank stares, or non-answers like "oh, it's always been available, you can e.g. put text in, get embeddings out". Yeah, but I want to do the reverse. It's like I was making so big a category error, asking such nonsensical question, that no one can even parse it. But am I?
- PeterisP 3y ago
  What exactly do you mean by "embeddings" there? The way a transformer-based LLM works is that right after tokenization you get a "vector of embedding vectors" i.e. an embedding for each token, and that gets processed by the actual transformer layers. If instead of a list of tokens you want to pass in a list of embedding vectors, that is trivial (assuming you're running a LLM with your own code locally), that would be a <5 line function. However, I'm not really sure when/why I would want to do that, as "averaging embeddings of different prompts, some general optimization pattern (textual inversion style, indeed), or just iteratively shifting the input vector in some direction, walking down the latent space and generating output every step" are NOT really enabled by that - since that inherently must be a vector of token-embedding vectors combined with positional embeddings, you can't just average different prompts, and there isn't a clear way how to tweak the individual token embeddings that would map to tweaking the whole prompt in some meaningful direction. In essence, your question seems to make sense if the text was converted to a single "embedding vector", but it's not, at every layer of a transformer model the state is a list of separate embeddings distinctly separated at a token level.
  - TeMPOraL 3y ago
    > If instead of a list of tokens you want to pass in a list of embedding vectors, that is trivial (assuming you're running a LLM with your own code locally), that would be a <5 line function. That sounds like what I wanted. I haven't got to the point of running open-source LLMs locally yet, so I didn't realize it's as easy as you say - rather, since I haven't seen it being mentioned as something you do with LLM, I assumed it must be nontrivial. > However, I'm not really sure when/why I would want to do that High-level dream: I want to explore the latent space of an LLM. Get a feel for how it's organized, and what of it is spurious, and what structures seem to be common among LLMs. Find answers to questions that will come to me when I "go there" and "look around". It might be a futile pursuit - but I feel it's not the idea that's the problem, but rather that high-dimensional spaces might be too big to explore without mathematical understanding that's way beyond mine. > as "[the specific ideas I listed]" are NOT really enabled by that - since that inherently must be a vector of token-embedding vectors combined with positional embeddings, you can't just average different prompts, In general, or at all? Am I wrong in suspecting that averaging the embeddings of "orange" and "apple" and "banana" may give us a new embedding that, fed to the LLM, will make it output something meaningful (vs. random)? From what I read today, elsewhere in the thread, it seems that this works as expected for Stable Diffusion family of models. And sure, if I take two random multi-token prompts, I expect the result will make LLM output some noise in the shape of word salad. But what if it's multiple closely-related prompts? I have my suspicions, but I want to find out for sure. > and there isn't a clear way how to tweak the individual token embeddings that would map to tweaking the whole prompt in some meaningful direction. Shifting prompts in some meaningful direction may not be trivial (or useful), but it's not the only interesting thing to do. What if I push a single token in the prompt a little bit in some direction? What if I push all of them? How things change with subsequent pushes? > In essence, your question seems to make sense if the text was converted to a single "embedding vector", but it's not, at every layer of a transformer model the state is a list of separate embeddings distinctly separated at a token level. I'm definitely not assuming the text can be converted to a single "embedding vector". My questions boil down to, where is debug-level access to the LLM? Why is everyone focused on stringing tokens, instead of looking at what's lurking between the tokens and their sequences, in areas of the latent space you can't easily get to without embeddings that don't map to any possible prompt. IDK, maybe my mental model of transformer models is completely broken, or maybe all those questions have been asked, and didn't lead to anything of interest, and I just can't seem to find the right papers covering this.

brucethemoose2 3y ago

Soft prompts and textual inversion, as I understand them, required considerable hardware resources and were effectively made obsolete by low precision LORA training. But the text token merging people do for negative prompts is fast iirc. Not that you are wrong. LLM prompting does seem quite primitive compared to zoo of encoding/prompting methods we have for SD... And some of it is directly applicable to LLMs. I never even thought about that before.

Der_Einzige 3y ago
All of it is directly applicable to LLMs. Textual Inversion/Soft Prompts are not made "obsolete" at all by Loras. Textual Inversion is far faster to train, extremely effective on newer models like SDXL which have better clip encoding, and has neat properties of not messing with the general weights of the rest of the model, which is sometimes desirable.
ijk 3y ago
Soft prompts for language models are still viable, but the most accessible version of open source soft prompt training was a TPU-based implementation that depended on features of Google Colab that were no longer available. You can modify it to run locally on a GPU, though it is a bit slow to train. It provides different advantages when compared to a LoRA.

TeMPOraL 3y ago

I occasionally remark or ask, how is it that LLM APIs are always either text -> text, or at best (for smaller models) text -> embeddings, with the idea being of you stuffing the latter in some database for future lookups - where the most interesting thing to expose should be embeddings -> text. As in, I want to be able to feed the LLM arbitrary embedding vectors - whether they come from averaging embeddings of different prompts, some general optimization pattern (textual inversion style, indeed), or just iteratively shifting the input vector in some direction, walking down the latent space and generating output every step, just to see what happens. I ask that, and I get blank stares, or non-answers like "oh, it's always been available, you can e.g. put text in, get embeddings out". Yeah, but I want to do the reverse. It's like I was making so big a category error, asking such nonsensical question, that no one can even parse it. But am I?

PeterisP 3y ago
What exactly do you mean by "embeddings" there? The way a transformer-based LLM works is that right after tokenization you get a "vector of embedding vectors" i.e. an embedding for each token, and that gets processed by the actual transformer layers. If instead of a list of tokens you want to pass in a list of embedding vectors, that is trivial (assuming you're running a LLM with your own code locally), that would be a <5 line function. However, I'm not really sure when/why I would want to do that, as "averaging embeddings of different prompts, some general optimization pattern (textual inversion style, indeed), or just iteratively shifting the input vector in some direction, walking down the latent space and generating output every step" are NOT really enabled by that - since that inherently must be a vector of token-embedding vectors combined with positional embeddings, you can't just average different prompts, and there isn't a clear way how to tweak the individual token embeddings that would map to tweaking the whole prompt in some meaningful direction. In essence, your question seems to make sense if the text was converted to a single "embedding vector", but it's not, at every layer of a transformer model the state is a list of separate embeddings distinctly separated at a token level.
- TeMPOraL 3y ago
  > If instead of a list of tokens you want to pass in a list of embedding vectors, that is trivial (assuming you're running a LLM with your own code locally), that would be a <5 line function. That sounds like what I wanted. I haven't got to the point of running open-source LLMs locally yet, so I didn't realize it's as easy as you say - rather, since I haven't seen it being mentioned as something you do with LLM, I assumed it must be nontrivial. > However, I'm not really sure when/why I would want to do that High-level dream: I want to explore the latent space of an LLM. Get a feel for how it's organized, and what of it is spurious, and what structures seem to be common among LLMs. Find answers to questions that will come to me when I "go there" and "look around". It might be a futile pursuit - but I feel it's not the idea that's the problem, but rather that high-dimensional spaces might be too big to explore without mathematical understanding that's way beyond mine. > as "[the specific ideas I listed]" are NOT really enabled by that - since that inherently must be a vector of token-embedding vectors combined with positional embeddings, you can't just average different prompts, In general, or at all? Am I wrong in suspecting that averaging the embeddings of "orange" and "apple" and "banana" may give us a new embedding that, fed to the LLM, will make it output something meaningful (vs. random)? From what I read today, elsewhere in the thread, it seems that this works as expected for Stable Diffusion family of models. And sure, if I take two random multi-token prompts, I expect the result will make LLM output some noise in the shape of word salad. But what if it's multiple closely-related prompts? I have my suspicions, but I want to find out for sure. > and there isn't a clear way how to tweak the individual token embeddings that would map to tweaking the whole prompt in some meaningful direction. Shifting prompts in some meaningful direction may not be trivial (or useful), but it's not the only interesting thing to do. What if I push a single token in the prompt a little bit in some direction? What if I push all of them? How things change with subsequent pushes? > In essence, your question seems to make sense if the text was converted to a single "embedding vector", but it's not, at every layer of a transformer model the state is a list of separate embeddings distinctly separated at a token level. I'm definitely not assuming the text can be converted to a single "embedding vector". My questions boil down to, where is debug-level access to the LLM? Why is everyone focused on stringing tokens, instead of looking at what's lurking between the tokens and their sequences, in areas of the latent space you can't easily get to without embeddings that don't map to any possible prompt. IDK, maybe my mental model of transformer models is completely broken, or maybe all those questions have been asked, and didn't lead to anything of interest, and I just can't seem to find the right papers covering this.