4 ms·

There is. It’s just that we use LLMs like idiots. We should have more tooling to play with the vectors directly. But a lot of the literature is built around peo

There is. It’s just that we use LLMs like idiots. We should have more tooling to play with the vectors directly. But a lot of the literature is built around people playing with chat gpt, for which this is not an option as a business decision by open ai and nothing else. In the image gen space which is mildly better but still not great we would just assign a positive or negative weight to the item. E.g. (seaborn:-1.3). This is a bit harder in LLMs as designed because the prompting space is a back and forth conversation and not a description of itself. It would be nice if we could do both more cleanly separated. If you ever try to read a doc into chat gpt and get it to summarize n ideas in a single prompt, remember that this is open AI’s fault. What we should be doing is caching the vector representing the doc and then passing n separate queries to extract n separate ideas about the same vector.

inciampati 2y ago
The problem is that transformers have a state space which includes attention over their input. Thus, it's very hard to manipulate. I think you're right, but it will require moving to more recurrent architectures.
Bluestein 2y ago
> We should have more tooling to play with the vectors directly. Ah! This.- PS. Who knows. This ("dealing with the vectors") might become analogous to open sourcing" some day ...
- jncfhnb 2y ago
  It more or less is. This is something you can do with models that have open weights. People claiming those models are not “open source” really have no idea about the “preferred form” of “code” to change an LLM
lossolo 2y ago
You compared it to diffusion models, which work entirely differently. Could you elaborate what you mean? I understand how transformers work and their implementation, but I'm unsure how with the current transformer architecture you can do what you described by "playing with vectors directly". Which vectors are you referring to? attention vectors?
- jncfhnb 2y ago
  The simplest example is contextual prompts. Let’s say you have a 3000 token prompt that you put in front of every query. You could A) have the LLM ingest and reprocess the same prompt every single time at needless cost Or B) process it one time and reuse the vector embedding of the prompt OpenAI forces you to do A.
  - lossolo 2y ago
    I'm confused now, how does it solve the need to write "I REPEAT"? As a strengthening the signal to follow instructions. Besides that, in transformer models, the computation of each token’s output vector in a sequence depends on all other tokens in the sequence due to the self-attention mechanism. This interdependency means that you cannot simply "reuse"/add on to an existing processed sequence without re-processing the entire sequence, as the presence of new tokens (user input) alters the calculations of the attention mechanisms throughout the sequence. So what you basically proposing is not possible in current architecture.
    - jncfhnb 2y ago
      Tokens are generally consumed sequentially. The model does not care what token 3001 is when it’s creating the embedding representation of tokens 1-3000. You can use the cached state of the 3000th token rather than reprocessing it each time.
      - lossolo 2y ago
        I think you are confused on how transformers work. You are confusing between static embeddings and the dynamic, contextual outputs calculated by transformers. While initial embeddings (representations BEFORE processing through the transformer layers) are relatively static, the output embeddings from each layer of the transformer are highly dynamic and context-dependent. They incorporate information from the entire input sequence via the attention mechanisms. So when transformers process an input sequence, they don't just look at each token in isolation but consider the ENTIRE sequence's context through complex inter-token relationships. This makes the technique of caching outputs non-trivial and context-specific. The sequential processing does not imply independence of tokens but underscores the integral role of context and sequence in generating accurate and coherent outputs. This should help you understand better how transformers work: https://bbycroft.net/llm https://bbycroft.net/llm

inciampati 2y ago

The problem is that transformers have a state space which includes attention over their input. Thus, it's very hard to manipulate. I think you're right, but it will require moving to more recurrent architectures.

Bluestein 2y ago

> We should have more tooling to play with the vectors directly. Ah! This.- PS. Who knows. This ("dealing with the vectors") might become analogous to open sourcing" some day ...

jncfhnb 2y ago
It more or less is. This is something you can do with models that have open weights. People claiming those models are not “open source” really have no idea about the “preferred form” of “code” to change an LLM

lossolo 2y ago

You compared it to diffusion models, which work entirely differently. Could you elaborate what you mean? I understand how transformers work and their implementation, but I'm unsure how with the current transformer architecture you can do what you described by "playing with vectors directly". Which vectors are you referring to? attention vectors?

jncfhnb 2y ago
The simplest example is contextual prompts. Let’s say you have a 3000 token prompt that you put in front of every query. You could A) have the LLM ingest and reprocess the same prompt every single time at needless cost Or B) process it one time and reuse the vector embedding of the prompt OpenAI forces you to do A.
- lossolo 2y ago
  I'm confused now, how does it solve the need to write "I REPEAT"? As a strengthening the signal to follow instructions. Besides that, in transformer models, the computation of each token’s output vector in a sequence depends on all other tokens in the sequence due to the self-attention mechanism. This interdependency means that you cannot simply "reuse"/add on to an existing processed sequence without re-processing the entire sequence, as the presence of new tokens (user input) alters the calculations of the attention mechanisms throughout the sequence. So what you basically proposing is not possible in current architecture.
  - jncfhnb 2y ago
    Tokens are generally consumed sequentially. The model does not care what token 3001 is when it’s creating the embedding representation of tokens 1-3000. You can use the cached state of the 3000th token rather than reprocessing it each time.
    - lossolo 2y ago
      I think you are confused on how transformers work. You are confusing between static embeddings and the dynamic, contextual outputs calculated by transformers. While initial embeddings (representations BEFORE processing through the transformer layers) are relatively static, the output embeddings from each layer of the transformer are highly dynamic and context-dependent. They incorporate information from the entire input sequence via the attention mechanisms. So when transformers process an input sequence, they don't just look at each token in isolation but consider the ENTIRE sequence's context through complex inter-token relationships. This makes the technique of caching outputs non-trivial and context-specific. The sequential processing does not imply independence of tokens but underscores the integral role of context and sequence in generating accurate and coherent outputs. This should help you understand better how transformers work: https://bbycroft.net/llm https://bbycroft.net/llm