10 ms·

Help me understand: when they say that the facts are stored as a linear function… are they saying that the LLM has a sort of N-dimensional “fact space” encoded

by derefr 3y ago

Help me understand: when they say that the facts are stored as a linear function… are they saying that the LLM has a sort of N-dimensional “fact space” encoded into the model in some manner, where facts are embedded into the space as (points / hyperspheres / Voronoi manifolds / etc); and where recalling a fact is — at least in an abstract sense — the NN computing / remembering a key to use, and then doing a key-value lookup in this space? If so: how do you embed a KV-store into an edge-propagated graphical model? Are there even any well-known techniques for doing that “by hand” right now? (Also, fun tangent: isn't the "memory palace" memory technique, an example of human brains embedding facts into a linear function for easier retrieval?)

bionhoward 3y ago
[Layer] Normalization constrains huge vectors representing tokens (input fragments) to positions on a unit ball (I think), and the attention mechanism operates by rotating the unconstrained ones based on the sum of their angles relative to all the others. I only skimmed the paper but believe the point here is that there are relatively simple functions hiding in or recoverable from the bigger network which specifically address certain categories of relationships between concepts. Since it would, in theory, be possible to optimize such functions more directly if they are possible to isolate, could this enable advances in the way such models are trained? Absolutely. After all, one of the best criticisms of “modern” AI is the notion we’re just mixing around a soup of linear algebra. Allowing some sense of modularity (reductionism) could make them less of a black box and more of a component driven approach (in the lagging concept space and not just the leading layer space)
jacobn 3y ago
The fundamental operation done by the transformer, softmax(Q.K^T).V, is essentially a KV-store lookup. The Query is dotted with the Key, then you take the softmax to pick mostly one winning Key (the Key closest to the Query basically), and then use the corresponding Value. That is really, really close to a KV lookup, except it's a little soft (i.e. can hit multiple Keys), and it can be optimized using gradient descent style methods to find the suitable QKV mappings.
- naveen99 3y ago
  Not sure there is any real lookup happening. Q,K are the same and sometimes even v is the same…
  - toxik 3y ago
    Q, K, V are not the same. In self-attention, they are all computed by separate linear transformation of the same input (ie the previous layer’s output). In cross-attention even this is not true, then K and V are computed by linear transformation of whatever is cross-attended, and Q is computed by linear transformation of the input as before.
    - ewild 3y ago
      yeah a common misconception people think because the input is the same they forget that their is a pre attention linear transofrmation for q k and v (using the decoder only version obv v is diff with encoder decoder bert style)
    - naveen99 3y ago
      It’s still a stretch to call that a look up.
thfuran 3y ago
>isn't the "memory palace" memory technique, an example of human brains embedding facts into a linear function for easier retrieval? I'm not sure I see how that's a linear function.
samus 3y ago
The memory palace is a hack that works because in an evolutionary sense our brain's purpose is to help us navigate our world and be effective in it. To do that, it has to be really good at remembering locations, to plot paths through and between them, and to translate that into speech or motion.

by derefr 3y ago

bionhoward 3y ago

[Layer] Normalization constrains huge vectors representing tokens (input fragments) to positions on a unit ball (I think), and the attention mechanism operates by rotating the unconstrained ones based on the sum of their angles relative to all the others. I only skimmed the paper but believe the point here is that there are relatively simple functions hiding in or recoverable from the bigger network which specifically address certain categories of relationships between concepts. Since it would, in theory, be possible to optimize such functions more directly if they are possible to isolate, could this enable advances in the way such models are trained? Absolutely. After all, one of the best criticisms of “modern” AI is the notion we’re just mixing around a soup of linear algebra. Allowing some sense of modularity (reductionism) could make them less of a black box and more of a component driven approach (in the lagging concept space and not just the leading layer space)

jacobn 3y ago

The fundamental operation done by the transformer, softmax(Q.K^T).V, is essentially a KV-store lookup. The Query is dotted with the Key, then you take the softmax to pick mostly one winning Key (the Key closest to the Query basically), and then use the corresponding Value. That is really, really close to a KV lookup, except it's a little soft (i.e. can hit multiple Keys), and it can be optimized using gradient descent style methods to find the suitable QKV mappings.

naveen99 3y ago
Not sure there is any real lookup happening. Q,K are the same and sometimes even v is the same…
- toxik 3y ago
  Q, K, V are not the same. In self-attention, they are all computed by separate linear transformation of the same input (ie the previous layer’s output). In cross-attention even this is not true, then K and V are computed by linear transformation of whatever is cross-attended, and Q is computed by linear transformation of the input as before.
  - ewild 3y ago
    yeah a common misconception people think because the input is the same they forget that their is a pre attention linear transofrmation for q k and v (using the decoder only version obv v is diff with encoder decoder bert style)
  - naveen99 3y ago
    It’s still a stretch to call that a look up.

thfuran 3y ago

>isn't the "memory palace" memory technique, an example of human brains embedding facts into a linear function for easier retrieval? I'm not sure I see how that's a linear function.

samus 3y ago

The memory palace is a hack that works because in an evolutionary sense our brain's purpose is to help us navigate our world and be effective in it. To do that, it has to be really good at remembering locations, to plot paths through and between them, and to translate that into speech or motion.