4 ms·

As someone who started reading ML papers 10 years ago, I find transformers pretty simple compared to most architectures. For example Google's Inception was a co

As someone who started reading ML papers 10 years ago, I find transformers pretty simple compared to most architectures. For example Google's Inception was a complex mesh of various sized convolutions, other models have an internal algorithm like Non Maximum Suppression for object detection, or Connectionist Temporal Classification for OCR, GANs use complicated probability theory for the loss function. Even LSTM is more complicated. If anything, we have been abandoning exotic neural nets in favour of a single architecture and that one is pretty simple, just linear layers (vector-matrix product), key-value products (matrix-matrix product), softmax (fancy normalisation), weighted averaging (a sum of products) and skip connections (an addition). Maybe it's become hard for me to see what is complicated about it, I'd be curious to know what part is difficult. Is it the embeddings, masking, multiple heads, gradient descent, ...? Embeddings have been famous for 10 years, ever since the king - man + woman = queen paper. You don't need to be able to derive the gradients for the network by hand to understand it. In short, a transformer is mixing information between tokens in a sequence and computing updates. The mixing part is the "self attention" or "cross attention". The updating part is the feed-forward sublayer. It has skip connections (adds the input to the output) in order to keep training stable.

chaxor 3y ago
Even when someone understands the architecture very well, there's still great utility in having a full graph representation of the full NN architecture. This could be for teaching students, or for doing analyses on the structure of the full network, etc.
Buttons840 3y ago
Simple is good. Especially in machine learning where a bug usually means that it kinda works, but not as well as it could. Also, when an off-the-shelf algorithm half works, it's good to be able to add you own tweaks to it, and again, this requires simplicity. For a complicated architecture to succeed, it's going to need to reliably achieve state of the art performance on everything without requiring any adjustment or tweaks.
rasbt 3y ago
Agreed, compared to other architectures, transformers are actually quite straight-forward. The complicated part comes more from training it in distributed setups, making the data loading and tensor parallelism work due to the large size etc. Like the vanilla architecture is simple, but the practical implementation for large-scale training can be a bit complicated.
lalaithion 3y ago
The things that still confuse me about transformers is: 1. Why do we _add_ the positional embedding to the semantic embedding? It seems like it means certain semantic directions are irreversibly with certain positions. 2. I don't understand why the attention head (which I can implement and follow the math of) is described as "key query value lookup". Specifically, the Q and K matrices aren't structurally distinct – the projections into them will learn different weights, but one doesn't start out biased key-ward and the other query-ward.
- visarga 3y ago
  Good questions. The first one: transformers are "permutation invariant" by nature, so if you permute the input and apply the opposite permutation to the output you get the exact same thing. The transformer itself has no positional information. RNNs by comparison have positional information by design, they go token by token, but the transformer is parallel and all tokens are just independent "channels". So what can be done? You put positional embeddings in it - either by adding them to the tokens (concatenation was also ok, but less efficient) or by inserting relative distance biases in the attention matrix. It's a fix to make it understand time. It's still puzzling this works, because mixing text tokens with position tokens seems to cause a conflict, but it doesn't in practice. The model will learn to use the embedding vector for both, maybe specialising a part for semantics and another for position. The second question. Neural nets find a way to differentiate the keys from queries by simply doing gradient descent. If we tell the model it should generate a specific token here, then it needs to fix the keys and queries to make it happen. The architecture is pretty dumb, the secret is the training data - everything the transformer learns comes from the training set. We should think about the training data when we marvel at what transformers can do. The architecture doesn't tell us why they work so well.
  - FartyMcFarter 3y ago
    > The architecture is pretty dumb, the secret is the training data If this were true, we could throw the same training data at any other "dumb" architecture and it would learn language at least as well/fast as transformers do. But we don't see that happening, so the architecture must be smartly designed for this purpose.
kk58 3y ago
In CNN the layers seem to learn geometric primitive and deeper layers seem to learn more complex geometric patterns loosely speaking. In transformer what do query key matrices learn? How are their weights somehow working to extract context no matter which word appears in which position?
- visarga 3y ago
  The transformer doesn't have the nice pyramid shape of CNNs, but it still needs multiple layers. There have been papers showing non-trivial interactions between successive layers, forming more complex circuits. https://transformer-circuits.pub/2021/framework/index.html https://transformer-circuits.pub/2021/framework/index.html (warning, advanced difficulty) The Q an K matrices learn how to relate tokens. Each of the heads will learn to extract a different relation. For example, one will link to the next token, another will link pronouns to their references, another would be matching brackets, etc. Check out the cute diagrams here: https://www.arxiv-vanity.com/papers/1904.02679/ https://www.arxiv-vanity.com/papers/1904.02679/ So each head (Q and K pair) is like a program doing a specific pattern of lookup.
  - kk58 3y ago
    Agree. So attention is like hierarchy of graphs where nodes are tokens and edges are attention scores per head. Now what's trippy is this node has position data. So node feature and position it appears is used to create a operator that projects a sequence to a semantic space. This seems to work for any modality of data.. so there is some thing about order in which data appears that seems to be linked to semantics and for me hints about some deep causal structure being latent in LLM
qumpis 3y ago
King-man got me thinking about the VAE paper for a second :)

chaxor 3y ago

Even when someone understands the architecture very well, there's still great utility in having a full graph representation of the full NN architecture. This could be for teaching students, or for doing analyses on the structure of the full network, etc.

Buttons840 3y ago

Simple is good. Especially in machine learning where a bug usually means that it kinda works, but not as well as it could. Also, when an off-the-shelf algorithm half works, it's good to be able to add you own tweaks to it, and again, this requires simplicity. For a complicated architecture to succeed, it's going to need to reliably achieve state of the art performance on everything without requiring any adjustment or tweaks.

rasbt 3y ago

Agreed, compared to other architectures, transformers are actually quite straight-forward. The complicated part comes more from training it in distributed setups, making the data loading and tensor parallelism work due to the large size etc. Like the vanilla architecture is simple, but the practical implementation for large-scale training can be a bit complicated.

lalaithion 3y ago

The things that still confuse me about transformers is: 1. Why do we _add_ the positional embedding to the semantic embedding? It seems like it means certain semantic directions are irreversibly with certain positions. 2. I don't understand why the attention head (which I can implement and follow the math of) is described as "key query value lookup". Specifically, the Q and K matrices aren't structurally distinct – the projections into them will learn different weights, but one doesn't start out biased key-ward and the other query-ward.

visarga 3y ago
Good questions. The first one: transformers are "permutation invariant" by nature, so if you permute the input and apply the opposite permutation to the output you get the exact same thing. The transformer itself has no positional information. RNNs by comparison have positional information by design, they go token by token, but the transformer is parallel and all tokens are just independent "channels". So what can be done? You put positional embeddings in it - either by adding them to the tokens (concatenation was also ok, but less efficient) or by inserting relative distance biases in the attention matrix. It's a fix to make it understand time. It's still puzzling this works, because mixing text tokens with position tokens seems to cause a conflict, but it doesn't in practice. The model will learn to use the embedding vector for both, maybe specialising a part for semantics and another for position. The second question. Neural nets find a way to differentiate the keys from queries by simply doing gradient descent. If we tell the model it should generate a specific token here, then it needs to fix the keys and queries to make it happen. The architecture is pretty dumb, the secret is the training data - everything the transformer learns comes from the training set. We should think about the training data when we marvel at what transformers can do. The architecture doesn't tell us why they work so well.
- FartyMcFarter 3y ago
  > The architecture is pretty dumb, the secret is the training data If this were true, we could throw the same training data at any other "dumb" architecture and it would learn language at least as well/fast as transformers do. But we don't see that happening, so the architecture must be smartly designed for this purpose.

kk58 3y ago

In CNN the layers seem to learn geometric primitive and deeper layers seem to learn more complex geometric patterns loosely speaking. In transformer what do query key matrices learn? How are their weights somehow working to extract context no matter which word appears in which position?

visarga 3y ago
The transformer doesn't have the nice pyramid shape of CNNs, but it still needs multiple layers. There have been papers showing non-trivial interactions between successive layers, forming more complex circuits. https://transformer-circuits.pub/2021/framework/index.html https://transformer-circuits.pub/2021/framework/index.html (warning, advanced difficulty) The Q an K matrices learn how to relate tokens. Each of the heads will learn to extract a different relation. For example, one will link to the next token, another will link pronouns to their references, another would be matching brackets, etc. Check out the cute diagrams here: https://www.arxiv-vanity.com/papers/1904.02679/ https://www.arxiv-vanity.com/papers/1904.02679/ So each head (Q and K pair) is like a program doing a specific pattern of lookup.
- kk58 3y ago
  Agree. So attention is like hierarchy of graphs where nodes are tokens and edges are attention scores per head. Now what's trippy is this node has position data. So node feature and position it appears is used to create a operator that projects a sequence to a semantic space. This seems to work for any modality of data.. so there is some thing about order in which data appears that seems to be linked to semantics and for me hints about some deep causal structure being latent in LLM

qumpis 3y ago

King-man got me thinking about the VAE paper for a second :)