It's interesting watching these attempts to understand Transformers. Are they Graph networks? Are they Hopfields networks? Are they convolutions? There was

It's interesting watching these attempts to understand Transformers. Are they Graph networks? Are they Hopfields networks? Are they convolutions? There was this research: https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreti... Turns out, GPT-like architectures appear to use the same representation throughout all the layers. So you can use the final head layer as a lens to see what words the network is thinking of as it ... "thinks". That's a bit contrary to what I imagined a GPT-like architecture was doing. I would have assumed that its embedding of ideas changed throughout the network, only reaching a sensible embedding near the end. At least from the perspective of the head layer, that doesn't appear to be the case. Facebook's research demonstrates that the MLP layer can be dropped in favor of more attention over learned knowledge vectors: https://ai.facebook.com/blog/making-transformer-networks-simpler-and-more-efficient/ https://ai.facebook.com/blog/making-transformer-networks-sim... > Reading new Transformer papers makes me feel that training these models requires something akin to black magic when determining the best learning rate schedule, warmup strategy and decay settings. I found GPT-*'s schedule straightforward. In fact, most training schedules these days are "boring". Adam with warmup and either a linear or cosine decay. CNNs have been doing that for awhile, and the hyperparameters now are fairly robust. If you get within an order of magnitude you'll land within a few percent of optimal accuracy. The OpenAI paper Scaling Laws for Neural Language Models does a good job of exploring the hyperparameters of GPT-like networks. It's a fascinating read. That paper suggests another thing about Transformers that we don't understand. Beyond some minimums, the layer count, embedding size, and number of attention heads _isn't important_. The most important factor in the performance of a model is simply the number of parameters. That's quite unusual, as the classic intuition is that adding more layers to a model improves performance. Yet for GPT-like architectures that isn't the case. You can get the same performance gains by just increasing the embedding size. 4 heads? 12 heads? Doesn't matter. Weird. > Initially introduced for machine translation, Transformers have gradually replaced RNNs in mainstream NLP. The architecture takes a fresh approach to representation learning: Doing away with recurrence entirely The more I study Transformers, the more I suspect that their success has more to do with our utter failure to train RNNs. In theory, RNNs have infinite attention. In practice, the only tool we have for optimizing models is backprop, and so to train an RNN we have to use BPTT. This defacto creates a learning horizon. There is absolutely no training signal that tells an RNN to remember something beyond the BPTT horizon. So why would it? Our training loop for RNNs involves giving them, for example, 1024 tokens, running them for 1024 iterations, and computing loss on 1024 predictions. A Transformer's training is nearly identical. They get 1024 tokens of context, make 1024 predictions, and compute loss. The difference? Let's consider a scenario where it's the last prediction, and that prediction should be a copy of the first word in the sequence. Very simple, right? Yet for an RNN to learn how to do that, it needs to backprop the loss through 1024 iterations of its model. Even for a simple 1 layer RNN that makes it look like a 1024 layer model. Vanishing gradients turned up to 11!! For a Transformer this is dead simple. Its attention mechanism allows the last column of the Transformer to directly use knowledge from the entire sequence. Not only can it see the first token in the sequence, it can see all the computations it previously performed on the sequence, all at once. All with easy backprop. No vanishing gradients here. So is it really any surprise that Transformers have dominated RNNs? Given the exact same compute, memory, parameters, etc Transformers make much more efficient use of the resources. (During training) To put it another way, a Transformer is exactly like an RNN. It just uses attention to access history rather than recurrence. And we know that backprop and recurrence are incompatible. So Transformers win. Of course, that's a huge problem. We have a temporary win. A HUGE win. But Transformers haven't solved what RNNs were meant to solve. By the end of a book, human brains can remember things from the beginning of the book. RNNs trained with BPTT cannot. Transformers cannot. Even Transformers with linear attention mechanisms cannot. There's another leap here left to do. We need some kind of long term memory mechanism. A big Transfomer model can probably approximate the active parts of a human brain. The image-gpt model and papers using Transformers in place of CNNs for classification tasks have shown that Transformers are a generic substrate. But the big missing puzzle piece is long term memory. Give a Transformer some method to query a bank of memory and I think we're going to see that next big leap. Plus, with access to a memory bank, a Transformer can free up all the resources its using today to behave _like_ long term memory and instead use them for more "thinking". Yet we have absolutely no mechanisms available to us today to build such a thing. To teach any kind of model to act as long term memory requires some way to show to an example from its distant past, see how well it remembers it, and then ... backprop that. But we can't backprop it, because we can't backprop across an entire book, let alone WikiText or WebText2. We're really back at square one. We need some way to make the theory of RNNs a reality. Transforms haven't bought us that. They've only bought us a short-term cheat in performance. EDIT: Just so it's clear, none of my comment is meant as a criticism of the linked article. I actually thought the OP was great and sources a lot of research in trying to understand the mechanisms of Transformers. Really I just springboarded off the article to dump some of my own musings.