5 ms·

This is confusing, using the semantic vectors arithmetic of embeddings is not very relevant to transformers and its completely missing the word 'attention'. I

by seydor 4y ago

This is confusing, using the semantic vectors arithmetic of embeddings is not very relevant to transformers and its completely missing the word 'attention'. I don't think transformers are that difficult to explain to people , but it is hard to explain "why" they work. But i think it's important for everyone to look under the hood and know that there are no demons underneath.

Analog24 4y ago
Embeddings and their relationship to each other are definitely relevant to transformers. Why do you think that's not the case?
- seydor 4y ago
  gptX embeddings aren't even words. Even so, the embedding relationship is useful but not the core of what transformers do to find relationships between words in sequences.
  - gcr 4y ago
    remember the word2vec paper? the surprising bit the authors were trying to show was that putting words in some embedding space with an appropriate loss naturally lends enough structure to those words to be able to draw robust, human-interpretable analogies. I agree with the sentiment that each individual dimension isn't meaningful, and I also feel like it's misleading for the article to frame it that way. But there's a grain of truth: the last step to predicting the output token is to take the dot product between some embedding and all the possible tokens' embeddings (we can interpret the last layer as just a table of token embeddings). Taking dot products in this space are equivalent to comparing the "distance" between the model's proposal and each possible output token. In that space, words like "apple" and "banana" are closer together than they are to "rotisserie chicken," so there is some coarse structure there. Doing this, we gave the space meaning by the fact that cosine similarity is meaningful proxy for semantic similarity. Individual dimensions aren't meaningful, but distance in this space is. A stronger article would attempt to replicate the word2vec analogy experiments (imo one of the more fascinating parts of that paper) with GPT's embeddings. I'd love to see if that property holds.
    - HarHarVeryFunny 4y ago
      I wouldn't say the interpretability of word2vec embeddings is suprising - it's just a reflection of words being defined by context/usage, and these embeddings being created based on that assumption.
  - Analog24 4y ago
    You can't just wave your hand and tell someone that words are broken up into sub-word tokens that are then transformed into a numerical representation to feed to a transformer and expect people to understand what is happening. How is anyone supposed to understand what a transformer does without understanding what the actual inputs are (e.g. word embeddings)? Plus, those embeddings directly related to the self attention scores calculated in the transformer. Understanding what an embedding is is extremely relevant.
nitnelave 4y ago
I was trying to keep the article at a level that everyone understands, from middle school up. I thought about going a bit deeper in the structure and mentioning attention, but my problem is that the intuitive concept of "attention" is quite different from the mathematical reality of an attention layer, and I'm sure I would have lost quite a few people there. It's always a trade-off :)
- seydor 4y ago
  Indeed the word 'attention' implies some active process, which is also unfortunate, because it is really similarity maps over long ranges. This process apparenty creates a cross-reference maps of items that are commonly co-activated. But then what do the different heads do? Some kind of different "modalities of cross-reference"? I struggle to understand why this thing works the way it does. It's possible that Vaswani et al. have made one of the greatest discoveries of this century that solved the language problem in an unintuitive, and yet very unappreciated way. It's also possible that there are other architectures that can simulate the same level of intelligence with such large numbers of parameters. I think you re right that it's not intuitive, it's like basic arithmetic is laughing at us
  - fastball 4y ago
    Multi-head attention just means that you're looking at all the words at once rather than only looking at one word at a time, and using that to generate the next word. So instead of using attention only on the last word you also have attention on the penultimate word and the one before that and the one before that, etc. I think it is fairly obvious why this gives better results than say an RNN – you are utilizing context better than in a recurrent system, which is also just closer to how a human brain works. When you read/write a sentence you're not really going one word at a time, you're thinking about all the words at once, even if the last word is technically the most important. The other clear benefit of transformers over an arch like RNNs (and what has probably made more of a difference imo) is that its properly parallelizable, which means you can do huge training runs in a fraction of the time. RNNs might be able to get to a level of coherence that approaches GPT-3, but with current hardware that would be very time-prohibitive.
masswerk 4y ago
> that there are no demons underneath Given that Lacan already proposed the unconscious as structured language-like more than half a century ago and described attention in his turn on Freud's impulse in favor of his concept of derive, we may say, this is pretty much where our own demons live. (I actually do think that revisiting Lacan in this context may be productive.)
- thanatropism 4y ago
  We need to form some sort of guild of engineers who think Deleuze, Latour, Lacan et caterva should be read within our disciplines.
  - masswerk 4y ago
    I second this. The STEM (= engineering) / humanities divide in software may miss some. (There had been times, when linguistics were still a major entry path into computing, where things were a bit different. Notably, this were also the times, which gave rise to most of the general paradigms. A certain amount of generality was even regarded a prerequisite to programming. Particularly, HN is such a great place, because it holds up this notion of generality.)
  - robwwilliams 4y ago
    Please no! Read systems neuroscience. Like Hassabis does. Or if of a philosophical persuasion, then Dennett or Rorty.
    - masswerk 4y ago
      Much of cognitive science reinvents wheels that had been established in the 1920s and 1930s already, namely in sociology of knowledge and related fields. fRMI actually often confirms what had been already observed in a psychoanalytic context. (I don't think it's a good general advice to totally ignore what is already known.)
      - robwwilliams 4y ago
gcr 4y ago
Whether there are demons underneath depends on whether you believe a lossy statistical snapshot across the Jungian subconscious of all human writing contains demons. It's a bit more plausible when we phrase it that way...
dado3212 4y ago
Is there an article that would meet your standard (discusses transformers, uses the word 'attention') that you could give to someone who has no familiarity with computer science?
- seydor 4y ago
  Jay alammar's Illustrated transformer, although this too is detailed. I think it's still worth taking a look, because really i don't think that people have yet "compressed" what transformers do intuitively. None of the concepts of the networks are particularly hard math - it's basic algebra. But the overall construction is complicated. https://jalammar.github.io/illustrated-transformer/ https://jalammar.github.io/illustrated-transformer/
int_19h 4y ago
The whole thing is magic, really. We assemble relatively mundane things that we understand together in certain specific ways, and somehow get something that can talk etc. A conjurer of old - who would similarly assemble bells and candles - would probably say that we summoned a demon.

by seydor 4y ago

Analog24 4y ago

Embeddings and their relationship to each other are definitely relevant to transformers. Why do you think that's not the case?

seydor 4y ago
gptX embeddings aren't even words. Even so, the embedding relationship is useful but not the core of what transformers do to find relationships between words in sequences.
- gcr 4y ago
  remember the word2vec paper? the surprising bit the authors were trying to show was that putting words in some embedding space with an appropriate loss naturally lends enough structure to those words to be able to draw robust, human-interpretable analogies. I agree with the sentiment that each individual dimension isn't meaningful, and I also feel like it's misleading for the article to frame it that way. But there's a grain of truth: the last step to predicting the output token is to take the dot product between some embedding and all the possible tokens' embeddings (we can interpret the last layer as just a table of token embeddings). Taking dot products in this space are equivalent to comparing the "distance" between the model's proposal and each possible output token. In that space, words like "apple" and "banana" are closer together than they are to "rotisserie chicken," so there is some coarse structure there. Doing this, we gave the space meaning by the fact that cosine similarity is meaningful proxy for semantic similarity. Individual dimensions aren't meaningful, but distance in this space is. A stronger article would attempt to replicate the word2vec analogy experiments (imo one of the more fascinating parts of that paper) with GPT's embeddings. I'd love to see if that property holds.
  - HarHarVeryFunny 4y ago
    I wouldn't say the interpretability of word2vec embeddings is suprising - it's just a reflection of words being defined by context/usage, and these embeddings being created based on that assumption.
- Analog24 4y ago
  You can't just wave your hand and tell someone that words are broken up into sub-word tokens that are then transformed into a numerical representation to feed to a transformer and expect people to understand what is happening. How is anyone supposed to understand what a transformer does without understanding what the actual inputs are (e.g. word embeddings)? Plus, those embeddings directly related to the self attention scores calculated in the transformer. Understanding what an embedding is is extremely relevant.

nitnelave 4y ago

I was trying to keep the article at a level that everyone understands, from middle school up. I thought about going a bit deeper in the structure and mentioning attention, but my problem is that the intuitive concept of "attention" is quite different from the mathematical reality of an attention layer, and I'm sure I would have lost quite a few people there. It's always a trade-off :)

seydor 4y ago
Indeed the word 'attention' implies some active process, which is also unfortunate, because it is really similarity maps over long ranges. This process apparenty creates a cross-reference maps of items that are commonly co-activated. But then what do the different heads do? Some kind of different "modalities of cross-reference"? I struggle to understand why this thing works the way it does. It's possible that Vaswani et al. have made one of the greatest discoveries of this century that solved the language problem in an unintuitive, and yet very unappreciated way. It's also possible that there are other architectures that can simulate the same level of intelligence with such large numbers of parameters. I think you re right that it's not intuitive, it's like basic arithmetic is laughing at us
- fastball 4y ago
  Multi-head attention just means that you're looking at all the words at once rather than only looking at one word at a time, and using that to generate the next word. So instead of using attention only on the last word you also have attention on the penultimate word and the one before that and the one before that, etc. I think it is fairly obvious why this gives better results than say an RNN – you are utilizing context better than in a recurrent system, which is also just closer to how a human brain works. When you read/write a sentence you're not really going one word at a time, you're thinking about all the words at once, even if the last word is technically the most important. The other clear benefit of transformers over an arch like RNNs (and what has probably made more of a difference imo) is that its properly parallelizable, which means you can do huge training runs in a fraction of the time. RNNs might be able to get to a level of coherence that approaches GPT-3, but with current hardware that would be very time-prohibitive.

masswerk 4y ago

> that there are no demons underneath Given that Lacan already proposed the unconscious as structured language-like more than half a century ago and described attention in his turn on Freud's impulse in favor of his concept of derive, we may say, this is pretty much where our own demons live. (I actually do think that revisiting Lacan in this context may be productive.)

thanatropism 4y ago
We need to form some sort of guild of engineers who think Deleuze, Latour, Lacan et caterva should be read within our disciplines.
- masswerk 4y ago
  I second this. The STEM (= engineering) / humanities divide in software may miss some. (There had been times, when linguistics were still a major entry path into computing, where things were a bit different. Notably, this were also the times, which gave rise to most of the general paradigms. A certain amount of generality was even regarded a prerequisite to programming. Particularly, HN is such a great place, because it holds up this notion of generality.)
- robwwilliams 4y ago
  Please no! Read systems neuroscience. Like Hassabis does. Or if of a philosophical persuasion, then Dennett or Rorty.
  - masswerk 4y ago
    Much of cognitive science reinvents wheels that had been established in the 1920s and 1930s already, namely in sociology of knowledge and related fields. fRMI actually often confirms what had been already observed in a psychoanalytic context. (I don't think it's a good general advice to totally ignore what is already known.)
    - robwwilliams 4y ago

gcr 4y ago

Whether there are demons underneath depends on whether you believe a lossy statistical snapshot across the Jungian subconscious of all human writing contains demons. It's a bit more plausible when we phrase it that way...

dado3212 4y ago

Is there an article that would meet your standard (discusses transformers, uses the word 'attention') that you could give to someone who has no familiarity with computer science?

seydor 4y ago
Jay alammar's Illustrated transformer, although this too is detailed. I think it's still worth taking a look, because really i don't think that people have yet "compressed" what transformers do intuitively. None of the concepts of the networks are particularly hard math - it's basic algebra. But the overall construction is complicated. https://jalammar.github.io/illustrated-transformer/ https://jalammar.github.io/illustrated-transformer/

int_19h 4y ago

The whole thing is magic, really. We assemble relatively mundane things that we understand together in certain specific ways, and somehow get something that can talk etc. A conjurer of old - who would similarly assemble bells and candles - would probably say that we summoned a demon.