4 ms·

> Is it just me or does anyone else cringe when they read "is/is not all you need" in the title of an AI-related paper? I believe it's a reference to 'attentio

> Is it just me or does anyone else cringe when they read "is/is not all you need" in the title of an AI-related paper? I believe it's a reference to 'attention is all you need,' which is a very famous paper now. I'm guessing you probably already know that -- did you cringe with it? It was a landmark paper so at least it's worth hyperbole in the title. Maybe I'm missing that its usage started before that. Reminds me of my ML prof who would complain about people using the word 'optimum' or 'optimized' in titles of papers... one could almost always optimize more, and with respect to what is not specified by the title.

haldujai 4y ago
Yes, aware of the reference and hence the cringe. 'Attention is all you need' is a clever play on words for a seminal paper whose title directly related to its main scientific contribution. Not sure that every other AI-related paper needs to copy this format, this trend is the academic equivalent of click-baiting (seemingly trying to associate with the Vaswani et al paper) and in my anecdotal experience the usage of this play on words seems inversely correlated with the paper's quality.
- earleybird 4y ago
  I believe it has an analogue with Heinlein's "there are two kinds of jokes; funny always and funny one time". Subsequent uses are a bit cringe in the same way that a subsequent telling of a funny-one-time joke is.
- ryanmcbride 4y ago
  Those kinds of references/cliches in titles always bug me too. Always feels lazy. I chalk this one up there with: Dr. Strangelove Titles - "<product>: or how I learned to stop <doing something> and love <idea>" Dairy Farmers of America reference - "Got <Product>?" Breakin' 2 reference - "<Product> 2, Electric Boogaloo" Honorable Mention, titles that end in phrases like "And That's a Good Thing!", "Here's What That Means", "This is Why That Matters"
  - hyperdimension 4y ago
    "Repeated Tropes Considered Harmful"
- nextaccountic 4y ago
  > 'Attention is all you need' is a clever play on words Can you elaborate? (I'm not a native speaker)
  - jrvarela56 4y ago
    I think the clever play on words is the fact that the title is cute and references the main finding directly: transformers have a mechanism called 'self-attention' that distinguishes them from previous approaches.
    - soulofmischief 4y ago
      Also likely referencing "Love is all you need", from The Beatles' https://en.wikipedia.org/wiki/All_You_Need_Is_Love https://en.wikipedia.org/wiki/All_You_Need_Is_Love
- wallscratch 4y ago
  I’m sorry, but what is the play on words in the title “Attention is all you need” here?
  - haldujai 4y ago
    "love is all you need" is the Beatles lyric they're referencing, the title could also be read as a statement on the basic human need for attention, like love, so it's a play on words for me. Do you prefer humorous allusion?
PartiallyTyped 4y ago
> Maybe I'm missing that its usage started before that. Afaik it wasn't used prior to it. While attention (and residual connections) were used with recurrent / auto regressive models, the paper showed that an encoder-decoder (non auto regressive and auto-regressive) architecture with just attention (and residuals) is sufficient to achieve great results given that you provide positional embeddings. So prior to transformers it was RNN + Attention (+ Residuals), but the paper claimed that Attention (+Residuals+Pos Embeddings) is all you need.
nmca 4y ago
It has been a wildly surprising fact over the last few years that attenion basically is all you need, in a sense much stronger than the one intended by the authors.

haldujai 4y ago

Yes, aware of the reference and hence the cringe. 'Attention is all you need' is a clever play on words for a seminal paper whose title directly related to its main scientific contribution. Not sure that every other AI-related paper needs to copy this format, this trend is the academic equivalent of click-baiting (seemingly trying to associate with the Vaswani et al paper) and in my anecdotal experience the usage of this play on words seems inversely correlated with the paper's quality.

earleybird 4y ago
I believe it has an analogue with Heinlein's "there are two kinds of jokes; funny always and funny one time". Subsequent uses are a bit cringe in the same way that a subsequent telling of a funny-one-time joke is.
ryanmcbride 4y ago
Those kinds of references/cliches in titles always bug me too. Always feels lazy. I chalk this one up there with: Dr. Strangelove Titles - "<product>: or how I learned to stop <doing something> and love <idea>" Dairy Farmers of America reference - "Got <Product>?" Breakin' 2 reference - "<Product> 2, Electric Boogaloo" Honorable Mention, titles that end in phrases like "And That's a Good Thing!", "Here's What That Means", "This is Why That Matters"
- hyperdimension 4y ago
  "Repeated Tropes Considered Harmful"
nextaccountic 4y ago
> 'Attention is all you need' is a clever play on words Can you elaborate? (I'm not a native speaker)
- jrvarela56 4y ago
  I think the clever play on words is the fact that the title is cute and references the main finding directly: transformers have a mechanism called 'self-attention' that distinguishes them from previous approaches.
  - soulofmischief 4y ago
    Also likely referencing "Love is all you need", from The Beatles' https://en.wikipedia.org/wiki/All_You_Need_Is_Love https://en.wikipedia.org/wiki/All_You_Need_Is_Love
wallscratch 4y ago
I’m sorry, but what is the play on words in the title “Attention is all you need” here?
- haldujai 4y ago
  "love is all you need" is the Beatles lyric they're referencing, the title could also be read as a statement on the basic human need for attention, like love, so it's a play on words for me. Do you prefer humorous allusion?

PartiallyTyped 4y ago

> Maybe I'm missing that its usage started before that. Afaik it wasn't used prior to it. While attention (and residual connections) were used with recurrent / auto regressive models, the paper showed that an encoder-decoder (non auto regressive and auto-regressive) architecture with just attention (and residuals) is sufficient to achieve great results given that you provide positional embeddings. So prior to transformers it was RNN + Attention (+ Residuals), but the paper claimed that Attention (+Residuals+Pos Embeddings) is all you need.

nmca 4y ago

It has been a wildly surprising fact over the last few years that attenion basically is all you need, in a sense much stronger than the one intended by the authors.