5 ms·

I do wonder whether people behind Attention is all you need paper Will receive Turing Award It is being cited often

by hardware2win 4y ago

mattcaldwell 4y ago
Came here expecting a Haiku.
- maxbond 4y ago
  The authors who wrote "Attention is all you need" - Turing candidates?
  - fastball 4y ago
    The people behind "Attention is all you need" Are often cited
    - andrelaszlo 4y ago
      Attention. Attention. Attention. - Ikkyū
- qwertyforce 4y ago
  Neural nets advance, Attention is all you need, Computing ascends. #by chatgpt
modeless 4y ago
The Adam optimizer is another possibility. It's unbelievably good and everyone uses it.
RC_ITR 4y ago
>Will receive Turing Award This is the weird thing - hopefully not! Hopefully there's even better NN models coming out every 5-10 years and we look back on transformers as 'just a phase' sort of like how we look back at RNN's (which were no less of an amazing achievement - look at the proliferation of voice assistants), as potentially obsolete technology today. Fore example, attention is great and does a really good job of simulating context in language, but what if we come up with a clever way to simulate symbology? Then we actually are back on the path to AGI and transformers will look like child's play.
- Beldin 4y ago
  > symbology Off-topic, but now I have William Dafoe going "What's the 'symbology' here? The symbolism ..." in my head (from Boondock Saints).
  - Gee101 4y ago
    Even thou I watched that movie 20 years ago. I will never forget that scene.
seydor 4y ago
I remember an interview with one of the founders of openAI, saying that if it wasn't the transformer architecture it would be something else. What really matters is the scale of the model. The transformer is only one of the possible configurations that work well with text. It seems they stuck to it because it is really so good so why break things.
albertzeyer 4y ago
The authors did not really expect it to be such a huge influence. You could also argue, it is a somewhat natural next step. This paper did not invent self-attention nor attention. Attention was already very popular, specifically for machine translation, and a few other papers already did use self-attention at that point in time. It was just the first paper which solely used attention and self-attention and nothing else.
mirekrusin 4y ago
Guy who said - “I don’t understand all of this, can we just throw more machines?” should get the award.
PartiallyTyped 4y ago
Attention existed before that paper and was incorporated to LSTMs until that point in time.

I do wonder whether people behind Attention is all you need paper Will receive Turing Award It is being cited often

mattcaldwell 4y ago

Came here expecting a Haiku.

maxbond 4y ago
The authors who wrote "Attention is all you need" - Turing candidates?
- fastball 4y ago
  The people behind "Attention is all you need" Are often cited
  - andrelaszlo 4y ago
    Attention. Attention. Attention. - Ikkyū
qwertyforce 4y ago
Neural nets advance, Attention is all you need, Computing ascends. #by chatgpt

modeless 4y ago

The Adam optimizer is another possibility. It's unbelievably good and everyone uses it.

RC_ITR 4y ago

>Will receive Turing Award This is the weird thing - hopefully not! Hopefully there's even better NN models coming out every 5-10 years and we look back on transformers as 'just a phase' sort of like how we look back at RNN's (which were no less of an amazing achievement - look at the proliferation of voice assistants), as potentially obsolete technology today. Fore example, attention is great and does a really good job of simulating context in language, but what if we come up with a clever way to simulate symbology? Then we actually are back on the path to AGI and transformers will look like child's play.

Beldin 4y ago
> symbology Off-topic, but now I have William Dafoe going "What's the 'symbology' here? The symbolism ..." in my head (from Boondock Saints).
- Gee101 4y ago
  Even thou I watched that movie 20 years ago. I will never forget that scene.

seydor 4y ago

I remember an interview with one of the founders of openAI, saying that if it wasn't the transformer architecture it would be something else. What really matters is the scale of the model. The transformer is only one of the possible configurations that work well with text. It seems they stuck to it because it is really so good so why break things.

albertzeyer 4y ago

The authors did not really expect it to be such a huge influence. You could also argue, it is a somewhat natural next step. This paper did not invent self-attention nor attention. Attention was already very popular, specifically for machine translation, and a few other papers already did use self-attention at that point in time. It was just the first paper which solely used attention and self-attention and nothing else.

mirekrusin 4y ago

Guy who said - “I don’t understand all of this, can we just throw more machines?” should get the award.

PartiallyTyped 4y ago

Attention existed before that paper and was incorporated to LSTMs until that point in time.