4 ms·

I still can not understand the name of "attention is what you need" study.

• yorwba 1y ago
  The "attention is all you need" paper did not invent attention mechanisms. It showed that existing models that were already using attention could have their non-attention parts removed and still worked. So those other parts were unnecessary and only attention was needed.