4 ms·

Attention is all you need.

echelon 1y ago
In today's hyper saturated world, attention is everything: - consumer marketing - politics - venture fundraising When any system has a few power law winners, it makes sense to grab attention. Look at Trump and Musk and now Altman. They figured it out. MrBeast... Attention, even if negative, wedges you into the system and everyone's awareness. Your mousey quiet competitors aren't even seen or acknowledged. The attention grabbers suck all the oxygen out of the room and win. If you go back and look at any victory, was it really better solutions, or was it the fact that better solutions led to more attention? "Look here" -> build consensus and ignore naysayers -> keep building -> feedback loop -> win It might not just be a societal algorithm. It might be one of the universe's fundamental greedy optimization algorithms. It might underpin lots of systems, including how we ourselves as individuals think and learn. Our pain receptors. Our own intellectual interests and hobbies. Children learning on the playground. Ant colonies. Bee swarms. The world is full of signals, and there are mechanisms which focus us on the right stimuli.
- peterlk 1y ago
  You’re absolutely right!
- ghurtado 1y ago
  Something flew approximately 10 miles above your head that would be a good idea for you to learn.
  - scubbo 1y ago
    There were plenty of kinder ways to let someone know that they had missed a reference - https://xkcd.com/1053/ https://xkcd.com/1053/
  - echelon 1y ago
    What makes you think I didn't know the reference? That paper is seminal and essential reading in this space. The intent was for you to read my comment at face value. I have a point tangential to the discussion at hand that is additive.
- lawlessone 1y ago
  Is this copypasted from LinkedIn?
  - kitsune1 1y ago
    [dead]
  - echelon 1y ago
    If you traverse back the fourteen years of my comment history (on this account - my other account is older), you'll find that I've always written prose in this form. LLMs trained on me (and the Hacker News corpus), not the other way around.
- alganet 1y ago
  You're not accounting for substrate saturation. If you could just spam annoy until you win, we'd be all dancing to remixed versions of Macarena.
dormento 1y ago
In case anyone missed the reference: https://arxiv.org/abs/1706.03762 https://arxiv.org/abs/1706.03762 > (...) We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely.

In today's hyper saturated world, attention is everything: - consumer marketing - politics - venture fundraising When any system has a few power law winners, it makes sense to grab attention. Look at Trump and Musk and now Altman. They figured it out. MrBeast... Attention, even if negative, wedges you into the system and everyone's awareness. Your mousey quiet competitors aren't even seen or acknowledged. The attention grabbers suck all the oxygen out of the room and win. If you go back and look at any victory, was it really better solutions, or was it the fact that better solutions led to more attention? "Look here" -> build consensus and ignore naysayers -> keep building -> feedback loop -> win It might not just be a societal algorithm. It might be one of the universe's fundamental greedy optimization algorithms. It might underpin lots of systems, including how we ourselves as individuals think and learn. Our pain receptors. Our own intellectual interests and hobbies. Children learning on the playground. Ant colonies. Bee swarms. The world is full of signals, and there are mechanisms which focus us on the right stimuli.

peterlk 1y ago
You’re absolutely right!
ghurtado 1y ago
Something flew approximately 10 miles above your head that would be a good idea for you to learn.
- scubbo 1y ago
  There were plenty of kinder ways to let someone know that they had missed a reference - https://xkcd.com/1053/ https://xkcd.com/1053/
- echelon 1y ago
  What makes you think I didn't know the reference? That paper is seminal and essential reading in this space. The intent was for you to read my comment at face value. I have a point tangential to the discussion at hand that is additive.
lawlessone 1y ago
Is this copypasted from LinkedIn?
- kitsune1 1y ago
  [dead]
- echelon 1y ago
  If you traverse back the fourteen years of my comment history (on this account - my other account is older), you'll find that I've always written prose in this form. LLMs trained on me (and the Hacker News corpus), not the other way around.
alganet 1y ago
You're not accounting for substrate saturation. If you could just spam annoy until you win, we'd be all dancing to remixed versions of Macarena.

dormento 1y ago

In case anyone missed the reference: https://arxiv.org/abs/1706.03762 https://arxiv.org/abs/1706.03762 > (...) We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely.