4 ms·
It’s hard to be disciplined about a black box though. That’s one reason why we’re all speeding off at a thousand miles per hour on transformers - the architectu
by jahewson 2y ago
It’s hard to be disciplined about a black box though. That’s one reason why we’re all speeding off at a thousand miles per hour on transformers - the architecture works, why try other things?
- exe34 2y agowhoever comes up with the next trick could win big.
- jack_pp 2y agoCould, or meta makes you irrelevant with by reproducing your trick and giving it away for free
- kranke155 2y agoYann LeCun certainly seems to be one of the more interesting people in the space. He’s a notable skeptic of Llm intelligence and incredibly smart.
- exe34 2y agoTo be fair I don't think anybody saw the boom in LLMs coming from the initial Attention paper. At the time it was one of many ideas that sounded like they had potential.
- lallysingh 2y agoWhoever chooses the next winning lottery numbers could win big...
- exe34 2y agothat's kinda my point, we don't know what the next winning move will be.
- jumpCastle 2y agoAttention was invented because Bengio lab had to be disciplined about a black box (google had more compute)