3 ms·
> each token needs to "look at" or "attend to" all other tokens to understand the context. > First token: Look at 1 token (cost: O(1^2)) Umm, is this right? T
by yellow_lead 2y ago
> each token needs to "look at" or "attend to" all other tokens to understand the context.
> First token: Look at 1 token (cost: O(1^2))
Umm, is this right? There is not 1 token existing before generating the first token, so how do you look at it? AI slop?
- Vampiero 2y ago> Umm, is this right? No way to know until you painstakingly verify every single assertion that the AI made! The author of this article certainly didn't, and the content was good enough to them. Trust me, AGI is almost there.
- t55 2y agoI did :) Yes, 80% of the post is generated, yet, I still reviewed everything! And if I had written everything from scratch, this would have probably taken a week rather than a few hours.
- t55 2y agoThe phrase, “the first token looks at 1 token,” is simply a shorthand for the self-attention step when the sequence length is one. Although there are no preceding tokens, we still treat it as an O(1^2) operation where the first token effectively attends to itself (or a special [BOS] token). This approach preserves the big-O analysis when summing over all tokens.