4 ms·
So basically multiple CLS tokens. Fwiw, I tried multiple global tokens in my chess neural net and didn't see any uplift compared to my baseline of just having
by Scene_Cast2 1y ago
So basically multiple CLS tokens.
Fwiw, I tried multiple global tokens in my chess neural net and didn't see any uplift compared to my baseline of just having one.
- ninja3925 1y agoNote that it's not done for performance reason but rather to generate clear feature maps.