21 ms·
? theyre not special tokens really
by whimsicalism 2y ago
? theyre not special tokens really
- jebarker 2y agoi'm not actually sure whether they're special tokens in the sense of being in the vocabulary
- whimsicalism 2y ago<think> might be i think "wait" is tokenized like any other in the pretraining