3 ms·
Not really, right? There are a ton of special tokens, like start of sequence etc., so what happens if there are two start of sequences predicted? It's a valid t
by SirSegWit 3y ago
Not really, right? There are a ton of special tokens, like start of sequence etc., so what happens if there are two start of sequences predicted? It's a valid token but cannot really be turned into something sensible, so it throws an error when converting tokens to plain text?
- minimaxir 3y agoSpecial tokens are handled by the application, not the model. They are still output before then.