2 ms·
My understanding is that transformer model scale quadratically with the sequence length so it makes sense to encode your text into as few tokens as possible. I
by sebzim4500 4y ago
My understanding is that transformer model scale quadratically with the sequence length so it makes sense to encode your text into as few tokens as possible.
In any case, I don't understand why this is called 'Token-Free'. There are still tokens, there is just one for each character.
- earleybird 4y agoI expect it is analogous to scannerless parsing - there's still a scanner, it just recognizes individual characters. It may be more about where you draw the line and how important it is to your understanding (or ability to convey the idea).