3 ms·
That's not what tokenized means here. Parent is asking to provide the model with separate characters rather than tokens, i.e. groups of characters.
by viraptor 2y ago
That's not what tokenized means here. Parent is asking to provide the model with separate characters rather than tokens, i.e. groups of characters.