4 ms·
Table 4's BERT/Semantic tokenizer examples are full of word splittings that only a chemist could love. The BERT/WordPiece splittings make more semantic sense to
by superkuh 4y ago
Table 4's BERT/Semantic tokenizer examples are full of word splittings that only a chemist could love. The BERT/WordPiece splittings make more semantic sense to me.
- sp332 4y agoThe problem is that each prefix gets a separate token, so the LLM has to learn from context that they refer to the same thing. In the semantic encoding, all of the words start with the same token, so the LLM can learn that the words are related with a lot less data.
- thomastjeffery 4y agoYour paragraph told me so much more than the abstract in the OP.