3 ms·
The problem is that each prefix gets a separate token, so the LLM has to learn from context that they refer to the same thing. In the semantic encoding, all of
by sp332 4y ago
The problem is that each prefix gets a separate token, so the LLM has to learn from context that they refer to the same thing. In the semantic encoding, all of the words start with the same token, so the LLM can learn that the words are related with a lot
less data.
- thomastjeffery 4y agoYour paragraph told me so much more than the abstract in the OP.