4 ms·
If you don't want the AI to use these words, why bother training it? Just mask those logits before sampling.
by Imnimo 4y ago
If you don't want the AI to use these words, why bother training it? Just mask those logits before sampling.
- bootsmann 4y agoI is a 1 letter token for gpt-2 (like every other letter), don't think its a lot different for gpt-3, so this is kinda hard as the transformer will want to use the token in big built words.