3 ms·
Is the " an" token the only way GPT-2 will ever produce the string " an"? Or will it sometimes combine separate " a" "n" tokens? I suppose separate tokens like
by johntb86 4y ago
Is the " an" token the only way GPT-2 will ever produce the string " an"? Or will it sometimes combine separate " a" "n" tokens? I suppose separate tokens like that won't be seen in the input string, so they'll never be predicted?
- sebzim4500 4y agoYeah, tokens exist for [ a] and [n] but during training it would still be considered incorrect to produce [ a][n] rather than [ an], so the model will never learn to output them.