3 ms·
The problem is the training set. If the training set contains something disruptive or of value, there is a nonzero percent chance it will be the next token. Box
by svaha1728 3y ago
The problem is the training set. If the training set contains something disruptive or of value, there is a nonzero percent chance it will be the next token. Boxes can attenuate that risk, but cannot set it to zero.
- dragonwriter 3y ago> If the training set contains something disruptive or of value, there is a nonzero percent chance it will be the next token. Boxes can attenuate that risk, but cannot set it to zero. I mean, if it was as simple as you describe ("nonzero percent chance that it will be next token"), then boxes could easily prevent it, the framework hosting the LLM can adjust logit bias on each token, so if it there was a readily-determinable forbidden-next-token, could prevent it with no problem. Unfortunately, the reality is...more complex.