3 ms·
This is a good call-out, and one that might be fixable by setting a repetition penalty in generation — or better in training.
by deepsquirrelnet 1y ago
This is a good call-out, and one that might be fixable by setting a repetition penalty in generation — or better in training.
- w10-1 1y agoI recall in 1980's, style-analyzing software reporting word counts for words with high frequency in the text but low frequency in general usage - jargon. The suggestion was to re-word, and it was often a relevant clue to re-think by examining why one word was doing so much work. Detecting that sounds like more than repetition, but still feasible probabilistically and relevant at both usage and conceptual levels.