3 ms·
At least it didn’t go into an infinite loop. Maybe there’s explicit logic to prevent that.
by layer8 4y ago
At least it didn’t go into an infinite loop. Maybe there’s explicit logic to prevent that.
- keepquestioning 4y agoWhat are thresholds used in generative text to stop generating characters?
- layer8 4y agoMust be a bit more sophisticated than that, as it forms a complete sentence.
- ml_basics 4y agoI'm not sure about GPT-3 specifically, but often these models are trained with an explicit "stop token", which the model can output when generating text to indicate that no more text should be produced. That said, there's no reason it couldn't go into an infinite loop, just that the models are pretty sophisticated and therefore are less likely to do this than earlier generations of models (like predictive text on smart phones in the past)
- stavros 4y agoThere's a penalty for generating text it's generated before (two penalties, actually, one for words and one for phrases).