10 ms·
What does the x_y signify? And what's going on with 0_0?? From my (very limited) understanding a fixed point/cycle like that should be vanishingly unlikely.
by andreareina 6y ago
What does the x_y signify? And what's going on with 0_0?? From my (very limited) understanding a fixed point/cycle like that should be vanishingly unlikely.
- minimaxir 6y agoThose are the temperatures (0.0, 0.7, etc.) that control the craziness. 0.0 means the model makes the optimal guess for each token. Unfortunately, both GPT-2 and GPT-3 have a tendency to enter loops. But it's not always bad: https://github.com/minimaxir/gpt-3-experiments/blob/master/examples/revengeofthesith/output_0_0.md https://github.com/minimaxir/gpt-3-experiments/blob/master/e...
- andreareina 6y agoThanks, didn't twig onto the fact that you linked a subtree of the whole repo. Weird that even with the nonzero temp the AskReddit prompt went a bit loopy. > https://github.com/minimaxir/gpt-3-experiments/blob/master/examples/revengeofthesith/output_0_0.md https://github.com/minimaxir/gpt-3-experiments/blob/master/e... Oh my goodness that is absurd in the most delightful way. Thanks for sharing that.