3 ms·
Often the smartest thing is to do nothing.
by drittich 1mo ago
Often the smartest thing is to do nothing.
- phatfish 1mo agoOr know when to shut up. A tangent, but can anyone ELI5 how models "know" when to stop generating tokens? Or what the method to stop them at the right point is?
- bigfudge 1mo agoI might be out of date but my understanding was that STOP was just another token that gets predicted.
- bulder 1mo agoThe model doesn't "know" how to generate tokens any more than it knows how to stop generating tokens. The sampler simply stops pulling values when it outputs a "stop token", which is a token the same way every other token is. That is to say, it stops when it's statistically the most likely to.
- phatfish 1mo agoOK thanks, so the neural net (that no one can explain fully) generates a "stop" signal at a certain point.