5 ms·
> You can deterministically just take the highest score or you can randomly sample in a way that makes it more likely to get a token with a higher score, accord
by madamelic 3y ago
> You can deterministically just take the highest score or you can randomly sample in a way that makes it more likely to get a token with a higher score, according to different schemes.
Ah ok, so it's like the difference between greedily always taking the shortest / highest value vs using A* and exploring alternatives in hopes of finding something better? That probably doesn't line up as a metaphor as it isn't "exploring" or running multiple parallel generations.
And it employs the method in the layers so that the benefit can be amplified rather than only employing the method in the final layer / output when it selects the output to take
- doctorpangloss 3y agoThe difference is between something that makes sense (the way the model is intended to be used) and something that makes almost no sense (this repository). > benefit In an intellectually honest way, it is not conclusive if there is a benefit to the approach in this repository. However, as a betting man: there is no benefit. There are a lot of surprises! Riffusion and LCMs for me were super surprising. With no examples or meaningful investigation, I don’t think this is going to yield anything.
- MacsHeadroom 3y agoIt's funny that you think the way transformer models are "intended" to be used is the result of intentional engineering and not an accident that ML academics are no closer to giving some causal explanation for why it works today than they were when Attention is All You Need was published.