3 ms·
I think the best way is to get a basic understanding of torch and then just reading what the code does. The model internals are hard (though for the transformer
by bootsmann 4y ago
I think the best way is to get a basic understanding of torch and then just reading what the code does. The model internals are hard (though for the transformer there are good step by steps online) but for this case it is basically playing with the output of the actual neural net inside the model. The net gives you a probability distribution to be next over all tokens (think words). Now it has proven to be useful not to pick the token with the highest probability but to employ different strategies. Top-k in this case means probabilistically sampling from the top k tokens instead of piking the best.