4 ms·
I thought generating text from language models was a deterministic operation, searching for the maximum likelihood sequence using beam search?
by blackbear_ 6y ago
I thought generating text from language models was a deterministic operation, searching for the maximum likelihood sequence using beam search?
- ansk 6y agoIt can be as deterministic as you want it to be -- there are parameters that control how much randomness is used during the sampling process. Finding the most probable sequence from the learned distribution is intractable for all but the shortest of sequences. As you've said, beam search is used, but this is just a local search heuristic and provides no guarantees of producing the most probable output.
- make3 6y agoIt is now common knowledge in the NLP community that beam search only works for situations where the output space is very constrained, specifically, in neural machine translation. In more open ended generation such as summarization, question answering and story generation, beam search leads to poor and repetitive outputs. Different (stochastic) sampling methods lead to more interesting, diverse and .. functional outputs.
- FL33TW00D 6y agoGreat blog post here exploring the different methods of generation: https://huggingface.co/blog/how-to-generate https://huggingface.co/blog/how-to-generate