4 ms·

> A better n-gram model would calculate the probability of an n-gram following another n-gram according to their frequency in the training corpus. With a larger

by shawnz 3y ago

> A better n-gram model would calculate the probability of an n-gram following another n-gram according to their frequency in the training corpus. With a larger corpus and better training techniques (some smoothing for one thing) you'd get much more coherent output. I don't understand what you mean here, isn't that exactly what an n-gram model is already designed to achieve where n > 1? Since this is a 2-gram model isn't it already doing this?

YeGoblynQueenne 3y ago
That's right, but a bi-gram model is just a big table of bi-grams and their probabilities (which are really their frequencies in a corpus normalised over the number of all bigrams in the corpus). You can use those probabilities in two ways: to calculate the probability of a string, by taking the bigrams in the string and multiplying their probabilities together; or by starting with some bigram (e.g. a search term entered by a user) and then selecting the next bigram according to its probability (i.e. its frequency in the corpus), or according to the probability of the entire string so-far (which you get by multiplying the probabilities of the bigrams so far); and repeat until you reach some maximum string-length. So you're right that my comment is a bit confusing: I refer to the way you sample from a bi-gram model, so as to generate a string. Sorry!