3 ms·
> In his dataset, all sentences start at the beginning of the line and end at it's end. Right, but the sentence start is taken from the first word of a random
by jstanley 7y ago
> In his dataset, all sentences start at the beginning of the line and end at it's end.
Right, but the sentence start is taken from the first word of a random state transition pair, not the first word of a random sentence.
- make3 7y agoThe first random state should be picked with start probability, which are equal to the frequency of a word beginning a sentence, which is exactly equal to picking the first word of a random sentence