4 ms·
The model returns probabilities across the full set of tokens. This restricts the tokens to those that conform to the grammar, and samples from those
by lyjackal 3y ago
The model returns probabilities across the full set of tokens. This restricts the tokens to those that conform to the grammar, and samples from those
- radarsat1 3y agoThis reminds me of something i did once using a 1d CRF at the output of a sequence model. I set "impossible" transitions to -inf so that transitions between certain state pairs were simply never predicted. I could imagine executing something like this that is conditional on the current grammatical context. On fact it's a bit surprising to me how little I see CRFs mentioned in the context of language models. They are useful whenever you want to model or learn transition probabilities.