2 ms·
Does this work in tandem with beam search or does it do greedy sampling?
by popinman322 3y ago
Does this work in tandem with beam search or does it do greedy sampling?
- btwillard 3y agoThe underlying approach can improve the performance of anything that requires the set of non-zero probability tokens at each step, and anything that needs to continue matching/parsing from a previous state.