3 ms·
No, the way it works is that the current output + potential next tokens to be sampled are checked with the grammar. All potential tokens that don't match are re
by SCHiM 3y ago
No, the way it works is that the current output + potential next tokens to be sampled are checked with the grammar. All potential tokens that don't match are removed. Then, with the list of valid tokens left, normal sampling strategies are used.