4 ms·
Depending on what sort of constraints you need on your output, a custom token sampler, logit bias, or verifying it against a grammar could do the trick.
by potatoman22 2y ago
Depending on what sort of constraints you need on your output, a custom token sampler, logit bias, or verifying it against a grammar could do the trick.