3 ms·
It doesn't help with repetition, but when it comes to force structure on the output data, this approach looks interesting: https://github.com/newhouseb/clownfi
by int_19h 4y ago
It doesn't help with repetition, but when it comes to force structure on the output data, this approach looks interesting:
https://github.com/newhouseb/clownfish https://github.com/newhouseb/clownfish
TL;DR: it exploits the fact that the model returns probabilities for all the possible following tokens to enforce a JSON schema on the output as it is produced, backtracking as needed.