3 ms·
There has been a huge explosion of awesome tooling which utilizes constrained text generation. Awhile ago, I tried my own hand at constraining the output of LL
by Der_Einzige 3y ago
There has been a huge explosion of awesome tooling which utilizes constrained text generation.
Awhile ago, I tried my own hand at constraining the output of LLMs. I'm actively working on this to make it better, especially with the lessons learned from repos like this and from guidance
https://github.com/hellisotherpeople/constrained-text-generation-studio https://github.com/hellisotherpeople/constrained-text-genera...
- rain1 3y agoThis looks incredible. Wow.
- killthebuddha 3y agoI agree, it looks great. A couple similar projects you might find interesting: - https://github.com/newhouseb/clownfish https://github.com/newhouseb/clownfish - https://github.com/r2d4/rellm https://github.com/r2d4/rellm The first one is JSON only and the second one uses regular expressions, but they both take the same "logit masking" approach as the project GP linked to.
- Der_Einzige 3y agoI love the love from you two - I am trying right now to significantly improve CTGS. I'm not actually using the "Logitsprocessor" from Huggingface, and I really ought to as it will massively speed up inference performance. Unfortunately, fixing up my current code to work with that will take quite awhile. I've started working on it but I am extremely busy these days and would really love for other smart people to help me on this project. If not here, I really want proper access to the constraints APIs (LogitsProcessor and the Constraints classes in Huggingface) in the big webUIs for LLMs like oogabooga. I'd love to make that an extension. I'm also upset at the "undertooling" in the world of LLM prompting. I wrote a snarky blog post about this: https://gist.github.com/Hellisotherpeople/45c619ee22aac6865ca4bb328eb58faf https://gist.github.com/Hellisotherpeople/45c619ee22aac6865c...