3 ms·
Hi, just wanted to say how much I appreciate your work. I'm curious if you have considered implementing Microsoft's Guidance (https://github.com/guidance-ai/gu
by mwieler 2y ago
Hi, just wanted to say how much I appreciate your work.
I'm curious if you have considered implementing Microsoft's Guidance (https://github.com/guidance-ai/guidance https://github.com/guidance-ai/guidance)? Their approach offers significant speed improvements, which I understand can sometimes be shortcoming of GBNF (e.g https://github.com/ggerganov/llama.cpp/issues/4218 https://github.com/ggerganov/llama.cpp/issues/4218).
- parthsareen 2y agoYes! I have checked guidance out, as well as a few others. Planning to refactor sampling in the near future which would include improving using grammars for sampling as well. Thanks for sharing!