3 ms·

Based on my personal experience I think that's a much better approach, so I wish you luck with it. Maybe somewhat counter-intuitively to how most people view L

Based on my personal experience I think that's a much better approach, so I wish you luck with it. Maybe somewhat counter-intuitively to how most people view LLMs, I strongly believe they're better when you constrain them a bit with some guardrails (E.g. pieces of a query, a bunch of existing queries, etc). Happily surprised you guys managed to get it down to only a 7% failure rate though! For how temperamental LLMs are and the seeming complexity of the task that's impressive.

phillipcarter 3y ago
> Happily surprised you guys managed to get it down to only a 7% failure rate though! Thanks! It, uhh, was quite a bit higher before we did some of that work though, heh. Since we can take a query and attempt to run it, we get good errors for anything that's ill-specified, and we can track it. Ideally we'd address everything with better prompt engineering, but it's certainly quicker to just fix stuff up after the fact when we know how to.
Der_Einzige 3y ago
Re: constraints, it turns out that banning tokens in a vocabulary is a great way to force models to be creative and follow syntactic or semantic constraints without errors: https://github.com/hellisotherpeople/constrained-text-generation-studio https://github.com/hellisotherpeople/constrained-text-genera...