I feel like the work on using CFGs with LLMs should be low-hanging fruit for improving code assistants, but perhaps someone more knowledgeable could chime in[1]

I feel like the work on using CFGs with LLMs should be low-hanging fruit for improving code assistants, but perhaps someone more knowledgeable could chime in[1], [2], [3]. At lot of the confabulations we see today - non-existent variables, imports and APIs, out-of-scope variables, etc - would seem (to me) to be meaningfully addressable with these techniques. Relatedly, I have gotten surprisingly great mileage out of treating confabulations, in the first instance, as a user (ie, my own) failure to adequately contextualise, rather than error. In a sense, CFGs give you sharper tools to do that contextualisation. I wonder how far the “sharper tools” approach will carry us. It seems, to this interested layman, consistent with Shannon’s work in statistical language modelling.[4] The term “prompt engineering” implies, to me, a focus on the instructive aspects of prompting, which is a bit unfortunate but also wholly consistent with the way I see colleagues and friends trying to interact with LLMs. Perhaps we should instead call it “context composition” or something, to emphasise the constructive nature. [1] https://github.com/outlines-dev/outlines https://github.com/outlines-dev/outlines [2] https://github.com/ggerganov/llama.cpp/pull/1773 https://github.com/ggerganov/llama.cpp/pull/1773 [3] https://www.reddit.com/r/LocalLLaMA/comments/156gu8c/d_constraining_llama_models_with_context_free/ https://www.reddit.com/r/LocalLLaMA/comments/156gu8c/d_const... [4] https://hedgehogreview.com/issues/markets-and-the-good/articles/language-machinery https://hedgehogreview.com/issues/markets-and-the-good/artic...