Super interesting! Since you're generating code with LLMs, you should check out this paper: https://arxiv.org/pdf/2405.15793 https://arxiv.org/pdf/2405.15793

Super interesting! Since you're generating code with LLMs, you should check out this paper: https://arxiv.org/pdf/2405.15793 https://arxiv.org/pdf/2405.15793 It uses smart feedback to fix the code when LLMs occasionally do hiccups with the code. You could also have a "supervisor LLM" that asserts that the resulting code matches the specification, and gives feedback if it doesn't.