7 ms·
As an author of this paper, I am very excited see the great discussion here! Several people mentioned the generation - compilation - fixing loop. Just want to
by LostBenjamin 1y ago
As an author of this paper, I am very excited see the great discussion here!
Several people mentioned the generation - compilation - fixing loop. Just want to remind you that our approach works for not only the generation step but also the fixing step. This is because fixing is essentially asking LLMs to generate a new version of the code. The paper actually has a "repair" experiment to demonstrate this and our approach achieves significant gain in this experiment, i.e., 37% relative improvement in functional correctness.
- tough 1y agoThank you for your research really impressive work!
- yewW0tm8 1y ago37% gain relative to what? What percent of generated functions were incorrect?
- LostBenjamin 1y agocompared to vanilla LLM decoding.