5 ms·
I'm curious how much of this is speculative vs from use? We are doing code generation for our users (analysts & data scientists), and I found going by method n
by lmeyerov 3y ago
I'm curious how much of this is speculative vs from use?
We are doing code generation for our users (analysts & data scientists), and I found going by method names & type signatures failed pretty hard, while contextual code snippets (example docs, usages, etc you get from a vector DB) worked quite well
An old program synthesis labmate did quite well pre-genAI by using type signatures, but my takeaway is neurosymbolic for more context + option pruning, not being terse for supporting denser summarization & composition. That'd be better... But maybe training would need to change for that to work, or something deeper?
Separately, I do agree it's interesting to think about what IDEs and langs and dev will look like 5 years from now.. smarter LLMs, bigger context windows, and supporting ecosystems change a lot...
- wokwokwok 3y agoI am speculating, I’ve only done this at a small scale and had anecdotally good results. I just put examples in the comment for the function signature. …but I can say with complete confidence that generating code from a coherent set of c header files works better than generating “one shot” full code files just from a high level goal and a file name. You’re basically reducing the problem space from literally anything to a subset that has an defined structure. (It helps that #include literally maps to a flat single header in c and having comments in c headers is typical, perhaps?)
- verdverm 3y agoyes, this is the essence of prompt engineering, you provide more context and it makes the job easier. The more specific you are, the better the results, up to a point. Sometimes they will have trouble breaking out of the context or examples to create the novelty we seek.