2 ms·

I did something similar to this but got stuck that the code generated would sometimes work, sometimes not for identical prompts. I also found that as an expert

I did something similar to this but got stuck that the code generated would sometimes work, sometimes not for identical prompts. I also found that as an expert in the topics it was easy to write a prompt that would generally build a reasonable data pipeline but I couldn't imagine if I just had some data, but not the expertise, I could do the same. How do you account for these issues?

cl42 4y ago
Fantastic questions! Re: working/not working at times -- this is still an issue. It's why I'm building PhaseLLM more broadly (https://github.com/wgryc/phasellm https://github.com/wgryc/phasellm) -- need a robust pipeline that can also "reset" parts of itself if an LLM makes errors or mistakes. You can see my prompts in this file: https://github.com/wgryc/phasellm/blob/main/demos-and-products/researchgpt/researchgpt.py https://github.com/wgryc/phasellm/blob/main/demos-and-produc... I autogenerate a fairly big starting prompt and keep resubmitting it. It describes the data set extensively, which helps quite a bit. That being said, a lot more can be done here around prompt optimization + making this more robust.