4 ms·
Nope! I'm not trying to suggest the LLM legitimately understands my query from a conceptual perspective via that question. What I am doing in that prompt is en
by cl42 3y ago
Nope! I'm not trying to suggest the LLM legitimately understands my query from a conceptual perspective via that question.
What I am doing in that prompt is ensuring the LLM can follow the instructions. I specifically ask it to write "yes" if it does. If it can't do that part, then I don't want it to even generate code or try to analyze my data.
Hence why I treat it as an assertion failure if it can't follow that instruction, and thus exit the app.
Thanks for reading the code + for the very thoughtful question.
- r-zip 3y agoDeep nets are notoriously overconfident in their predictions. So why do you expect this approach to succeed?
- cl42 3y agoThe broader package I’m working on (PhaseLLM) is specifically focused on devtools for observability and robustness of LLM-powered products. I agree with you that there are lots issues with LLMs and taking them to production. I’m hoping products like this + making them robust will help improve the research as well as the UX.
- nullsense 3y ago>Deep nets are notoriously overconfident in their predictions Sounds a lot like humans