3 ms·
No problem! I guess I will make a plug myself --we've been working on a similar 'prompt engineering' tool, ChainForge (https://github.com/ianarawjo/ChainForge h
by fatso784 3y ago
No problem! I guess I will make a plug myself --we've been working on a similar 'prompt engineering' tool, ChainForge (https://github.com/ianarawjo/ChainForge https://github.com/ianarawjo/ChainForge). It's targeted towards slightly different users and use cases than promptfoo --geared more towards early-stage, 'quick-and-dirty' explorations of differences between prompts and models for less experienced programmers, versus the kind of continuous benchmarking and verification testing power that promptfoo offers.
I particularly like promptfoo's support for CI, which I haven't seen anywhere else, and is very important for developers pushing prompts into production (esp since OpenAI keeps updating their models every few months...).