25 ms·

Ask HN: Prompt manager for developers / services?

3 points by theogravity 3y ago

I'm working on a system that utilizes OpenAI / ChatGPT and hard-code a lot of prompt or chunks of prompts that would be combined together in variables. Is there a product out there that allows me or a product person to: - Store a list of prompts in a key/value like fashion - Has an API where I can request a list of keys and get the prompts corresponding to them - Has version history - Can assign a version of a prompt to a particular environment (eg dev, prod) - Lets others edit prompts - (Optional) Can test prompts against an LLM - (Optional) The API / SDK can listen for updates to prompts like how Launch Darkly can push flag changes in real-time. I've searched for "prompt manager", "ai prompt manager", "prompt management tool", and variants. The tools that do come up seem to be geared for non-developers or has no API for me to just fetch plain text prompts for use in our own LLM in our own infrastructure. The core of it is probably a hosted versioned K/V system with ACL, and an API to access that K/V wrapped around a GUI for management.

breckenedge 3y ago
Like promptfoo? https://github.com/promptfoo/promptfoo https://github.com/promptfoo/promptfoo
- theogravity 3y ago
  Promptfoo looks like it's mainly built for testing prompts and the API is framed around testing. The readme states: "The goal: test-driven prompt engineering, rather than trial-and-error." Which is an optional part of my requirements.
colesouth 3y ago
I'm building a consumer facing prompt manager (https://promptfolder.com/ https://promptfolder.com/ - no API yet), but I've looked at most of the other tools... unfortunately I haven't seen anything that is a solid fit for what you're looking for. You might want to contact the devs behind PromptPal (https://github.com/PromptPal/PromptPal https://github.com/PromptPal/PromptPal), they seem to be some of the more technically advanced and fast moving people building in this area... maybe they can hack something together for you. AIPRM is one of the more popular prompt managers, but as of May they don't have a public API (https://forum.aiprm.com/t/use-aiprm-prompt-templates-api/3824/13 https://forum.aiprm.com/t/use-aiprm-prompt-templates-api/382...).
- theogravity 3y ago
  I actually wrote an issue on the PromptPal GH around 30 minutes before my HN post. They use Metamask / Ethereum addresses for auth / account management, which sounds crazy to me for a product that's completely divorced from web3 / cryptocurrency. Promptfolder isn't what I'm looking for unfortunately based on the template variable assignment the example has. I really just want to store text and get it back via an API. The text just happens to be a prompt.
  - colesouth 3y ago
    Pivoting from crypto to AI... many such cases :) Sounds like there is maybe some code debt involved!
juntao 3y ago
Maybe you can try https://flows.network/ https://flows.network/, which supports environment variables when creating an app. You can also manage the environment variables later with their UI. Click on https://flows.network/flow/createByTemplate/Telegram-ChatGPT https://flows.network/flow/createByTemplate/Telegram-ChatGPT, you can see you could set up system_prompt.