3 ms·
Ask HN: How do you version GPT prompts
This is for the purpose of tracking improvements of a code completion or a chatGPT prompt.
Are you using an existing solution, rolling your own or a mix?
- verdverm 4y agoGit
- okoma 4y agoWhat structure do you use if you wanted to check the accuracy of an older prompt with an updated or extended data set?
- verdverm 4y agoThis question is slightly different because you have to track a testing set and the metrics. I would look to the many ML workflow tools
- okoma 4y agoThanks for pointing towards the right direction. I'll edit the original question. To rephrase, I am looking for a tool to do model lifecycle management https://github.com/kelvins/awesome-mlops#model-lifecycle https://github.com/kelvins/awesome-mlops#model-lifecycle and wonder if there is any one in particular that you'd think is better suited for prompts, i.e. an array of objects with templated text
- okoma 4y agoI found a bunch of resources here: https://alphasec.io/prompt-engineering-playgrounds-marketplaces-guides/ https://alphasec.io/prompt-engineering-playgrounds-marketpla... Good candidates for what I am looking are OpenAI Playground (once it supports versioning and variables for chat), PromptLayer (once it supports chat prompts), EveryPrompt (chat prompts missing). More dev oriented, LangChain https://js.langchain.com/docs/getting-started/guide-chat https://js.langchain.com/docs/getting-started/guide-chat which can then be easily versioned.