5 ms·
I could add a couple things from my own experiences. Storing prompts in a database seemed like a good idea, but in practice it ended up being a disater. Storin
by fswd 3y ago
I could add a couple things from my own experiences. Storing prompts in a database seemed like a good idea, but in practice it ended up being a disater. Storing the prompt in a python/typescript file, up front at the top, works well. Using OpenAI playground with it's ability to export a prompt works well, or even better, something in gradio running in vscode with debugging mode, works even better. Few shots with refinements works really well. LangChain did not work well for any of my cases, I might go as boldly as saying that using langchain is bad practice.
- ukuina 3y agoSo much "Yes!" for LangChain being bad practice. An unnecessarily bloated abstraction over what should be simple API calls.
- yinser 3y agoDo you have a recommendation of how to easily connect a language model to a python repl, apify, bash shell, and composable chaining structures if not langchain? I find those structures invaluable but am curious where else I could build these programs.
- ntonozzi 3y agoIt’s great for prototyping and seeing what is possible but for running in production you’ll likely need to write it yourself, and it will just take a few minutes.
- jmccarthy 3y agoCould be we're in a (short?) interregnum analogous to pre-Rails Ruby: there are lots of nascent frameworks, but the dominant one hasn't been born yet. FWIW - DIY is working well for me.
- kordlessagain 3y agoCheck out https://www.featurebase.com/blog/doctorgpt-harnessing-the-power-of-semantic-knowledge-graphs-for-unstructured-data https://www.featurebase.com/blog/doctorgpt-harnessing-the-po... and the DoctorGPT repo for doing some things with documents and LLMs, without a framework. I use simple templating and vectors to assemble prompts.
- ukuina 3y agoThe current trend for productionizing LLM-based applications is to write your own (really thin) wrappers around the actual LLM call. The majority of your code should be business logic independent of the LLM, anyway: information retrieval, user interface, response logging, and so on. In my experience with cloud-based and local models, LLM chaining compounds errors; I would urge you to look at few-shot, single-query interaction models for business applications using LLMs as a new unit of compute.
- Dwood023 3y agoCould you explain how storing prompts was a disaster?
- fswd 3y agoPrompts should be really easy to cut and paste from an editor to a playground. Updating a database is unnecessary friction with no real benefit.
- phillipcarter 3y agoIt's delightfully hacky, but we actually have our prompt (that we parameterize later) stored in a feature flag right now, with a few variations! I actually can't believe we shipped with that, but hey, it works? Each variation is pulled from a specific version in a separate repo where we iterate on the prompt. We're going to likely settle on just storing whatever version of the prompt is considered "stable" as a source file, but for now this isn't actively hurting us, as far as we can tell, and there's a lot of prompt engineering left to do.
- ntonozzi 3y agohttps://thedailywtf.com/articles/the-inner-json-effect https://thedailywtf.com/articles/the-inner-json-effect
- darkteflon 3y agoI’m keeping all our prompts in a json file (along with some helpful metadata for us humans). No idea if I’m doing it right.