3 ms·

100% this! What is worse is that LangChain hides their prompts away, I had to read the source code and mess with private variables of nested classes just to cha

100% this! What is worse is that LangChain hides their prompts away, I had to read the source code and mess with private variables of nested classes just to change a single prompt from something like RetrievalQA, and not only that, the default prompt they use is actually bad, they are lucky things work because GPT-3.5 and GPT-4 are damn smart machines, with any other open LLM, things break. I was hoping for good defaults, but they are not, the prompt I wrote over 6 months ago little after the launch of ChatGPT to do some of the same things work much better. Would you have anything you can share with us about those "several features using highly sophisticated LLM chains that do all manner of reasoning", I'm really curious about the challenges, the process and insights there

rchaves 3y ago
this inspired me on writing a new section in my project "Prompts on the outside" (https://github.com/rogeriochaves/litechain#prompts-on-the-outside https://github.com/rogeriochaves/litechain#prompts-on-the-ou...)
sjnair96 3y ago
Can you share some insights/examples, if you can, on how you improved the prompts? One I feel is particularly poor is the next question generation/past question condensation prompts which are used to refine the user's input based on the history, so that the query includes all the context required for the question, and hence, incorporating "memory".
- rchaves 3y ago
  Yeah I never know where memory goes exactly in langchain, it's not exactly clear all the time. But sure, the main insight I remember is this, take a look at their MULTI_PROMPT_ROUTER_TEMPLATE: https://github.com/hwchase17/langchain/blob/560c4dfc98287da1bc0cfc1caebbe86d1e66a94d/langchain/chains/router/multi_prompt_prompt.py https://github.com/hwchase17/langchain/blob/560c4dfc98287da1... It's a lot of instructions for an LLM, they seem to forget an LLM is an auto-completion machine, and which data it is trained on. Using <<>> for sections is not a normal thing, it's not markdown, which probably the thing read way more often on the internet, instead of open json comments, why not type signatures, instead of so many rules, why not give it examples? It is an autocomplete machine! They are relying too much on the LLM being smart because they probably only test stuff in GPT-4 and 3.5, but with GPT4All models this prompt was not working at all, so I had to rewrite it, for simple routing, we don't even need json, carying the `next_inputs` here is weird if you don't need it. So this is my version of it: https://gist.github.com/rogeriochaves/b67676977eebb1936b9b5c2cd0e5154d https://gist.github.com/rogeriochaves/b67676977eebb1936b9b5c... It's so basic it's dumb, yet it is more powerful, as it does not rely on GPT-4 level intelligence, it's just what I needed

rchaves 3y ago

this inspired me on writing a new section in my project "Prompts on the outside" (https://github.com/rogeriochaves/litechain#prompts-on-the-outside https://github.com/rogeriochaves/litechain#prompts-on-the-ou...)

sjnair96 3y ago

Can you share some insights/examples, if you can, on how you improved the prompts? One I feel is particularly poor is the next question generation/past question condensation prompts which are used to refine the user's input based on the history, so that the query includes all the context required for the question, and hence, incorporating "memory".

rchaves 3y ago
Yeah I never know where memory goes exactly in langchain, it's not exactly clear all the time. But sure, the main insight I remember is this, take a look at their MULTI_PROMPT_ROUTER_TEMPLATE: https://github.com/hwchase17/langchain/blob/560c4dfc98287da1bc0cfc1caebbe86d1e66a94d/langchain/chains/router/multi_prompt_prompt.py https://github.com/hwchase17/langchain/blob/560c4dfc98287da1... It's a lot of instructions for an LLM, they seem to forget an LLM is an auto-completion machine, and which data it is trained on. Using <<>> for sections is not a normal thing, it's not markdown, which probably the thing read way more often on the internet, instead of open json comments, why not type signatures, instead of so many rules, why not give it examples? It is an autocomplete machine! They are relying too much on the LLM being smart because they probably only test stuff in GPT-4 and 3.5, but with GPT4All models this prompt was not working at all, so I had to rewrite it, for simple routing, we don't even need json, carying the `next_inputs` here is weird if you don't need it. So this is my version of it: https://gist.github.com/rogeriochaves/b67676977eebb1936b9b5c2cd0e5154d https://gist.github.com/rogeriochaves/b67676977eebb1936b9b5c... It's so basic it's dumb, yet it is more powerful, as it does not rely on GPT-4 level intelligence, it's just what I needed