3 ms·
My best guess is that Langchain might be a good thing if one plans on switching between LLM implementations. Then again, this reminds me of the fantasy that us
by ravenstine 3y ago
My best guess is that Langchain might be a good thing if one plans on switching between LLM implementations. Then again, this reminds me of the fantasy that using an ORM will make one's application plug-n-play with any relational database (guaranteed or your money back).
Otherwise, I have to agree. Langchain to a large extent seems to base its existence on a problem that barely exists. Outside of LLMs as services, the challenging part about LLMs is figuring out how to get one up and running locally. The hard part isn't writing an application that can work with one. Maintaining "memory" of conversations is relatively trivial, and though a framework might give me a lot of stuff for free, it doesn't seem worth giving up the precision of writing code to do things in a very specific way.
Perhaps Langchain is "good" for programming noobs who might benefit from living in just the Langchain universe. The documentation provides enough baby steps that someone who has maybe a few months of experience writing Python can whip something together. However, I'm really giving it the benefit of the doubt here. I really hope noobs aren't getting into programming because they want to build "the next ChatGPT", inherit a bunch of bad ideas about what programming is from Langchain, and then enter the workforce with said ideas.
- treprinum 3y agoPrompting is the most difficult part and that is LLM-bound and not very transferable. One can make all kinds of prompts, might want to glue them into some semi-intelligent state machine making its own decisions (=agent), run multi-task prompts (a single LLM call returns multiple answers to various NLP tasks). I guess LangChain helps with the semi-intelligent part but its problem (and the problem of LlamaIndex) is that they were originally written for GPT-2/3 and then new LLMs obliterate the need to do their specific chaining. Running basic prompt templates in both almost always guarantees higher latency for a minimal gain in accuracy. I guess it's all about whether you believe the most recent LLMs to be good enough to do their own adequate decision making inside their own hallucinations, or if you need to enforce it externally. If the latter, you use LangChain or LlamaIndex. If the former, you rely on OpenAI functions/Claude 2 iterative prompting with minimal Python glue. LangChain and LlamaIndex also have some nice functions like document imports, RAG, re-ranking but one can simply copy the corresponding code and use it standalone without the rest of the library.
- kristiandupont 3y ago>Maintaining "memory" of conversations is relatively trivial In my experience, it is actually surprisingly hard. I guess it depends on just how "human" you want it to feel. I wrote about it here: [link redacted]