4 ms·

Langchain is the weirdest thing. It just does not make any sense to me. I see absolutely no benefit and plenty of cost associated.

ravenstine 3y ago
My best guess is that Langchain might be a good thing if one plans on switching between LLM implementations. Then again, this reminds me of the fantasy that using an ORM will make one's application plug-n-play with any relational database (guaranteed or your money back). Otherwise, I have to agree. Langchain to a large extent seems to base its existence on a problem that barely exists. Outside of LLMs as services, the challenging part about LLMs is figuring out how to get one up and running locally. The hard part isn't writing an application that can work with one. Maintaining "memory" of conversations is relatively trivial, and though a framework might give me a lot of stuff for free, it doesn't seem worth giving up the precision of writing code to do things in a very specific way. Perhaps Langchain is "good" for programming noobs who might benefit from living in just the Langchain universe. The documentation provides enough baby steps that someone who has maybe a few months of experience writing Python can whip something together. However, I'm really giving it the benefit of the doubt here. I really hope noobs aren't getting into programming because they want to build "the next ChatGPT", inherit a bunch of bad ideas about what programming is from Langchain, and then enter the workforce with said ideas.
- treprinum 3y ago
  Prompting is the most difficult part and that is LLM-bound and not very transferable. One can make all kinds of prompts, might want to glue them into some semi-intelligent state machine making its own decisions (=agent), run multi-task prompts (a single LLM call returns multiple answers to various NLP tasks). I guess LangChain helps with the semi-intelligent part but its problem (and the problem of LlamaIndex) is that they were originally written for GPT-2/3 and then new LLMs obliterate the need to do their specific chaining. Running basic prompt templates in both almost always guarantees higher latency for a minimal gain in accuracy. I guess it's all about whether you believe the most recent LLMs to be good enough to do their own adequate decision making inside their own hallucinations, or if you need to enforce it externally. If the latter, you use LangChain or LlamaIndex. If the former, you rely on OpenAI functions/Claude 2 iterative prompting with minimal Python glue. LangChain and LlamaIndex also have some nice functions like document imports, RAG, re-ranking but one can simply copy the corresponding code and use it standalone without the rest of the library.
- kristiandupont 3y ago
  >Maintaining "memory" of conversations is relatively trivial In my experience, it is actually surprisingly hard. I guess it depends on just how "human" you want it to feel. I wrote about it here: [link redacted]
  - gardnr 3y ago
    I'm allergic to medium.com these days.
    - cachehit 3y ago
      Sorry, I know that's a feeling that several people share. To be clear, the post is not behind the paywall, but you can also read it on dev.to, if you want: [link redacted]j
rgrieselhuber 3y ago
As much as I hate to say so, I share the sentiment. It feels very much like the days of AbstractSingletonProxyFactoryBean. Abstractions have diminishing returns.
- msikora 3y ago
  The problem is that LangChain is almost everywhere using the wrong abstraction. I've used the JS version on a project and it was probably more trouble than it's worth. Eventually I ended up using only the most basic things out of it (and even those are kind of broken). I thought the problems were due to it being a translation of a Python library but I've heard that one is very bad as well.
  - powerapple 3y ago
    I share the same sentiment. I see the value of different integrations, probably can save you some time of coding. But overall, it is trying to make something simple more complicated (as least to make them look like so) with all those unnecessary abstractions.
- woah 3y ago
  It's like the massive WordPress deluxe slideshow plugins
kelseyfrog 3y ago
Langchain has compelling examples but once you begin asking, "How will I deploy this to production?" you find yourself replacing it bit by bit until there's nothing left of it to use.
msikora 3y ago
Anyone knows of a good alternative? This seem like such an obvious gap in the "market". Interested especially for JS/TS.
- minimaxir 3y ago
  When the best alternative is "do it yourself," there really can't be a market-friendly alternative.
- treprinum 3y ago
  Microsoft Guidance: https://github.com/guidance-ai/guidance https://github.com/guidance-ai/guidance
- FranklinMaillot 3y ago
  I haven't tried any of them but there's semantic-kernel and autogen by Microsoft, and Haystack. https://github.com/microsoft/semantic-kernel https://github.com/microsoft/semantic-kernel https://github.com/microsoft/autogen https://github.com/microsoft/autogen https://haystack.deepset.ai/ https://haystack.deepset.ai/
- deleted 3y ago
  [deleted]
- stevenhuang 3y ago
  i'll throw my hat in for txtai, see this for a refreshingly simple RAG implementation: https://neuml.hashnode.dev/custom-api-endpoints https://neuml.hashnode.dev/custom-api-endpoints
wokwokwok 3y ago
It’s fun to jump on the hater bandwagon, but did you try using it? If you look at the documentation (1), the API surface is relatively trivial and obvious to me. Every interaction is a prompt template + an LLM + an output parser. What’s so hard to understand about this? Is writing an output parser that extends “BaseOutputParser” really that bad? The parser and LLM are linked using: “chatPrompt.pipe(model).pipe(parser);” How… verbose. Complicated. People who like to have a go at langchain seem to argue that this is “so trivial” you could just do it yourself… but also not flexible enough, so you should do it yourself. Don’t get me wrong, I think they’ve done some weird shit (LCEL), but the streaming and batching isn’t that weird. You see no benefit in using it? Ok. …but come on, it’s not that stupid; I would prefer it was broken into smaller discrete packages (large monolithic libraries like this often end up with lots of bloated half baked stuff in them), and I’d rather it focused on local models, not chatgpt… …but come on. It’s not that bad. No benefit? You’ve implemented streaming and batched actions yourself have you? The API is complicated. The documentation kind of sucks. …but the fundamentals are fine, imo. It irritates me to see people shitting on this project when they haven’t tried it; I don’t even particularly like it… but if you haven’t actually used it, ffs, don’t be a dick about it. If you have used it, maybe a more nuanced take than “it does not make any sense to me” is more helpful to people considering if they want to use it, or parts of it, or what the cost of implementing those parts themselves might be. I personally think these templates (like https://github.com/langchain-ai/langchain/blob/master/templates/llama2-functions/llama2-functions.ipynb https://github.com/langchain-ai/langchain/blob/master/templa...) don’t offer any meaningful value, lack documentation and context and fail to explain the concepts they’re using… but they at least demonstrate how to do various tasks. It probably a valuable reference resource, but not a starting point for people. [1] - https://js.langchain.com/docs/get_started/quickstart https://js.langchain.com/docs/get_started/quickstart
- minimaxir 3y ago
  > The API is complicated. The documentation kind of sucks. …but the fundamentals are fine, imo. That's damning by faint praise.
  - wokwokwok 3y ago
d4rkp4ttern 3y ago
On the subject of cost. I looked at how they do Relevance Extraction, I.e., extract portions of a passage that are relevant to a query (a critical step in a RAG pipeline), and they do it the obvious way, I.e have the LLM parrot out the relevant sentences one by one (“Parrot” not inspired by their logo I swear;). Before looking at their LLMChainExtractor that does this, I had already implemented this massively more efficiently by pre-annotating the passage with sentence numbers and having the LLM simply tell you the relevant sentence numbers. It was natural to do this in an agent loop with function-calling in the Langroid[1] framework. This is both faster (since it has to generate very few tokens) as well as cheaper (remember token generation cost is double the input cost when using GPT4 at least). I was actually surprised that LangChain doesn’t do it this way. Just an example of how we shouldn’t assume the established implementations are the best ones and one should always be skeptical and take a fresh look. I posted about this a couple days ago— https://www.linkedin.com/posts/pchalasani_rag-llm-langchain-activity-7124823472869953537-9T1T https://www.linkedin.com/posts/pchalasani_rag-llm-langchain-... [1] Langroid: https://github.com/langroid/langroid https://github.com/langroid/langroid

ravenstine 3y ago

My best guess is that Langchain might be a good thing if one plans on switching between LLM implementations. Then again, this reminds me of the fantasy that using an ORM will make one's application plug-n-play with any relational database (guaranteed or your money back). Otherwise, I have to agree. Langchain to a large extent seems to base its existence on a problem that barely exists. Outside of LLMs as services, the challenging part about LLMs is figuring out how to get one up and running locally. The hard part isn't writing an application that can work with one. Maintaining "memory" of conversations is relatively trivial, and though a framework might give me a lot of stuff for free, it doesn't seem worth giving up the precision of writing code to do things in a very specific way. Perhaps Langchain is "good" for programming noobs who might benefit from living in just the Langchain universe. The documentation provides enough baby steps that someone who has maybe a few months of experience writing Python can whip something together. However, I'm really giving it the benefit of the doubt here. I really hope noobs aren't getting into programming because they want to build "the next ChatGPT", inherit a bunch of bad ideas about what programming is from Langchain, and then enter the workforce with said ideas.

treprinum 3y ago
Prompting is the most difficult part and that is LLM-bound and not very transferable. One can make all kinds of prompts, might want to glue them into some semi-intelligent state machine making its own decisions (=agent), run multi-task prompts (a single LLM call returns multiple answers to various NLP tasks). I guess LangChain helps with the semi-intelligent part but its problem (and the problem of LlamaIndex) is that they were originally written for GPT-2/3 and then new LLMs obliterate the need to do their specific chaining. Running basic prompt templates in both almost always guarantees higher latency for a minimal gain in accuracy. I guess it's all about whether you believe the most recent LLMs to be good enough to do their own adequate decision making inside their own hallucinations, or if you need to enforce it externally. If the latter, you use LangChain or LlamaIndex. If the former, you rely on OpenAI functions/Claude 2 iterative prompting with minimal Python glue. LangChain and LlamaIndex also have some nice functions like document imports, RAG, re-ranking but one can simply copy the corresponding code and use it standalone without the rest of the library.
kristiandupont 3y ago
>Maintaining "memory" of conversations is relatively trivial In my experience, it is actually surprisingly hard. I guess it depends on just how "human" you want it to feel. I wrote about it here: [link redacted]
- gardnr 3y ago
  I'm allergic to medium.com these days.
  - cachehit 3y ago
    Sorry, I know that's a feeling that several people share. To be clear, the post is not behind the paywall, but you can also read it on dev.to, if you want: [link redacted]j

rgrieselhuber 3y ago

As much as I hate to say so, I share the sentiment. It feels very much like the days of AbstractSingletonProxyFactoryBean. Abstractions have diminishing returns.

msikora 3y ago
The problem is that LangChain is almost everywhere using the wrong abstraction. I've used the JS version on a project and it was probably more trouble than it's worth. Eventually I ended up using only the most basic things out of it (and even those are kind of broken). I thought the problems were due to it being a translation of a Python library but I've heard that one is very bad as well.
- powerapple 3y ago
  I share the same sentiment. I see the value of different integrations, probably can save you some time of coding. But overall, it is trying to make something simple more complicated (as least to make them look like so) with all those unnecessary abstractions.
woah 3y ago
It's like the massive WordPress deluxe slideshow plugins

kelseyfrog 3y ago

Langchain has compelling examples but once you begin asking, "How will I deploy this to production?" you find yourself replacing it bit by bit until there's nothing left of it to use.

msikora 3y ago

Anyone knows of a good alternative? This seem like such an obvious gap in the "market". Interested especially for JS/TS.

minimaxir 3y ago
When the best alternative is "do it yourself," there really can't be a market-friendly alternative.
treprinum 3y ago
Microsoft Guidance: https://github.com/guidance-ai/guidance https://github.com/guidance-ai/guidance
FranklinMaillot 3y ago
I haven't tried any of them but there's semantic-kernel and autogen by Microsoft, and Haystack. https://github.com/microsoft/semantic-kernel https://github.com/microsoft/semantic-kernel https://github.com/microsoft/autogen https://github.com/microsoft/autogen https://haystack.deepset.ai/ https://haystack.deepset.ai/
deleted 3y ago
[deleted]
stevenhuang 3y ago
i'll throw my hat in for txtai, see this for a refreshingly simple RAG implementation: https://neuml.hashnode.dev/custom-api-endpoints https://neuml.hashnode.dev/custom-api-endpoints

wokwokwok 3y ago

It’s fun to jump on the hater bandwagon, but did you try using it? If you look at the documentation (1), the API surface is relatively trivial and obvious to me. Every interaction is a prompt template + an LLM + an output parser. What’s so hard to understand about this? Is writing an output parser that extends “BaseOutputParser” really that bad? The parser and LLM are linked using: “chatPrompt.pipe(model).pipe(parser);” How… verbose. Complicated. People who like to have a go at langchain seem to argue that this is “so trivial” you could just do it yourself… but also not flexible enough, so you should do it yourself. Don’t get me wrong, I think they’ve done some weird shit (LCEL), but the streaming and batching isn’t that weird. You see no benefit in using it? Ok. …but come on, it’s not that stupid; I would prefer it was broken into smaller discrete packages (large monolithic libraries like this often end up with lots of bloated half baked stuff in them), and I’d rather it focused on local models, not chatgpt… …but come on. It’s not that bad. No benefit? You’ve implemented streaming and batched actions yourself have you? The API is complicated. The documentation kind of sucks. …but the fundamentals are fine, imo. It irritates me to see people shitting on this project when they haven’t tried it; I don’t even particularly like it… but if you haven’t actually used it, ffs, don’t be a dick about it. If you have used it, maybe a more nuanced take than “it does not make any sense to me” is more helpful to people considering if they want to use it, or parts of it, or what the cost of implementing those parts themselves might be. I personally think these templates (like https://github.com/langchain-ai/langchain/blob/master/templates/llama2-functions/llama2-functions.ipynb https://github.com/langchain-ai/langchain/blob/master/templa...) don’t offer any meaningful value, lack documentation and context and fail to explain the concepts they’re using… but they at least demonstrate how to do various tasks. It probably a valuable reference resource, but not a starting point for people. [1] - https://js.langchain.com/docs/get_started/quickstart https://js.langchain.com/docs/get_started/quickstart

minimaxir 3y ago
> The API is complicated. The documentation kind of sucks. …but the fundamentals are fine, imo. That's damning by faint praise.
- wokwokwok 3y ago

d4rkp4ttern 3y ago

On the subject of cost. I looked at how they do Relevance Extraction, I.e., extract portions of a passage that are relevant to a query (a critical step in a RAG pipeline), and they do it the obvious way, I.e have the LLM parrot out the relevant sentences one by one (“Parrot” not inspired by their logo I swear;). Before looking at their LLMChainExtractor that does this, I had already implemented this massively more efficiently by pre-annotating the passage with sentence numbers and having the LLM simply tell you the relevant sentence numbers. It was natural to do this in an agent loop with function-calling in the Langroid[1] framework. This is both faster (since it has to generate very few tokens) as well as cheaper (remember token generation cost is double the input cost when using GPT4 at least). I was actually surprised that LangChain doesn’t do it this way. Just an example of how we shouldn’t assume the established implementations are the best ones and one should always be skeptical and take a fresh look. I posted about this a couple days ago— https://www.linkedin.com/posts/pchalasani_rag-llm-langchain-activity-7124823472869953537-9T1T https://www.linkedin.com/posts/pchalasani_rag-llm-langchain-... [1] Langroid: https://github.com/langroid/langroid https://github.com/langroid/langroid