21 ms·

Why we no longer use LangChain for building our AI agents

• te_chris 2y ago
  The thing that blows my mind is that this wasn’t obvious to them when they first looked at langchain
• CharlieDigital 2y ago
  Bigger problem might be using agents in the first place. We did some testing with agents for content generation (e.g. "authoring" agent, "researcher" agent, "editor" agent) and found that it was easier to just write it as 3 sequential prompts with an explicit control loop. It's easier to debug, monitor, and control the output flow this way. But we still use Semantic Kernel[0] because the lowest level abstractions that it provides are still very useful in reducing the code that we have to roll ourselves and also makes some parts of the API very flexible. These are things we'd end up writing ourselves anyways so why not just use the framework primitives instead? [0] https://github.com/microsoft/semantic-kernel https://github.com/microsoft/semantic-kernel
  • Kiro 2y ago
    What's the difference? I thought "agents" was just a fancier word for sequential prompts.
    • ec109685 2y ago
      Some folks try to orchestrate the whole operation by a higher level prompt that essentially uses function calls to more specific prompts. Versus just using the LLM’s for specific tasks and heuristics / own code for the orchestration. But I agree there is a lot of anthropomorphizing that over states current model capabilities and just confuses things in general.
    • CharlieDigital 2y ago
      Typically, the term "agents" implies some autonomous collaboration. In an agent workflow, the flow itself is non-deterministic. One agent can work with another agent and keep cycling between themselves until an output is resolved that meets some criteria. An agent itself is also typically evaluating the terminal condition for the workflow.
    • ilaksh 2y ago
      "Agent" means that it outputs JSON with a function call name and parameters which you execute and usually then feed the results back to the LLM.
    • refulgentis 2y ago
      It's also used to mean "characters interacting with each other" and sort of message passing between them. Not sure but I get the sense thats what the author is using it as
    • mstipetic 2y ago
      Sequential prompts with an occasional cron job
    • isaacfung 2y ago
      Some "agents" like the minecraft bot Voyager(https://github.com/MineDojo/Voyager https://github.com/MineDojo/Voyager) have a control loop, they are given a high level task and then they use LLM to decide what actions to take, then evaluate the result and iterate. In some LLM frameworks, a chain/pipeline just uses LLM to process input data(classification, named entitiy extraction, summary, etc).
  • huevosabio 2y ago
    What does semantic kernel do for you? It isn't immediately obvious from the Readme.
    • CharlieDigital 2y ago
      SK does a lot of the same things that Langhain does at a high level. The most useful bits for us are prompt templating[0], "inlining" some functions like `recall` into the text of the prompt [1], and service container [2] (useful if you are using multiple LLM services and models for different types of prompts/flows). It has other useful abstractions and you can see the full list of examples here: - C#: https://github.com/microsoft/semantic-kernel/tree/main/dotnet/samples/Concepts https://github.com/microsoft/semantic-kernel/tree/main/dotne... - python: https://github.com/microsoft/semantic-kernel/tree/main/python https://github.com/microsoft/semantic-kernel/tree/main/pytho... --- [0] https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/PromptTemplates/PromptyFunction.cs https://github.com/microsoft/semantic-kernel/blob/main/dotne... [1] https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Memory/TextMemoryPlugin_MultipleMemoryStore.cs#L283 https://github.com/microsoft/semantic-kernel/blob/main/dotne... [2] https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Kernel/CustomAIServiceSelector.cs https://github.com/microsoft/semantic-kernel/blob/main/dotne...
    • whoknowsidont 2y ago
      I'm not OP, but it's just C#/.NET glue and "sample" code for Azure, OpenAI, and a few others (if I were to generously describe it). It doesn't actually "do" anything or provide useful concepts. I wouldn't use it for anything, personally, even to read.
• deckar01 2y ago
  I recently unwrapped linktransformer to get access to some intermediate calculations and realized it was a pretty thin wrapper around SentenceTransformer and DBScan. It would have taken me so much longer to get similar results without copying their defaults and IO flow. It’s easy to take for granted code you didn’t have to develop from scratch. It would be interesting if there was a tool that inlined dependency calls and shook out unvisited branches automatically.
  • luke-stanley 2y ago
    From memory, I recall Vulture might do something like that!
• danielmarkbruce 2y ago
  Yup. The problem with frameworks is they assume (historically mostly but not always correctly) that layers of abstraction mean one can forget about the layers below. This just doesn't work with LLMs. The systems are closer to biology or something.
  • nosefurhairdo 2y ago
    Very much depends on the framework. I'm currently building a GitHub App with the Probot framework, which mostly just handles authentication boilerplate and some testing niceties, then just gives you an authenticated GitHub API client (no facade/abstraction). Then of course there's the many web application frameworks, because nobody in their right mind would want to implement http request parsing themselves (outside of academic exercises). In fact, I would argue that most popular frameworks exist precisely because it's often more time efficient to forget about underlying details. All computer software is built on abstraction. The key is picking the right level of abstraction for your use case.
    • danielmarkbruce 2y ago
      Reread the thread and the comment. It's about the LLM frameworks and acknowledges that most non LLM frameworks historically are helpful and correct in abstracting away details.
      • nosefurhairdo 2y ago
        Ah the bit in parentheses was worded such that I misunderstood your point.
        • danielmarkbruce 2y ago
          That bit is poorly worded. I should have had a comma before the last word. My bad.
  • randomdata 2y ago
    It often took quite a long time for those historic frameworks to get the abstraction right. Survivorship bias sees us forget all the failed attempts. I'm unconvinced there is no room for a framework here because LLMs are somehow special. LangChain just missed the mark. Unsurprisingly so, it being an early attempt, not to mention predating general availability of the LLM chatbots that have come to define the landscape.
• Kydlaw 2y ago
  IMO LangChain provides very high level abstractions that are very useful for prototyping. It allows you to abstract away components while you dig deeper on some parts that will deliver actual value. But aside from that, I don't think I would run it in production. If something breaks, I feel like we would be in a world of pain to get things back up and running. I am glad they shared their experience on that, this is an interesting data point.
• cyanydeez 2y ago
  In some sense, this could be retitled "We no longer use training wheels on our bikes"
• maximilianburke 2y ago
  I just pulled out LangChain from our AI agents; we now have much smaller docker images and the code is a lot easier to understand.
• etse 2y ago
  My reading of the article is that because LangChain is abstracted poorly, frameworks should not be used, but that seems a bit far. my experience is that Python has a frustrating developer experience for production services. So I would prefer a framework with better abstractions and a solid production language (performance and safety), over no framework and Python (if those were options)
  • lolinder 2y ago
    For most of what people are doing with AI you don't need Python because you don't need the ML ecosystem. You're either going to be talking to some provider's API (in which case there are wrappers aplenty and even if there weren't their APIs are simple and trivial to wrap yourself) or you're going to self-host a model somewhere, in which case you can use something like ollama to give yourself an easy API to code against. All of the logic of stringing prompts and outputs together can easily happen in basically any programming language with maybe a tiny bespoke framework customized to your needs. Calling these things "AI agents" makes them sound both cooler and more complicated than they actually are or need to be. It's all just taking the output from one black box and sticking it into the input of another, the same kind of work frontline programmers have been doing for decades.
    • ilaksh 2y ago
      They become agents when the LLM output is function calls.
      • lolinder 2y ago
        So the output of Black Box A is an instruction to give Black Box B a piece of data X and then give the resulting output back to BBA. We're still just wiring up black boxes to each other the same as we've always done, and we still don't need an abstraction for that.
  • autokad 2y ago
    prompt engineering requires the ability to see what is happening at various steps and langchain makes that harder if not impossible. honestly I don't need that much abstraction.
• dcole2929 2y ago
  I've seen a lot of stuff recently about how LangChain and other frameworks for AI/LLM are terrible and we shouldn't use them and I can't help but think that people are missing the point. If you need strong customization or flexibility frameworks of any kind are almost always the wrong choice, whether you're building a website or an AI agent. That's kind of the whole point of a framework. Opinionated workflows that enable a specific kind of application. Ideally the goal is to cover 80% of the cases and provide escape hatches to handle the other 20% until you can successfully cover those too. As someone new to the space I have zero opinions of whether LangChain is better than writing it all yourself, but I can certainly say that, I at least, appreciate having a proscribed way of doing things, and I'm okay with the idea that I may get to a place where it no longer serves my needs. It's also worth noting that the benefit of LangChain is the ability to "chain" together these various AI links. Is there a better easier way to do that? Probably, but LangChain removes that overhead.
  • riwsky 2y ago
    As the article points out, the difference between frameworks for building a website vs building an LLM agent is that we have decades more industrial experience behind our website-building opinions. I’ve used heavyweight frameworks before, and would understand your defense in the context of eg complaints about Spring Boot—but Langchain isn’t Spring; it really does kinda suck, for reasons that go beyond the inherent trade offs of using any framework.
  • ilaksh 2y ago
    I think that yes, there is a better way. You have a function that calls the API, then take the output and call another function that calls the API, inserting the first output into the second one's prompt using an f-string or whatever. You can have a helper function that has defaults for model params or something. You don't need an abstraction at all really. Inserting the previous output into the new prompt is one line of code, and calling the API is another line of code. If you really feel like you need to abstract that then you can make an additional helper function. But often you want to do different things at each stage so that doesn't really help.
• altdataseller 2y ago
  Langchain reminds me of GraphQL. A technology that a lot of ppl seem to hype about, sounds like something you should use because all the cool kids use it, but at the end of the day just makes things unncessarily complicated.
  • OutOfHere 2y ago
    GraphQL actually holds value in my view as it gives custom SQL-like functionality instead of basic JSON APIs. With it, you can do fewer calls and retrieve only the attributes you need. Granted, if SQL were directly an API, then GraphQL wouldn't hold too much value. Langchain has no such benefit.
    • andybak 2y ago
      Surely SQL is an API? The line between language and API is fairly blurry.
      • newzisforsukas 2y ago
        Can you elaborate?
        • manquer 2y ago
          What GP means is it is a Programmable Interface, any interface you can interact against is an API. That means any programing complete language is an API, so are sign languages or human languages. While nobody does it , SQL implementations have network API, authentication, authorization, ACL/RBAC, serialization, Business logic all the things you use in RESTful apis can all be done with just db servers. You can expose in theory a direct SQL API to clients to consume without any other language or other components to the stack . Most SQL servers use some layer on top of TCP/IP to connect their backends to frontend .libpq is the client which does this in postgreSQL for example . You could either wrap that in Backend SQL server with an extension and talk to browser and other clients in HTTP[1], or you can write a wasm client in browsers to directly talk to TCP/IP port on the SQL server Perhaps if you are oracle , that makes sense, but for no one else, they do build and push products that basically do parts of this . [1] projects like postgREST basically do this .
• djohnston 2y ago
  Idk, dude spends the post whining about writing multi agent architecture and doesn’t mention langgraph once. Reads like a lead who failed to read the docs.
  • 2C64 2y ago
    LangGraph is the primary reason I use LangChain - being able to express my flow as a state machine has been a boon to both the design of my platform as well as my own productivity.
  • esafak 2y ago
    How does langgraph stack up against the alternatives?
• andrewfromx 2y ago
  "When abstractions do more harm than good" I'll take this for $2000 please and if i get the daily double, bet it all.
• wouldbecouldbe 2y ago
  Everyone in my office is talking about ai agents as a magic bullet, driving me crazy
• Treesrule14 2y ago
  Has anyone else found a good way to swap out models between companies, Langchain has made it very easy for us to swap between openai/anthropic etc
  • hyperliner 2y ago
    [dead]
  • riwsky 2y ago
    The point is that you don’t need a framework for that; the APIs are already similar enough that it should be obvious how to abstract over them using whatever approach is natural in your programming language of choice.
    • refulgentis 2y ago
      I have a consumer app that swaps between the 5 bigs and wholeheartedly agree, except, God help you if you're doing Gemini. I somewhat regret hacking it into the same concepts as everyone else. I should have built stronger separation boundaries with more general abstractions. It works fine, I haven't had any critical bugs / mistakes, but it's really nasty once you get to the actual JSON you'll send. Google's was 100% designed by a committee of people who had never seen anyone else's API, and if they had, they would have dismissed it via NIH. (disclaimer: ex-Googler, no direct knowledge)
      • __cayenne__ 2y ago
        luckily Google now support's using the OpenAI lib https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/call-gemini-using-openai-library https://cloud.google.com/vertex-ai/generative-ai/docs/multim...
      • Jensson 2y ago
        > Google's was 100% designed by a committee of people who had never seen anyone else's API Google made their API before the others had one, since they were the first with making these kind of language models. Its just that it has been an internal API before.
• nosefrog 2y ago
  Anyone who has read LangChain's code would know better than to depend on it.
  • whydid 2y ago
    A heuristic that I use when judging code quality is a search for "datas" or "metadatas".
• infecto 2y ago
  LangChain itself blows my mind as one of the most useless libraries to exist. I hope this does not come off the wrong way but so many people told me they were using it so it was easy to move been models. I just did not understand it, these are simple API calls that felt like Web Dev 101 when starting a new product. Maybe its that so many new people were coming into the field using LLM but it surprised me as even what I thought were experienced people were struggling. Its like LLMs brought out the confusion in people. It was interesting as a library at the very beginning to see how people were thinking about patterns but pretty useless in production.
  • refulgentis 2y ago
    Ah, the halcyon days of March 2023, we were a while loop away from AGI. I remember there was something that was It for like a month, to the point that whoever built the framework was treating a cocktail napkin on which they scribbled, whatever, "act, evaluate, decide next action, repeat", as if it was a historical talisman. And I wasn't sure! Maybe it was!
    • causal 2y ago
      Yeah I thought the consensus against LangChain was formed a year ago, surprised to still be seeing these articles.
      • refulgentis 2y ago
        Just chit chatting, not a strong claim, more a hot take I turn over in my mind: my guess is 40% of software engineers did a AI pivot the last 18 months, so there's a massive market for frameworks, and there's an inclination to go beyond REST requests, find something that just does it for you / can do all the cool patterns you'll find in research papers. Incredible amount of bad info out there, whether its the 10th prompting framework that boils down to a while loop and just drives up token costs, the 400 LLM tokenizer library that can only do GPT-3.5/4.0, the Nth app that took XX ex-FAANG and $XX mil and a year to get another web app, or another iOS-only OpenAI client with background blur,m memory thats an array of strings injected into every call It's at the point where I'm hoping for a cooling down even though I'm launching something*, and think it's hilarious people rant about it all just being hype and think people agree. * TL;Dr consumer app with 'chain gui', just hand people an easy to use GUI like playground.openai.com / console.anthropic.com, instead of getting cute and being the Nth team to try to launch a full grade assistant on a monthly plan matching openai pricing, shoving 6000K+ prompts with each request and not showing them
• elijahbenizzy 2y ago
  I really like the idea of "good" and "bad" abstractions. I have absolutely built both. This sentiment is echoed in this comment in reddit comment as well: https://www.reddit.com/r/LocalLLaMA/comments/1d4p1t6/comment/l6g1b3t/ https://www.reddit.com/r/LocalLLaMA/comments/1d4p1t6/comment.... Similarly to this post, I think that the "good" abstractions handle application logic (telemetry, state management, common complexity), and the "bad" abstractions make things abstract away tasks that you really need insight into. This has been a big part of our philosophy on Burr (https://github.com/dagworks-inc/burr https://github.com/dagworks-inc/burr), and basically everything we build -- we never want to tell how people should interact with LLMs, rather solve the common problems. Still learning about what makes a good/bad abstraction in this space -- people really quickly reach for something like langchain then get sick of abstractions right after that and build their own stuff.
  • laborcontract 2y ago
    > the "bad" abstractions make things abstract away tasks that you really need insight into. Yup. People say to use langchain to prototype stuff before it goes into production but I find it falls flat there. The documentation is horrible and they explain absolutely zero about the methods they use, so the only way to “learn” is by reading their spaghetti code.
    • elijahbenizzy 2y ago
      Agreed — also I’m generally against prototyping stuff and then entirely rewriting it for production as the default approach. It’s a nice idea but nobody ever actually rewrites it (or they do and it’s exceedingly painful). In true research it makes sense, but very little of what engineers do falls under that category. Instead, it’s either “welp, pushed this to prod and got promoted and it’s someone else’s problem” or “sorry, this valuable thing is too complex to do right but this cool demo got me promoted...”
• gravenate 2y ago
  Hard Agree, Semantic Kernal, On the other hand seems to actually be a value add on top of the simple API calls. Have you guys tried it ?
• muzani 2y ago
  Langchain was released in October 2022. ChatGPT was released in November 2022. Langchain was before chat models were invented. It let us turn these one-shot APIs into Markov chains. ChatGPT came in and made us realize we didn't want Markov chains; a conversational structure worked just as well. After ChatGPT and GPT 3.5, there were no more non-chat models in the LLM world. Chat models worked great for everything, including what we used instruct & completion models for. Langchain doing chat models is just completely redundant with its original purpose.
  • fnordpiglet 2y ago
    We use instruct models extensively as we find smaller models fine tuned to our prompts perform better when general chat models that are much larger. This lets us run inference that can be 1000x cheaper than 3.5, meaning both money saving and much better latencies.
    • muzani 2y ago
      This feels like a valid use for langchain then. Thanks for sharing. Which models do you use and for what use cases? 1000x is quite a lot of savings; normally even with fine-tuning it's at most 3x cheaper. Any cheaper we'd need to get like $100k of hardware.
    • deleted 2y ago
      [deleted]
  • isaacfung 2y ago
    I am not sure what you mean by "turn these one-shot APIs into Markov chains." To me, langchain was mostly marketed as a framework that makes RAG easy by providing integration with all kinds of data sources(vector db, pdf, sql db, web search, etc). Also older models(including initial chatgpt) had limited context lengths. Langchain helped you to manage the conversation memory by splitting it up and storing the pieces in a vector db. Another thing langchain did was implementing the react framework(which you can implement with a few lines of code) to help you answer multi hop problems.
• __loam 2y ago
  Langchain has always been open source and has always sucked. I'm shocked anyone still uses it when you can see it for yourself.
• jsemrau 2y ago
  LCEL is such a weird paradigm that I never got the hang of. Why | use | pipes?
  • elbear 2y ago
    I found it weird as well to see that. I didn't know LangChain overrode Python syntax. But, if you're familiar with Linux/Unix, this should be familiar. You are piping the output of one function as the input of another function.
• matusp 2y ago
  This echoes our experience with LangChain, although we have abandoned it before putting it into production. We found out that for simple use cases it's too complex (as mentioned in the blog), and for complex use cases it's too difficult to adapt. We were not able to identify what is the sweet spot when it is worth it to use it. We felt like we can easily code ourselves most of its functionality very quickly and in a way that fits our requirements.
  • localfirst 2y ago
    i've never seen a HN thread where everybody just unanimously agrees and wow I definitely will not be recommending Langchain or using it personally after reading through all the horror stories. seems like another case of creating busysoftware. doesn't add value, rather takes away value through needless pedantry, but has enough github stars for people to take a look anyways
• elbear 2y ago
  It would have been great if the article provided a more realistic example. The example they use is indeed more complex than the openai equivalent, but LangChain allows you to use several models from several providers. Also, it's true that the override of the pipe character is unexpected. But it should make sense, if you're familiar with Linux/Unix. And I find it shows more clearly that you are constructing a pipeline: prompt | model | parser
  • bestcoder69 2y ago
    I can already use multiple backends by writing different code. The value-add langchain would need to prove is whether i can get better results using their abstractions compared to me doing it manually. Every time I’ve looked at how langchain’s prompts are constructed, they went wayyy against LLM vendor guidance so I have doubts. Also the downside of not being able to easily tweak prompts based on experiments (crucial!) And not to mention the library doesn’t actually live up to this use case, and you immediately (IME) run into “you actually can’t use a _Chain with provider _ if you want to use their _ API”, so I ultimately did have to care about whats supposed to be abstracted over
    • elbear 2y ago
      Your comment gives better reasons than the article for not using LangChain.
  • drdaeman 2y ago
    Yeah, I was kind of surprised. The premise of the article started as "LangChain abstractions are off" and then the complaint was about... just a very simple pipeline? I honestly don't care about the syntax (as long as it's sane enough), and `|` operator overloading isn't the worst one. Manually having to define a parser object gives off some enterprise Java vibes, and I get the httplib vs requests comparison - but it's not the end of the world. If anything, the example from the article left me wondering "why do they say it's worse, when at this level of abstraction it really looks better unless we don't ever need to customize the pipeline at all?" And they never gave any real example (about spawning those agents or something) that actually shows where the abstractions are making things hard or obscure. Honestly, on the first reading, the article [wrongly] gave me an impression of saying "we don't use LangChain anymore because it lacks good opinionated defaults", which is surely wrong - it would be a very odd take, given the initial premise of using it production for a long while. (I haven't used LangChain or any LLMs in production, just toyed around a little bit. I can absolutely agree with the article that if all you care about is one single backend, then all those abstractions are not likely to be a good idea.)
• spullara 2y ago
  every good developer i know that has started using langchain stopped after realizing that they need more control than it provides. if you actually look at what is going on under the hood by looking at the requests you would probably stop using it as well.
• zby 2y ago
  I am always suspicious with frameworks. There are two reasons of that. First is that because of the inversion of control they are more rigid than libraries. This is quite fundamental - but there are cases where the trade off is totally worth it. The second one is because of how they are created - it often starts with an application which is then gradually made generic. This is good for advertising - you can always show how useful the framework with an application that uses it. But this "making it generic" is a very tricky process that often fails. It is a top down, the authors need to imagine possible uses and then enable them in the framework - while with libraries the users have much more freedom to discover them in a bottom up process. Users always have surprising ideas. There are now libraries that cover some of the features of Langchain. There is Instructor and mine LLMEasyTools for function calling, there is LiteLLM for API unification.
  • mike986 2y ago
    from the first glance of the example, it seems like the 1st use case is invoking a function (selected by chatgpt) and the 2nd use case is similar to instructor. can you comment how your library differs from instructor (what yours can do that instructor can't and vice versa?) thanks
• czechdeveloper 2y ago
  I used langchain in one project and I do regret choosing it over just writing everything over direct API. I feel their pain. It had advantage of having standardized API, so I could switch local LLM to OpenAI and just compare results in a heartbeat, but when I wanted anything out of ordinary (ie. get logprobs), there was just no way.
• wg0 2y ago
  Sorry noob question - where can I read more about this "agents" paradigm? Is one agent's output directly calling/invoking another agent? Or there's already fixed graph of information flow with each agent (I presume some prompt presets/templates like "you are an expert this only respond in that") sorts of? Also, how much success people have or had with automating the E2E tests for their various apps by stringing such agents together themselves EDIT: Typos
  • CGamesPlay 2y ago
    Fundamentally, "Agent" refers to anything that operates in an "observe-act" loop. So in the context of LLMs, an agent sees an observation (like the code base and test output) and produces an action (like a patch), and repeats.
  • zEddSH 2y ago
    > Also, how much success people have or had with automating the E2E tests for their various apps by stringing such agents themselves together? There’s a few startups in the space doing this like QA Tech in Stockholm, and others even in YC (but I forgot the name). I’m skeptical of how successful they’ll be, not just from complex test cases but things like data management and mistakingly affecting other tests. Interesting to follow just in case though, E2E is a pain!
  • pavi2410 2y ago
    I want to learn about agents too!
  • hcks 2y ago
    Don’t waste your time, it’s been around since GPT3, and had no results so far. Also notice how no frontier lab is working on it.
    • zby 2y ago
      Letting the LLM to decide what to do is a powerful technique. For example one pass RAG is very limited: https://zzbbyy.substack.com/p/why-iterative-thinking-is-crucial https://zzbbyy.substack.com/p/why-iterative-thinking-is-cruc... To make it iterative you need the cede the control to the LLM.
• ZiiS 2y ago
  The "good abstraction" has a bug; slightly undermines the argument.
• geuis 2y ago
  I built my first commercial LLM agent back in October/November last year. As a newcomer to the LLM space, every tutorial and youtube video was about using LangChain. But something about the project had that "bad code" smell about it. I was fortunate in that the person I was building the project for was able to introduce me to a few other people more experienced with the entire nascent LLM agent field and both of them strongly steered me away from LangChain. Avoiding going down that minefield ridden path really helped me out early on, and instead I focused more on learning how to build agents "from scratch" more or less. That gave me a much better handle on how to interact with agents and has led me more into learning how to run the various models independently of the API providers and get more productive results.
  • SCUSKU 2y ago
    I've only ever played around with it and not built out an app like you have, but in my experience the second you want to go off script from what the tutorials suggest, it becomes an impossible nightmare of reading source code trying to get a basic thing to work. LangChain is _the_ definition of death by abstraction.
    • emporas 2y ago
      I have read the whole source of LangChain in Rust (there are no docs anyway), and it definitely seems over-engineering. The central premise of the project, of complicated chains of prompts is not useful to many people, and not to me either. On the other hand it took some years into the web, for some web frameworks to emerge and make sense, like Ruby on Rails. Maybe in 3-4 years time, complicated chains of commands to different A.I. engines will be so difficult to get right that a framework might make sense, and establish a set of conventions. Agents, another central feature of LangChain, are not proved to be very useful as well, for the moment.
  • deleted 2y ago
    [deleted]
  • gazarullz 2y ago
    Which alternatives have you been introduced to?
• nprateem 2y ago
  Wasn't it obviously pointless from the outset? Posts like this raise questions about the technical decisions of the company more than anything else IMO. Strange they'd want to publicise making such poor decisions.
• sandGorgon 2y ago
  shameless plug - i build a JS/TS framework which tries to solve the abstraction problem. we use a json variant called jsonnet (created at google. expressive enough for kubernetes). https://github.com/arakoodev/EdgeChains/tree/ts/JS/edgechains/examples https://github.com/arakoodev/EdgeChains/tree/ts/JS/edgechain... examples of these jsonnet for react COT chains - https://github.com/arakoodev/EdgeChains/blob/ts/JS/edgechains/examples/react-chain/jsonnet/main.jsonnet https://github.com/arakoodev/EdgeChains/blob/ts/JS/edgechain... P.S. we also build a webassembly compiler that compiles this down to wasm and deploy on hardware.
• Turskarama 2y ago
  This is so common I think it could just about be a lemma: Any tool that that helps you to get up and running quicker by abstracting away boilerplate will eventually get in the way as your projects complexity increases.
• fragebogen 2y ago
  I'd challenge some of these criticisms and give my 2c on this. I've spent the last 6 months working on a rather complex chat with routes, agents, bells and whistles sort of system. Initially, time to POC was short, so I picked it to get quick at my feet. Eventually, I thought. The code base isn't enormous, I can easily rewrite it, but I'd like to see what people mean with "abstraction limiting progress" kind of statements. I've now kept building this project for another 6 months and I must say the more I work with it and understand its philosophy. It's not that complicated. The philosophy is just different from many other python projects. The LCEL pipes for example is a really nice way to think of modularity. Want to switch out one model for another? Well just import another model and replace the old. Want to parse it more strictly, exchange the parser. The fact that everything is an instance of `RunnableSerializable` is a really convenient way of making things truly modular. Want to test your pipe syncronously? Easy just use `.stream()` instead of `.astream()` and get on with it. I think my biggest hurdle was understanding how to debug and pipe components, but once I got familiarized with it, I must say it made me grow as a python dev and appreciate the structure and thought behind it. Where complexity arise is when you have a multi-step setup, some sync and some async. I've had to break some of these steps up in code, but otherwise it gives me tons of flexibility to pick and chose components. My only real complaint would be lack of documentation and outdated documentation, I'm hardly the only one, but it really is frustrating sometimes to understand what some niche module can and cannot do.
• Oras 2y ago
  The comments are good example that hype > quality. 99% of docs mentioning LangChain or showing a code example with LangChain. Wherever you look at tutorials or YouTube videos, you will see LangChain. They take the credit of being the first framework to abstract LLM calls and other features such as reading data from multiple sources (before function calling was a thing). Langchain was first, got popular, and hence for new comers they think it’s the way, until they use it.
• _pdp_ 2y ago
  We also built our own system that caters for our customers' needs.
• StrauXX 2y ago
  I don't like langchain that much either. It's not as bad as LLAmaIndex and Haystack in regards to extreme overengineering and overabstracting but it still is bad. The reason I still use Langchain is that often times I need to be able to swap out LLM service providers, embedding models and so on for clients. Thats really the only part about langchain that really works well. Btw. you don't have to actually chain langchain entities. You can use all of them directly. That makes the magic framework code issue much more tolerably as Langchain turns from a framework into a library.
  • sramam 2y ago
    Have you considered LiteLLM?
  • freezed8 2y ago
    (jerry here from llamaindex) wait do you have specific examples of "overengineering and overabstracting" from llamaindex? very open to feedback and suggestions on improvement - we've spent a lot of work making sure everything is customizable
  • darkteflon 2y ago
    I don’t think that’s fair to Llama-Index. LI is much more focused, better documented and frankly way easier to use than LangChain, with much lower - or negligible - cognitive overhead. Plus, it plays nice with most everything else. Even if you’re mostly working just with a provider SDK and other lightweight, low-dependency convenience wrappers for stuff you know you’ll almost always need (e.g. Instructor for structured output and retry), you can easily sprinkle LI in where you need it as a wrapper over common context retrieval patterns. Unlike LangChain, which is a nightmare to pull out once you’ve started working with it - LI can be cleanly excised if you change your mind.
• Havoc 2y ago
  There was a Reddit thread in langchain sub a while back basically saying exactly this (plus same comments as here)
• greo 2y ago
  I am not a fan of LangChain. And I would never use it for any of my projects. LLM is already a probabilistic component that is tricky to integrate into a solid deterministic system. An abstraction wrapper that bloats the already fuzzy component just increases the complexity for no apparent benefit.
• fforflo 2y ago
  LLM frameworks like LangChain are causing a java-fication or Python . Do you want a banana? You should first create the universe and the jungle and use dependency injection to provide every tree one at a time, then create the monkey that will grab and eat the banana.
  • 9dev 2y ago
    Well. I'm working on a product that relies on both AI assistants in the user-facing parts, as well as LLM inference in the data processing pipeline. If we let our LLM guy run free, he would create an inscrutable tangled mess of Python code, notebooks, Celery tasks, and expensive VMs in the cloud. I know Pythonista's regard themselves more as artists than engineers, but the rest of us needs reliable and deterministically running applications with observability, authorization, and accessible documentation. I don't want to drop into a notebook to understand what the current throughput is, I don't want to deploy huge pickle and CSV files alongside my source to do something interesting. LangChain might not be the answer, but having no standard tools at all isn't either.
    • dartos 2y ago
      Sounds like your LLM guy just isn’t very good. Langchain is, when you boil it down, an abstraction over text concatenation, staged calls to open ai, and calls to vector search libraries. Even without standard tooling, an experienced programmer should be able to write an understandable system that does those things.
      • 9dev 2y ago
        Fair point. The overlap of machine learning savvy, experienced engineer, and ready to work for a startup's salary in Germany just isn't too big.
      • randomdata 2y ago
        > Sounds like your LLM guy just isn’t very good. That's the central idea here. Most guys available to hire aren't. Hence why they get constrained into a framework that limits the damage they can cause. In other areas of software development the frameworks are quite mature at this point so it works well enough. This AI/LLM/whatever you want to call it area of development, however, hadn't garnered much interest until recently, and thus there isn't much in the way of frameworks to lean on. But business is trying to ramp up around it, thus needing to hire those who aren't good to fill seats. Like the parent says, LangChain may not be the framework we want, but it is the one we have, which beats letting the not-very-good developers create some unconstrained mess. If you win the lottery by snagging one of the small few good developers out there, then certainly you can let them run wild engineering a much better solution. But not everyone is so fortunate.
• JSDevOps 2y ago
  The dude on that blog is trying way too hard to look like Sam Altman which is fucking weird.
• deleted 2y ago
  [deleted]
• andix 2y ago
  Are there better abstractions? I wanted to look into Microsoft's Semantic Kernel, which seems to be a direct competitor of LangChain. Are there any other options? https://learn.microsoft.com/en-us/semantic-kernel/overview https://learn.microsoft.com/en-us/semantic-kernel/overview
• gexaha 2y ago
  that's a nice AI image with octopi
• captaincaveman 2y ago
  I think LangChain basically tried to do a land grab, insert itself between developers and LLM's. But it didn't add significant value and seemed to dress it up by adding abstractions that didn't really make sense. It was that abstraction gobbledygook smell that made me cautious.
  • iknownthing 2y ago
    Looks like they've parlayed it into some kind of business https://www.langchain.com/ https://www.langchain.com/
    • bestcoder69 2y ago
      They’ve been growth hacking the whole time pretty much, optimizing for virality. Eg integrating with every ai thing under the sun, so they could publish a seo-friendly “use gpt3 with someVecDb and lang chain” page, but for every permutation you can think. Easy for them to write since langchains abstractions are just unnecessary wrappers. They’ve also had meetups since very early on. The design seems to make langchain hard to remove since you’re no longer doing functional composition like you’d do in normal python - you’re combining Chains. You can’t insert your own log statements in between their calls so you have to onboard to langsmith for observability (their saas play). Now they have a DSL with their own binary operators :[ VC-backed, if you couldn’t guess already
• d4rkp4ttern 2y ago
  Frustration with LangChain is what led us (ex-CMU/UW-Madison researchers) to start building Langroid[1], a multi-agent LLM framework. We have been thoughtful about designing the right primitives and abstractions to enable a simple developer experience while supporting sophisticated workflows using single or multiple agents. There is an underlying loop-based orchestration mechanism that handles user interaction, tool handling and inter-agent handoff/communication. We have companies using Langroid in production. [1] Langroid: https://github.com/langroid/langroid https://github.com/langroid/langroid
• codelion 2y ago
  Many such cases. It is very hard to balance composition and abstraction in such frameworks and libraries. And LLMs being so new it has taken several iterations to get the right patterns and architecture while building LLM based apps. With patchwork (https://github.com/patched-codes/patchwork https://github.com/patched-codes/patchwork) an open-source framework for automating development workflows we try hard to avoid it by not abstracting unless we see some client usage. As a result you do see some workflows appear longer with many steps but it makes it easier to compose them.
• mark_l_watson 2y ago
  I was an early enthusiast of both LangChain and LlamaIndex (and I wrote a book using both frameworks, free to read online [1]) but I had some second thoughts when I started when I started writing LLM examples for my Common Lisp and Racket books that were framework-free, even writing simple vector data stores from scratch. This was, frankly, more fun. For my personal LLM hacking in Python, I am starting down the same path: writing simple vector data stores in NumPy, write my own prompting tools and LLM wrappers, etc. I still think that for many developers LangChain and LlamaIndex are very useful (and I try to keep my book up to date), but I usually write about things of most interest to me and I have been thinking of rewriting a new book on framework-free LLM development. [1] https://leanpub.com/langchain/read https://leanpub.com/langchain/read
• clarionbell 2y ago
  LangChain approach struck me as interesting, but I never really saw much inherent utility in it. For our production code we went with direct use of LLM runtime libraries and it was more than enough.
  • randomdata 2y ago
    We've had success with non-developers using some of the visual tools built on top of LangChain to build exploratory models in order to prove a concept. LangChain does seem well suited to providing the "backend" for that type of visual node-based modelling. Of course, once the model is proven it is handed off to developers to build something more production-worthy.
• monarchwadia 2y ago
  I'm the author of Ragged, a lightweight connector that makes it easy to connect to and work wth language models. Think about it like an ORM for LLMs --- a unified interface designed to make it easy to work with LLMs. Just wanted to plug my framework in case people are looking for an alternative to building their own connector components. https://monarchwadia.medium.com/use-openai-in-your-javascript-project-the-easy-way-with-ragged-6928d4d7f186 https://monarchwadia.medium.com/use-openai-in-your-javascrip...
• seany62 2y ago
  Glad to see I'm not the only one experiencing this. The agents framework I use is moving very fast and its not uncommon for even minor versions to break my current setup
• sc077y 2y ago
  Damn I built a RAG agent during the past 3 months and a half for my internship. And literally everyone in my company was asking me why I wasn't using llangchain or llamaindex like I was a lunatic. Everyone else that built a rag in my company used llangchain, one even went into prod. I kept telling them that it works well if you have a standard usage case but the second you need to something a little original you have to go through 5 layers of abstraction just to change a minute detail. Furthermore, you won't really understand every step in the process, so if any issue arises or you need to be improve the process you will start back at square 1. This is honestly such a boost of confidence.
  • ramoz 2y ago
    Wise perspective from an intern. The type of pragmatism we love.
  • puppymaster 2y ago
    you are heading the right direction. It's amazing to see seasoned engineers go through the mental gymnastic of justifying installing all those dependencies and arguing about vector db choices when the data fit in ram and the swiss knife is right there: np.array
  • paraph1n 2y ago
    Could someone point me towards a good resource for learning how to build a RAG app without llangchain or llamaindex? It's hard to find good information.
    • kolinko 2y ago
      You can start by reading up about how embeddings work, then check out specific rag techniques that people discovered. Not much else is needed really.
    • bestcoder69 2y ago
      openai cookbook! Instructor is a decent library that can help with the annoying parts without abstracting the whole api call - see it’s docs for RAG examples.
• deleted 2y ago
  [deleted]
• jostmey 2y ago
  Learning LangChain is effort, but not as much as truly understanding deep learning, so you learn LangChain and it feels like progress, when it may not be
• bratbag 2y ago
  I made the same choice for our stack last year. We initially had problems diagnosing issues inside LangChain and were hitting weird issues with some elements of function calling, so we experimented with a manual reconstruction of exactly what we needed and it was faster, more resilient and easier to maintain. I can see how switching models might be easier using LangChain as an abstraction layer, but that doesn't justify making everything else harder.
• whitej125 2y ago
  I used LangChain early on in it's life. People crap on their documentation but at least at that point in time I had no problem with it. I like reading source code so I'd find myself reading the code for further comprehension anyway. In my case - I'm a seasoned engineer who was discovering LLMs and thought LangChain suited that way of learning pretty well. When it came to building anything real beyond toy examples, I quickly outgrew it and haven't looked back. We don't use any LC in production. So while LC does get a lot of hate from time to time (as you see in a lot of peers posts here) I do owe them some credit for helping bridge my learning of this domain.
• hwchase17 2y ago
  Hi HN, Harrison (CEO/co-founder of LangChain) here, wanted to chime in briefly I appreciate Fabian and the Octomind team sharing their experience in a level-headed and precise way. I don't think this is trying to be click-baity at all which I appreciate. I want to share a bit about how we are thinking about things because I think it aligns with some of the points here (although this may be worth a longer post) > But frameworks are typically designed for enforcing structure based on well-established patterns of usage - something LLM-powered applications don’t yet have. I think this is the key point. I agree with their sentiment that frameworks are useful when there are clear patterns. I also agree that it is super early on and super fast moving field. The initial version of LangChain was pretty high level and absolutely abstracted away too much. We're moving more and more to low level abstractions, while also trying to figure out what some of these high level patterns are. For moving to lower level abstractions - we're investing a lot in LangGraph (and hearing very good feedback). It's a very low-level, controllable framework for building agentic applications. All nodes/edges are just Python functions, you can use with/without LangChain. It's intended to replace the LangChain AgentExecutor (which as they noted was opaque) I think there are a few patterns that are emerging, and we're trying to invest heavily there. Generating structured output and tool calling are two of those, and we're trying to standardize our interfaces there Again, this is probably a longer discussion but I just wanted to share some of the directions we're taking to address some of the valid criticisms here. Happy to answer any questions!
  • causal 2y ago
    I appreciate that you're taking feedback seriously, and it sounds like you're making some good changes. But frankly, all my goodwill was burnt up in the days I spent trying to make LangChain work, and the number of posts I've seen like this one make it clear I'm not the only one. The changes you've made might be awesome, but it also means NEW abstractions to learn, and "fool me once..." comes to mind. But if you're sure it's in a much better place now, then for marketing purposes you might be better off relaunching as LangChain2, intentionally distancing the project from earlier versions.
    • ctxc 2y ago
• iknownthing 2y ago
  I tried LangChain a while ago for a RAG project. I liked how I could just plug into different vector stores to try them out. But I didn't understand the need for the abstractions around the API calls. It's not that hard to just call these APIs directly and its not that hard to create whatever prompt you'd like.
• zackproser 2y ago
  Here's a real world example of a custom RAG pipeline built with Langchain https://zackproser.com/chat https://zackproser.com/chat I did a full tutorial with source code that's linked at the top of that page ^ Fwiw I think it's a good idea to build with and without Langchain for deeper understanding.
• hcks 2y ago
  LangChain is a critical thinking test and orgs using it are ngmi
• bastawhiz 2y ago
  Genuine question: can someone point me to a use case where langchain makes the problem easier to solve than using the openai/anthropic/ollama SDKs directly? I've gotten a lot of advice to use langchain, but the docs haven't really shown me how it simplifies the task, or at least not more than using an SDK directly. I really want to at least understand when to use this as a tool but so far I've been failing to figure it out. Some of the things that I tried applying it for: - Doing a kind of function calling (or at least, implementing the schema validation) for non-gpt models - parsing out code snippets from responses (and ignoring the rest of the output) - Having the output of a prompt return as a simple enum without hallucinations - process a piece of information in multiple steps, like a decision tree, to create structured output about some text (is this a directory listing or a document with content? What category is it? Is it NSFW? What is the reason for it being NSFW?) Any resources are appreciated
  • starik36 2y ago
    It makes it simple (and uniform) to switch providers.
    • bastawhiz 2y ago
      Is that really it?
    • localfirst 2y ago
      theres already solutions for this but even this i feel like is a wasted effort unless you have the token volume to justify high availability
  • mikeqq2024 2y ago
    Mind tell what kind of scenario you are tring to solve?
    • bastawhiz 2y ago
      I literally just want to know what use cases langchain serves. I've built four or five different applications at this point, and it was easy to enough to use various SDKs. Where does langchain come in?
• xyst 2y ago
  Never been a fan of ORM for databases. So why would that change with AI/LLM “prompt engineering”? Author confirms my point.
• sabrina_ramonov 2y ago
  You used langchain for a simple replacement of OpenAI API calls — of course it will increase complexity for no benefit. The benefits of langchain are: (1) unified abstraction across multiple different models and (2) being able to plug this coherently into one architecture. If you’re just calling some OpenAI endpoints, then why use it in the first place?
• ricklamers 2y ago
  FWIW I think LangChain has evolved a lot and is a nice time saver once you figure out the patterns it uses. The LangSmith observability is frankly fantastic to quickly get a sense of how your expected LLM flow engineering ends up working out in practice. So much FUD here, unwarranted IMO. Don’t forget, reading code is harder than writing it, doesn’t warrant throwing out the baby with the bath water. Don’t fall for NIH :) Haven’t had issues running in prod recently either since they’ve matured their packaging with core/community/partner etc. For agentic use cases look at LangGraph for a cleaner set of primitives that give you the amount of control needed there.
• cyounkins 2y ago
  Is there a lighter weight solution that abstracts the interfaces so I can swap GPT4 with Claude, including function calling?
• resource_waste 2y ago
  LangChain tutorials be like: Go to foo_website and put your credit card to get their API. Then go to bar_website, get their API. Then go to yayeee_website and get their API. Then go to... But unironically. I actually counted 4 APIs in some 'how to' article. I ended up DIYing that with 0 APIs. Whoever got into langchain planted their APIs. That is why it sucks.
• dmezzetti 2y ago
  An alternative is using txtai (https://github.com/neuml/txtai https://github.com/neuml/txtai). It's lightweight and works with both local and remote LLMs. Here is an example article that shows how to use OpenAI calls with txtai: https://neuml.hashnode.dev/rag-with-llamacpp-and-external-api-services https://neuml.hashnode.dev/rag-with-llamacpp-and-external-ap...
• createaccount99 2y ago
  A lot of competition in the field, and just about all of them (llamaindex/autogpt/langchain/others?) appear as "build sdk, build saas on top" type of products. Curious thing, but I'd rather not partake myself.
• isaacphi 2y ago
  I had the same impression after working through the LangChain tutorials. The one thing I'd like to ask about is Observability. LangChain has some tools around observability that seem genuinely useful to me, and specific to working with LLMs. Are there ways to use only these tools, or alternative observability tools you recommend for working with LLMs?