10 ms·
Re-implementing LangChain in 100 lines of code
- sia_m 3y agoThere are a few ways to use Langchain. Firstly, the docs are a mess. What I personally did, I followed the notebook from OpenAI cookbook on embedding a code base, and one on embedding the docs, and querying over that with GPT-4. After a while of doing that, I realised like many others that it's too high of an abstraction. In the end I think you're better off just looking at their source code, and just looking at how they've implemented the stuff in normal python and then adapting it for your own needs.
- sia_m 3y agoThe same goes for gpt-index.
- justanotheratom 3y agoGiven that the company has $200 million valuation, that is $2 million per line of code! just kidding. Still, I would like to understand $200 million valuation of langchain.ai.
- throwaway888abc 3y agoIt's a great momentum to dominate eco system. Heard about Docker (that was more than http fetch wrapper) ?
- ukuina 3y agoI think we can safely say this did not end well for Docker.
- nextworddev 3y agoLangchain is kinda like web3 in how it got memed on Twitter basically to adoption
- reducesuffering 3y agoVC's are in full blown FOMO mode for things they barely understand. Even the engineering backgrounds are pretty lost; imagine the finance backgrounds that have barely wrote a lick of code.
- objektif 3y agoIf you think VC people understand finance I have some bad news for you. Just wanting 10x is not understanding finance.
- ColinEberhardt 3y ago$2 million per line? that is one of the most productive days of coding I have ever had! I'll start my VC fuding round tomorrow ;-)
- rahimnathwani 3y agoRelated comments about ReAct: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=https%3A%2F%2Farxiv.org%2Fabs%2F2210.03629&sort=byPopularity&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
- cube2222 3y agoYeah, the basics of LangChain are fairly simple, and reimplementing a loop like that in Go, including tool usage, was very straightforward when I was writing Cuttlefish[0] (a toy desktop chat app for ChatGPT that can use stuff like your local terminal or Google). The magic in LangChain, though, is the ecosystem. I.e. they have integrations with tons of indexes, they have many tool implementations, etc. This is the real value of LangChain. The core ReAct loop is quite trivial (as this article demonstrates). [0]: https://github.com/cube2222/cuttlefish https://github.com/cube2222/cuttlefish
- loveparade 3y agoAm I the only one who is not convinced by the value proposition of langchain? 99% of it are interface definitions and implementations for external tools, most of which are super straightforward. I can write integrations for what my app needs in less than an hour myself, why bring in a heavily opinionated external framework? It kind of feels like the npm "left-pad" to me. Everyone just uses it because it seems popular, not because they need it.
- gumby 3y ago> I can write integrations for what my app needs in less than an hour myself, Or just ask ChatGPT to do it... Joking aside, I think 'npm "left-pad"' describes it perfectly.
- Spivak 3y agoLangchain is absolutely perfect though, it's bad enough that you'll be driven to write something better out of pure frustration but gives you enough good ideas and breadcrumbs to actually do it. It's probably the best on-ramp for "practical uses of llms" because it scratches just the right developer itch.
- 3y ago
- fchief 3y agoWe ported the core of LangChain to Ruby, and while it is way more that 100 lines, I would give similar feedback as the author. Here is the repo if anyone is interested https://github.com/BoxcarsAI/boxcars https://github.com/BoxcarsAI/boxcars
- lxe 3y agoI've been working with langchain and llamaindex and did notice that it's a pretty hefty abstraction on top of pretty simple concepts and I also eventually ended up dropping both and simply write the underlying code without the framework on top.
- deleted 3y ago[deleted]
- KevinBenSmith 3y agoPretty much sums up my experience as well.
- zomglings 3y agoDitto. But with faiss and openai api.
- fbrncci 3y agoI work with Langchain on a daily basis now, and so often I find myself asking; do I really need a whole LLM framework for this? At this point, the assistant I am writing, will likely be more stable rewritten in pure Python. The deeper and more complex the application becomes, the more of a risk Langchain seems to become to keeping it maintainable. But even at less complex levels, if I want to do this: 1. Have a huge dataset of documents. 2. Want to ask questions and have an LLM chat conversation based on these documents. 3. Be able to implement tools like math, wiki or Google search on top of the retrieval. 4. Implement memory management for longer conversations. Its still a lot more straightforward to maintain it in Python. The only thing where it becomes interesting is having agents execute async, which is not that easy replicate, but at the moment agents are not that helpful. Not trying to diss Langchain too much here, because its such an awesome framework, but I can't help seeing past it other than just being a helpful tool to understand LLM's and LLM programming for now.
- theturtletalks 3y agoWon’t ChatGPT eventually eat your lunch? Once ChatGPT allows uploading documents (embeddings), what good will your app be? The tools you’re talking about like math, wiki, or search are already built as plugins on ChatGPT. I see so many AI apps being built, but I think ChatGPT will be general enough to cover 85-90% of use-cases using the chat UI.
- fbrncci 3y agoIt's more or less an advanced personal project to stay on top of the LLM learning curve, rather than just being exposed to news and press releases. I also have an appetite for further wrapping my mind around all of this. I already work in the AI space as web-developer on the B2B & enterprise side of things. My opinion here is that there are going to be loads of use-cases and necessary plugins, which for privacy, legal and security reasons need a proprietary solution and won't be able to interface with any third party APIs, plugins or frameworks.
- theturtletalks 3y ago
- rcme 3y agoLangChain has been so frequently discussed that I thought it must be this amazing piece of software. I was recently reading about vector databases and how they can be used to provide context to LLMs. I came across a LangChain class called RetrievalQA, which takes in a vector database and a question and produces and answer based on documents stored in the vector db. My curiosity was piqued! How did it work? Well... it works like this: prompt_template = """Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer. {context} Question: {question} Helpful Answer:""" My sense of wonder was instantly deflated. "Helpful Answer:". Seriously? I think LLMs are cool, but this made me realize people are just throwing darts in the dark here.
- theptrk 3y agoI tried to make a youtube video exploring the code and it was fairly short https://www.youtube.com/watch?v=Joby-58DuBE https://www.youtube.com/watch?v=Joby-58DuBE. I think if the prompts were put front and center in the documentation it would be clear up a lot of mystery.
- nextworddev 3y agoIt got a lot of traction on non-coders thinking it’s doing some magic, but if you read the source it boils down to some brittle prompts and lots of Python class boilerplate. There’s some useful parts though
- rcme 3y agoJust out of curiosity, what are the useful parts?
- nextworddev 3y agoIt’s good for building vector indices without worrying about writing adapters to milvus, pinecone, qdrant etc separately - in case you want to switch out later
- 19h 3y ago
- okhat 3y agoThere’s always DSP for those who need a lightweight but powerful programming model — not a library of predefined prompts and integrations. It’s a very different experience from the hand-holding of LangChain, but it packs reusable magic in generic constructs like annotate, compile, etc that work with arbitrary programs. https://github.com/stanfordnlp/dsp/ https://github.com/stanfordnlp/dsp/
- barking_biscuit 3y agoDidn't know about this. Looks promising!
- ukuina 3y agoI cannot praise Deepset Haystack enough for how simple they make things compared to LangChain, between the Preprocessor, the Reader/Retriever, and the PromptNode - the APIs, docs, and tutorials are quite easy to modify to your use-case. Not affiliated, just a happy defector from LangChain.
- iamflimflam1 3y agoIs it open source? Wasn’t clear from the website.
- KevinBenSmith 3y agoAs someone who has created several LLM-based applications running in production, my personal experience with langchain has been that it is too high of an abstraction for steps that in the end are actually fairly simple. And as soon as you want to slightly modify something to better accomodate your use-case, you are trapped in layers & layers of Python boiler plate code and unnecessary abstractions. Maybe our llm applications haven’t been complex enough to warrent the use of langchain, but if that’s the case, then I wonder how many of such complex applications actually exist today. -> Anyways, I came away feeling quite let down by the hype. For my own personal workflow, a more “hackable” architecture would be much more valuable. Totally fine if that means it’s less “general”. As a comparison, I remember the early days of HugginfaceTransformers where they did not try to create a 100% high-level general abstraction on top of every conceivable Neural Network architecture. Instead, each model architecture was somewhat separate from one another, making it much easier to “hack” it.
- 19h 3y agoComparing Langchain to Hugging Face Transformers is apples and oranges. One is for research, one is for production. Production ML requires more abstraction, not less.
- crazyedgar 3y agoI disagree. Production systems don't need to be full of AbstractSingletonProxyFactoryBeans which is basically what LangChain is. For example, Linux certainly isn't.
- dpbrinkm 3y ago>someone who has created several LLM-based applications running in production Come and talk about what you are doing and challenges of it at our LLM in production virtual conference? https://home.mlops.community/home/events/llm-in-prod-part-ii-2023-06-20 https://home.mlops.community/home/events/llm-in-prod-part-ii...
- valyagolev 3y agowe're still in the stage of LLM adoption when we can have "eye-opening" simple discoveries weekly. Langchain has momentum because of this, as a library of simple ideas. this period will end, and if they don't figure out the next step they're gone
- adityapurwa 3y agoI got the chance to try Langchain as part of a hiring process. I was already having my eye on it for a personal projects though. The moment I tried it and went through the docs, the entire abstraction feels weird for me. I know a bit here and there about LLM, but Langchain make me feels like Im learning something entirely new. How agent and tools work and how to write one wasnt straightforward from the docs, and the idea of having an AI attach itself to an eval or writing its own error/hallucination-prone API request based on a docs doesnt give me a lot of confidence. The hiring assignment specifically mentioned to use Langchain thought, so I did. But just as a glorified abstraction to call GPT and parses the NL output as JSON. I did the actual API call, post-processing, etc. manually. Which I have granular control over it. Also cheaper in terms of token usages. You could say I ended writing my own agent/tool that doesnt exactly match Langchain specifications but it works. I guess Langchain had its use case. But it feels pretty weird to use for me.
- triyambakam 3y agoWhat type of position was the assessment for?
- adityapurwa 3y agoIt was a senior full stack position for a company that build products for researchers.
- minimaxir 3y agoLangChain and the ReAct paper that helped codify the implementation are both less than a year old. A hiring assignment suggesting it is…weird.
- adityapurwa 3y agoIts for a company that helps researchers, so I guess that’s why.
- zyang 3y agoI'm glad I wasn't the only one that felt Langchain had a ton of redundant abstractions engineered to gain clout for vc money. Here is an example: AnalyzeDocumentChain[1] just wraps RecursiveCharacterTextSplitter[2]. It serves no real purpose except padding the api doc. [1] https://js.langchain.com/docs/modules/chains/other_chains/analyze_document https://js.langchain.com/docs/modules/chains/other_chains/an... [2] https://js.langchain.com/docs/modules/chains/other_chains/summarization https://js.langchain.com/docs/modules/chains/other_chains/su...
- crazyedgar 3y agoIt's worse than that. The documentation is a confusing mess that completely omits the explanation of key default parameters and details. And the abstractions are horrendously brittle. And difficult to fix, because there are too many layers. The best use of LangChain is probably just looking at the included prompts in the source code for inspiration.
- saulpw 3y agoI also was underwhelmed by langchain, and started implementing my own "AIPL" (Array-Inspired Pipeline Language) which turns these "chains" into straightforward, linear scripts. It's very early days but already it feels like the right direction for experimenting with this stuff. (I'm looking for collaborators if anyone is interested!) https://github.com/saulpw/aipl https://github.com/saulpw/aipl
- nestorD 3y agoFor me Langchain is glue code between a lot of commonly used LLM building blocks and prompts. It is great to get a prototype 80% of the way there fast in order to validate an idea or run something short lived. I suspect that, if you want to go further (simpler code, better control message length, reliability, etc), you will be better served by implementing the functionality you need yourself.
- crazyedgar 3y agoThere's no suspicion. Our experience shows that for the remaining 20% of work, LangChain actually gets in the way and must eventually be removed.
- lynx23 3y agoHmm, this is great food for thought! I am working on a Haskell based REPL for GPT, called GPTi[1] which might benefit from this approach. [1] https://github.com/mlang/gpti https://github.com/mlang/gpti
- havercosine 3y agoPersonal experience: was using LangChain and its output parsers for getting structured data. It was having a very high error rates (probably prompt was becoming too long and confusing). But it is a just prompt + some parsing logic. Replaced it with straight asking openAI GPT for json that matches some Rust struct / Python data-class. The errors went down and got one extra dependency out from the project. Tried to use its self hosted embeddings but the implementation (strangely) seemed tied to something called Run-house. Not to belittle the library, but most of it is a very thin wrapper classes that reek of premature abstraction, couple with hit-n-miss docs. At this point, given the hype, it is primarily optimized for cooking up demoes quickly. But not sure if the valuations or production use is justified.
- shri_krishna 3y agoFor the calculator tool I suggest instead to just generate Javascript as an output with temperature set to 0 (system prompt set to something along the lines of: "Generate native Javascript code only. Don't provide any explanations. Don't import any extraneous libraries") and then eval that Javascript code in a VM. Deno is a good candidate for this as it has good security settings with access to filesystem and network turned off by default. You can use something like deno-vm [1] to execute it separate from your running process too. Setting GPT-4 as model works even better. I have seen it perform better than Wolfram Alpha in many cases so I am wondering why OpenAI chose to integrate with Wolfram Alpha for this. GPT-4 was able to solve some really complex math problems I threw at it. [1]: https://www.npmjs.com/package/deno-vm https://www.npmjs.com/package/deno-vm
- zbyforgotpass 3y agoThe problem with all these new fields is that the first code that gets popular is from people who are good at marketing not those who are good at programming.
- d4rkp4ttern 3y agoI’ll repeat what I’ve said in another thread the other day — To put together a basic question/answer demo that didn't quite fit the LangChain templates, I had to hunt a bunch of doc pages and and cobble together snippets from multiple notebooks. Sure, the final result was under 30 lines of code, BUT: It uses fns/classes like `load_qa_with_sources_chain` and `ConversationalRetrievalChain`, and to know what these do under the hood, I tried stepping into the debugger, and it was a nightmare of call after call up and down the object hierarchy. They have verbose mode so you can see what prompts are being generated, but there is more to it than just the prompts. I had to spend several hours piecing together a simple flat recipe based on this object hierarchy hunting. It very much feels like what happened with PyTorch Lightning -- sure, you can accomplish things with "just a few lines of code", but now everything is in one giant function, and you have to understand all the settings. If you ever want to do something different, good luck digging into their code -- I've been there, for example trying to implement a version of k-fold cross-validation: again, an object-hierarchy mess.
- convexfunction 3y agoIf you know little about prompt engineering and want to throw together a demo of something that kind of works extremely quickly, or experiment with an LLM agent exactly as it's defined in some paper, LangChain is pretty useful. If you want to develop a real LLM application, you're probably better off skipping the library completely, or at least fully understand each abstraction to make sure it does everything you want before you decide you want to incorporate it.