6 ms·

What's the difference between LangChain, llama indexand others like autollm?

31 points by johhns4 3y ago

Do you use any of these and if so which one is the best to use? I know there is some skepticism around using these wrappers in the first place.

eigenvalue 3y ago
I found all of them to be quite bloated and annoying to use directly, which is why I made my own FastAPI based one, Swiss Army Llama. I’m obviously biased, but I far prefer it: https://github.com/Dicklesworthstone/swiss_army_llama https://github.com/Dicklesworthstone/swiss_army_llama
- rocket-science 3y ago
  Love the name! To add to it, we created rocket-röschti (rocketrösti for anyone not familiar with Swiss German) for very similar reasons. Also, back when we started there was not a lot to work with. But obviously now, it is also a good idea looking at AutoGen. https://github.com/rocket-science-ch/rocketrosti https://github.com/rocket-science-ch/rocketrosti
ramblerman 3y ago
Without specifying what you want to do, it's a bit like asking which kitchen knife you should buy. I think they are all relatively convoluted but can help with some boilerplate, but most of the magic is the llm itself. Setting up a vector search is not that complex, but langchain has many premade connectors for a POC. I personally haven't seen many actual production usages (or heard of) people really using the auto part (i.e. llm -> llm -> llm feedback loop). It looks cool in theory but has many issues that make it impractical and too expensive imo.
hm-nah 3y ago
Add AutoGen to the list. It’s quite new, but promising.
dmezzetti 3y ago
Adding txtai to the list for consideration. Couple relevant links. 1. https://neuml.hashnode.dev/custom-api-endpoints https://neuml.hashnode.dev/custom-api-endpoints 2. https://github.com/neuml/txtai https://github.com/neuml/txtai
warthog 3y ago
Yeah - had problems with llama-index trying to run it. Langchain has been good so far but looking forward to experimenting with auto-gen for agents particularly. They are racing to be the one that rules it all however the documentation gets quite convoluted. Check their Twitter, they retweet a new framework or use-case or connection every second. That makes it very difficult for me to understand what are a couple best frameworks to set these things up
- yding 3y ago
  What were the issues you ran into with running LlamaIndex?
d4rkp4ttern 3y ago
You might want to check out the Langroid Multi-Agent LLM app framework https://GitHub.com/Langroid/Langroid https://GitHub.com/Langroid/Langroid We’ve prioritized code clarity and kept abstractions to a minimum. I recently posted about a numbering trick Langroid uses that is massively cheaper and faster than LangChain for the “Relevance Extraction” phase in a RAG pipeline: https://www.linkedin.com/posts/pchalasani_rag-llm-langchain-activity-7124823472869953537-9T1T https://www.linkedin.com/posts/pchalasani_rag-llm-langchain-...
screye 3y ago
Langchain is a wrapper. If you adopt the wrapper, it allows you to use a bunch of (1st party) and community implementations of LLM inference seamlessly. It allows easy integration, the thinness of its wrapper and velocty means that consistency, stability and overall architectural opinion is kept to a minimum. (For better or so worse). Langchain does everything, but it doesn't do anything that well. Think early huggingface. Llamaindex is similar but more opinionated. Some might find it better architected. Harder to get up and running and has fewer integrarions. Txt.io is meant to be a library with a small set of high demand features. Stable out of the box. But, it is hard to get it to work for you exactly how you need it. Less customizable. Solid implementation for RAG, QA, simple chaining style approaches. Haven't used the others. I really like langchain. It's wrappers are thin, so you customize whichever component you want to make it work for you. Its 101-> 201 step is really annoying though. Lots of conflicting documentation and need to spend a good deal of time understanding many (unnecessary to some) layers for even the most simple usecasd.
- gtirloni 3y ago
  How can it be a thin wrapper and also have many unnecessary layers for even the most simple use cases?
v3ss0n 3y ago
Langhchain is bad code quality , badly optimized , rushed , bad quality of code version of Haystack. Haystack is proper framework with proper pipelines and Q/A architecture architecture . If you had tried and moving to production you should consider haystack. TxtAI is also a proper framework with a lot easier interface. THey all do the same job , quality differs.
- lagrange77 3y ago
  > bad code quality Got some examples for this claim?
fbrncci 3y ago
The funny thing about Langchain is (and a lot of developers will attest to this), is the more "serious" your application becomes, or your app becomes production ready. The more you start to understand what Langchain is doing, and you will start to rewrite the custom classes to your own ones, and the custom logic to do your own stuff. Until at some point you realize, that you don't need a wrapper for an API, and can do everything with about 10% of the code. Langchain is like a backwards 101 course into APIs for LLM's, where you start out trying to learn a new framework, but through debugging sessions end up understanding whats really being done.

31 points by johhns4 3y ago

Do you use any of these and if so which one is the best to use? I know there is some skepticism around using these wrappers in the first place.

eigenvalue 3y ago

I found all of them to be quite bloated and annoying to use directly, which is why I made my own FastAPI based one, Swiss Army Llama. I’m obviously biased, but I far prefer it: https://github.com/Dicklesworthstone/swiss_army_llama https://github.com/Dicklesworthstone/swiss_army_llama

rocket-science 3y ago
Love the name! To add to it, we created rocket-röschti (rocketrösti for anyone not familiar with Swiss German) for very similar reasons. Also, back when we started there was not a lot to work with. But obviously now, it is also a good idea looking at AutoGen. https://github.com/rocket-science-ch/rocketrosti https://github.com/rocket-science-ch/rocketrosti

ramblerman 3y ago

Without specifying what you want to do, it's a bit like asking which kitchen knife you should buy. I think they are all relatively convoluted but can help with some boilerplate, but most of the magic is the llm itself. Setting up a vector search is not that complex, but langchain has many premade connectors for a POC. I personally haven't seen many actual production usages (or heard of) people really using the auto part (i.e. llm -> llm -> llm feedback loop). It looks cool in theory but has many issues that make it impractical and too expensive imo.

hm-nah 3y ago

Add AutoGen to the list. It’s quite new, but promising.

dmezzetti 3y ago

Adding txtai to the list for consideration. Couple relevant links. 1. https://neuml.hashnode.dev/custom-api-endpoints https://neuml.hashnode.dev/custom-api-endpoints 2. https://github.com/neuml/txtai https://github.com/neuml/txtai

warthog 3y ago

Yeah - had problems with llama-index trying to run it. Langchain has been good so far but looking forward to experimenting with auto-gen for agents particularly. They are racing to be the one that rules it all however the documentation gets quite convoluted. Check their Twitter, they retweet a new framework or use-case or connection every second. That makes it very difficult for me to understand what are a couple best frameworks to set these things up

yding 3y ago
What were the issues you ran into with running LlamaIndex?

d4rkp4ttern 3y ago

You might want to check out the Langroid Multi-Agent LLM app framework https://GitHub.com/Langroid/Langroid https://GitHub.com/Langroid/Langroid We’ve prioritized code clarity and kept abstractions to a minimum. I recently posted about a numbering trick Langroid uses that is massively cheaper and faster than LangChain for the “Relevance Extraction” phase in a RAG pipeline: https://www.linkedin.com/posts/pchalasani_rag-llm-langchain-activity-7124823472869953537-9T1T https://www.linkedin.com/posts/pchalasani_rag-llm-langchain-...

screye 3y ago

Langchain is a wrapper. If you adopt the wrapper, it allows you to use a bunch of (1st party) and community implementations of LLM inference seamlessly. It allows easy integration, the thinness of its wrapper and velocty means that consistency, stability and overall architectural opinion is kept to a minimum. (For better or so worse). Langchain does everything, but it doesn't do anything that well. Think early huggingface. Llamaindex is similar but more opinionated. Some might find it better architected. Harder to get up and running and has fewer integrarions. Txt.io is meant to be a library with a small set of high demand features. Stable out of the box. But, it is hard to get it to work for you exactly how you need it. Less customizable. Solid implementation for RAG, QA, simple chaining style approaches. Haven't used the others. I really like langchain. It's wrappers are thin, so you customize whichever component you want to make it work for you. Its 101-> 201 step is really annoying though. Lots of conflicting documentation and need to spend a good deal of time understanding many (unnecessary to some) layers for even the most simple usecasd.

gtirloni 3y ago
How can it be a thin wrapper and also have many unnecessary layers for even the most simple use cases?

v3ss0n 3y ago

Langhchain is bad code quality , badly optimized , rushed , bad quality of code version of Haystack. Haystack is proper framework with proper pipelines and Q/A architecture architecture . If you had tried and moving to production you should consider haystack. TxtAI is also a proper framework with a lot easier interface. THey all do the same job , quality differs.

lagrange77 3y ago
> bad code quality Got some examples for this claim?

fbrncci 3y ago

The funny thing about Langchain is (and a lot of developers will attest to this), is the more "serious" your application becomes, or your app becomes production ready. The more you start to understand what Langchain is doing, and you will start to rewrite the custom classes to your own ones, and the custom logic to do your own stuff. Until at some point you realize, that you don't need a wrapper for an API, and can do everything with about 10% of the code. Langchain is like a backwards 101 course into APIs for LLM's, where you start out trying to learn a new framework, but through debugging sessions end up understanding whats really being done.