4 ms·
Could someone in the know compare this with LangChain (https://github.com/hwchase17/langchain https://github.com/hwchase17/langchain)?
by j0e1 4y ago
Could someone in the know compare this with LangChain (https://github.com/hwchase17/langchain https://github.com/hwchase17/langchain)?
- teruakohatu 4y agoLooks like the same general idea: https://github.com/deepset-ai/haystack https://github.com/deepset-ai/haystack
- antti909 4y agoSee above - Haystack started a few years ago as a result of us working with some large enterprise clients on implementing extractive QA at scale. Now evolving to also allow the backend builders to mimic what's available from, e.g. OpenAI+plugins, but with their own set of models, and being able to mix&match best available components and technology.
- nextworddev 4y agoMost of the core ideas came from a paper called React, they all kind of riff on the idea of self-inspection / introspection to augment the context or plan action
- antti909 4y agoFor the Agents? Yes, indeed. Referred in the article.
- ukuina 4y agoI would consider Haystack to be the more batteries-included, easier to use (but harder to customize) of the two. They have a good emphasis on local model use.
- antti909 4y agoThanks :) Working on it. Re local models - indeed, all started with using the Transformer models for extractive QA and semantic search. With the Promptnode, and/or the Agents it's also now possible to combine local models/pipelines & 'LLMs' freely.
- antti909 4y agoLangChain is very cool tho :))
- antti909 4y agoHaystack has been around for a while now, and we've been mostly specializing in the extractive QA. The focus has been indeed on making the use of local Transformer models most easy and convenient for a backend application builder. You can build very reliable and sometimes quite elaborate NLP pipelines with Haystack (e.g., extractive or generative QA, summarization, document similarity, semantic search, FAQ-style search, etc. etc.) with either Transformer models, LLMs, or both. With the Agents you can also put an Agent on top of your pipelines and use a prompt-defined control to find the best underlying tool and pipeline for the task. Haystack has always included all the necessary 'infrastructure' components - pre-processing, indexing, several document stores to choose from (ES/OS, Pinecone, Weavite, Milvus, now Qdrant, etc.) and the means to evaluate and fine-tune Transformer models.
- d4rkp4ttern 4y agoThanks for clarifying. The support for local LLMs seems very interesting — would a haystack agent call out to a separately “running” self-hosted LLM via an API (REST, etc) or would it need to actually load up the model and directly query it (e.g model.generate(<prompt>) ) ? Also it seems like the functionality of haystack subsumes those of langchain and llama-index (fka GPT-index) ?
- antti909 4y agoTo be precise - I don't think I'm saying 'local LLMs' above :) But technically possible, I guess, just hasn't been part of what's officially available. (There are also licensing issues still.) To answer your question about the APIs - the Agent itself queries OpenAI via REST to break the prompt down into tasks, then works with the underlying tools/pipelines using Python API (and then, e.g., a Transformer model that's part of the pipeline has to be 'loaded' into a GPU). Part of those pipelines might be using Promptnode (that can work with hosted LLMs via REST, but could also work with a local LLM). Re 'subsume' - well, that depends :) But arguably, you can build an NLP Python backend with Haystack only, of course.. Regardless of how complex your underlying use case is, or whether it's extractive, generative or both.
- tholor 4y ago