Prompting is the most difficult part and that is LLM-bound and not very transferable. One can make all kinds of prompts, might want to glue them into some semi-

Prompting is the most difficult part and that is LLM-bound and not very transferable. One can make all kinds of prompts, might want to glue them into some semi-intelligent state machine making its own decisions (=agent), run multi-task prompts (a single LLM call returns multiple answers to various NLP tasks). I guess LangChain helps with the semi-intelligent part but its problem (and the problem of LlamaIndex) is that they were originally written for GPT-2/3 and then new LLMs obliterate the need to do their specific chaining. Running basic prompt templates in both almost always guarantees higher latency for a minimal gain in accuracy. I guess it's all about whether you believe the most recent LLMs to be good enough to do their own adequate decision making inside their own hallucinations, or if you need to enforce it externally. If the latter, you use LangChain or LlamaIndex. If the former, you rely on OpenAI functions/Claude 2 iterative prompting with minimal Python glue. LangChain and LlamaIndex also have some nice functions like document imports, RAG, re-ranking but one can simply copy the corresponding code and use it standalone without the rest of the library.