3 ms·
LangChain has been so frequently discussed that I thought it must be this amazing piece of software. I was recently reading about vector databases and how they
by rcme 3y ago
LangChain has been so frequently discussed that I thought it must be this amazing piece of software. I was recently reading about vector databases and how they can be used to provide context to LLMs. I came across a LangChain class called RetrievalQA, which takes in a vector database and a question and produces and answer based on documents stored in the vector db. My curiosity was piqued! How did it work? Well... it works like this:
prompt_template = """Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer.
{context}
Question: {question}
Helpful Answer:"""
My sense of wonder was instantly deflated. "Helpful Answer:". Seriously? I think LLMs are cool, but this made me realize people are just throwing darts in the dark here.
- theptrk 3y agoI tried to make a youtube video exploring the code and it was fairly short https://www.youtube.com/watch?v=Joby-58DuBE https://www.youtube.com/watch?v=Joby-58DuBE. I think if the prompts were put front and center in the documentation it would be clear up a lot of mystery.
- nextworddev 3y agoIt got a lot of traction on non-coders thinking it’s doing some magic, but if you read the source it boils down to some brittle prompts and lots of Python class boilerplate. There’s some useful parts though
- rcme 3y agoJust out of curiosity, what are the useful parts?
- nextworddev 3y agoIt’s good for building vector indices without worrying about writing adapters to milvus, pinecone, qdrant etc separately - in case you want to switch out later
- 19h 3y agoThe class structure provides a clean API for building on, even if the internals are basic. With some refinement, it could be a good starting point for more advanced models.
- fzliu 3y agoLangChain is meant to reduce/remove the amount of boilerplate code needed to build a lot of applications with LLMs. I see your point, but I still think LangChain is useful for a particular segment of early stage developers.
- fbrncci 3y agoBut wait, there is more! In Langchain you can build constitutional chains ontop of your chains, to validate if the answer was really helpful, by doing just one more API call, with a new prompt, asking if the answer answered the question based on the initial prompt, in an helpful way! And if it didn't, revise the answer with another API call to be more helpful! And then you can chain these chains with even further API calls until you went through as much prompts you think are necessary to answer a single sentence question (What is the weather today on the moon?).
- 19h 3y agoIf it makes you feel any better, the researchers building these things don't really know what they're doing either. They just throw data and compute at the problem and hope for the best.
- m3kw9 3y agoIt basically gathers text that are similar to what you are asking and feed it into the prompt, yes. No magic. The worse part is that if you ask “please get me the summary to this doc” it will actually search the vector db using the entire question. It’s not very smart. Depending on how you split the embedding you could end up with a bunch of crap
- artathacanadian 3y agoWe’re building an easier-to-use langchain that lets you preprocess inputs and remove unnecessary wrapper text by going like “please get me the summary to this {{INPUT}}”
- Karrot_Kream 3y agoThe default "document splitter" just splits along double newlines. It took me aback when I realized they made an entire class around this.
- ryanjshaw 3y agoI had the same experience and kept wondering if I was missing something important. I'm not a fan of Python, so I was anxious about not using the thing everybody recommended, but for my project I ultimately went with what I know well (C#). I've happily had zero issues. LangChain docs and tutorials were useful for understanding the popular practices for approaching AI-driven development, but the biggest challenge by far has been getting a baseline prompt and measuring performance of alternative implementations against that in a sensible way that doesn't break the bank. Mitchell Hashimotos's Prompt Engineering article [1] was way more helpful in this regard than anything I saw in LangChain. To that end I've also been working on a tool to save me money by caching requests and responses, blocking unexpectedly expensive requests, keeping a granular history of requests for prompt cost analysis, etc. Maybe I should open source it and get some VC bux too? [1] https://mitchellh.com/writing/prompt-engineering-vs-blind-prompting https://mitchellh.com/writing/prompt-engineering-vs-blind-pr...
- manojlds 3y agoDon't tell this to the VCs
- hospitalJail 3y agoI believe it can reduce tokens. It essentially obfuscates text as well if you want the security theatre of your prompts being hidden. Most importantly it makes more complex chains where multiple questions are asked and uses the response from the llm to decide if you need a further question.
- convexfunction 3y agoI'm not sure if "helpful answer" as opposed to "answer" makes much of a difference in answer quality -- I'd believe it helps a little, just don't know that it's been studied -- but a lot of silly stuff like that does definitely make a big difference in response quality on certain tasks. "Let's think step by step:" at the end of your prompt is probably the best-known one: https://arxiv.org/pdf/2205.11916.pdf https://arxiv.org/pdf/2205.11916.pdf