6 ms·
LangChain: Build AI apps with LLMs through composability
- alonger1999 4y agoHi
- deleted 4y ago[deleted]
- LASR 4y agoThis is highly useful. I am prototyping new features where I work, on top of GPT3. To get it beyond fancy demos and actually delivering customer utility, you need a LOT of work to build in robustness and correctness. Prompt chains is where I’ve landed at the moment. This library seems very relevant and timely.
- jsemrau 4y ago>you need a LOT of work to build in robustness and correctness. While bias is the more politically charged problem, variance is the problem of the current iteration that needs to be solved.
- alooPotato 4y agoIs your usage of GPT3 user facing in realtime? When you implement prompt chains, doesn't the UX become really slow?
- LASR 4y agoIt works well if you continously provide progress to the user. In my specific use case, the data generated is progressively displayed to the user, and refined in many further chain steps. They see the evolution of this. So it feels like it's fast, but also that a bunch of work is happening.
- dandiep 4y agoYes.
- bluecoconut 4y agoThis is great! I love seeing how rapidly in the past 6 months these ideas are evolving. I've been internally calling these systems "prompt machines". I'm a strong believer that chaining together language model prompts is core to extracting real, and reproducible value from language models. I sometimes even wonder if systems like this are the path to AGI as well, and spent a full month 'stuck' on that hypothesis in October. Specific to prompt-chaining: I've spent a lot of time ideating about where "prompts live" (are they best as API endpoint, as cli programs, as machines with internal state, treated as a single 'assembly instruction' -- where do "prompts" live naturally) and eventually decided on them being the most synonymous with functions (and api endpoints via the RPC concept) mental model I've developed (sharing in case it resonates with anyone else) a "chain" is `a = 'text'; b = p1(a); c = p2(b)` where p1 and p2 are LLM prompts. What comes next (in my opinion) is other programming constructs: loops, conditionals, variables (memory), etc. (I think LangChain represents some of these concepts as their "areas" -> chain (function chaining), agents (loops), memory (variables)) To offer this code-style interface on top of LLMs, I made something similar to LangChain, but scoped what i made to only focus on the bare functional interface and the concept of a "prompt function", and leave the power of the "execution flow" up to the language interpreter itself (in this case python) so the user can make anything with it. https://github.com/approximatelabs/lambdaprompt https://github.com/approximatelabs/lambdaprompt I've had so much fun recently just playing with prompt chaining in general, it feels like the "new toy" in the AI space (orders of magnitude more fun than dall-e or chat-gpt for me). (I built sketch (posted the other day on HN) based on lambdaprompt) My favorites have been things to test the inherent behaviors of language models using iterated prompts. I spent some time looking for "fractal" like behavior inside the functions, hoping that if I got the right starting point, an iterated function would avoid fixed points --> this has eluded me so far, so if anyone finds non-fixed points in LLMs, please let me know! I'm a believer that the "next revolution" in machine-written code and behavior from LLMs will come when someone can tame LLM prompting to self-write prompt chains themselves (whether that is on lambdaprompt, langchain, or something else!) All in all, I'm super hyped about LangChain, love the space they are in and the rapid attention they are getting~
- gandalfgeek 4y agoLambdaPrompt looks really cool. Love the Pythonic expression as good ol' functions, which makes the code read very naturally. What applications were you envisioning when you built it?
- dang 4y agoRelated ongoing thread: GPT-3.5 and Wolfram Alpha via LangChain - https://news.ycombinator.com/item?id=34422122 https://news.ycombinator.com/item?id=34422122
- brotchie 4y agoHell yeah. Been thinking about this in my spare cycles a lot! The missing thing when interacting with GPT3 was building up layers of re-usable abstraction that could be mixed together: i.e. a prompt that's great at summarization, a prompt that great at generating a list of 5 x related ideas, a prompt that's great at sorting a list of inputs according to some verbal sorting description. Had a play around with this early-GPT3 days by creating a Python decorator that let you easily write "prompt functions" and then you could combine these to create higher-level prompt generating machines. LangChain takes this to the logical end-point, awesome!
- hooande 4y agoIsn't this a chat interface to various apis? I guess that's the point. But it doesn't seem like that is utilizing the power of large language models. Maybe I'm not understanding this. Using a Google search api + a calculator to answer a question is cool [0]. but... we could already do that? [0] https://langchain.readthedocs.io/en/latest/modules/agents/examples/custom_tools.html https://langchain.readthedocs.io/en/latest/modules/agents/ex...
- KennyFromIT 4y agoThey have a dedicated chat bot for their docs that might be able to help you find answers to your questions... https://chat.langchain.dev/ https://chat.langchain.dev/
- user- 4y agoMe > can i use docker? AI > Yes, you can use Docker with LangChain. For more information, please see the Docker Installation Guide in the LangChain documentation. Then it links to a 404 lol. I checked the docs and there is nothing about docker in them. I wonder why its incorrect here
- andreigheorghe 4y agomaybe it's trained on non-published documentation pages?
- sebastiennight 4y agoI'm very curious as to how they are able to limit the chat bot's knowledge to their own documentation. It seems as if it should also have the entire knowledge of the LLM, but > Who was Thomas Jefferson? Outputs > Hmm, I'm not sure. I'm an AI assistant for the open source library LangChain. You can find more information about LangChain at https://langchain.readthedocs.io https://langchain.readthedocs.io.
- rafaquintanilha 4y ago
- cs702 4y agoVery cool. By far the most interesting aspect of this, for me, is that we're now seeing tools for building software infrastructure with layers of APIs that operate on -- gasp! -- natural language, which is notoriously prone to imprecision and ambiguity. And yet it works remarkably well. It's hard not to look at all this, mouth agape, in awe. Part of me wonders, though: Wouldn't it be better if we could compose LLMs by passing sequences of embeddings (e.g., in a standardized high-dimensional space), which are much richer representations of LLM input, internal, and output states?
- dandiep 4y agoPerhaps, but language is the common denominator in a multi-model world. E.g., I pass the GPT output into other models which are fine tuned for that sub domain. You can do embedding to embedding conversion, but not sure it's worth the effort.
- cs702 4y agoImagine if OpenAI made GPT3's final hidden states available via an API ("GPT3 deep sequence embeddings v1.0"), next to each generated text token: [(text_token, deep_emb), (text_token, deep_emb), ...]. You and anyone else could build apps on top. Those hidden states would incorporate much more, and much richer, information than the text. Higher-level models could be trained to act on such information!
- firasd 4y agoI think embeddings are actually low-'resolution' representations. Like GPT's ability to parse and calculate the structure of the sentence "Hello, how are you?" is not represented in the embedding for the sentence. The embedding is a 1-dimensional vector and inside the model it interacts with other texts in 10,000+ dimensions
- neuronexmachina 4y agoI wonder if there'd be any use for a "ontological" representation, somewhere in-between a natural-language string and its embedding in a particular LLM. Maybe something that balances human-readability, LLM-composability, lack of brittleness, insight into the local structure of the embedding, etc.
- throwaway20222 4y agoThis is amazing. Thank you to all the contributors! I am working on a project in the old media space and we were planning to build several of these elements ourselves if something like this didn’t come along. Love the open source nature and would love to contribute.
- keepquestioning 4y agogamechanger how can we use ChatGPT to design a 3d model?
- lgas 4y agoThis doesn't really seem like the right thread for this question, but as long as we are here... you might be interested in POINT-E https://huggingface.co/spaces/openai/point-e https://huggingface.co/spaces/openai/point-e or Stable Dreamfusion https://colab.research.google.com/drive/1MXT3yfOFvO0ooKEfiUUvTKwUkrrlCHpF?usp=sharing https://colab.research.google.com/drive/1MXT3yfOFvO0ooKEfiUU...
- jarbus 4y agoAhhh, I was hoping to work on something like this after I completed my PhD!
- dandiep 4y agoLangChain is very cool. Templates and composability are the way forward. I'm guessing that most folks haven't created composable prompts, so a few experiences on why this stuff is necessary. I've been building a language learning app and it makes heavy use of GPT-3 [1], which has made clear a number of things for me: 1) Composability is fundamental to leveraging language models. You can't get language models to just generate things in one go. It's rather like a human. If you're writing a book, you start with an idea, then an outline, then a chapter outline, then writing paragraphs... Or in our case, generate a flashcard deck description, then vocab, then example sentences, etc. 2) Externalized prompt templates are also important. Engineers need to be able interface with experts who can create custom prompts. E.g. in our case, I need experts who speaks to build prompts specific to other languages for a language learning app [2]. 3) Unit testing is critical. There is no linter for a prompt. I have made so many typos over the last year that broke things. OpenAI has released several new models over the last 2 years. Anthropic is coming out with a model. You need to have assurances your prompts work. I've actually had to start building a basic unit tester for our prompts because of this... [3] (please someone else do this so I don't have to) 4) external data is the next step forward for large language models. E.g. in my case, someone may want to learn about the history/culture of a country and we may want to reference existing articles on it since LLMs are known to hallucinate. We need to be able to interface with the web and databases easily. I'm not convinced that LangChain is the write layer of abstraction for this. I suspect/hope that the next version of GPT will have some significant advances in this regard. 1. https://squidgies.app https://squidgies.app 2. https://github.com/squidgyai/squidgy-prompts https://github.com/squidgyai/squidgy-prompts - open source composable prompts for Squidgies 3. https://github.com/squidgyai/squidgy-prompts/tree/main/tests https://github.com/squidgyai/squidgy-prompts/tree/main/tests - unit test in YAML
- tennisfan118 4y agoBuilding https://promptable.ai https://promptable.ai to solve testing!
- dandiep 4y agoAwesome... but I can't tell from your website what exactly it is.
- owlninja 4y agoI like to think I am smart and in the loop - but would this allow me to feed all sorts of PowerPoint presentations, word docs, text files, etc... so that new hires could get a concise answer to business processes?
- dandiep 4y agoThere's a lot of work going on in this area right now. Check out GPT Index: https://github.com/jerryjliu/gpt_index https://github.com/jerryjliu/gpt_index https://github.com/jerryjliu/gpt_index/blob/main/examples/paul_graham_essay/TestEssay.ipynb https://github.com/jerryjliu/gpt_index/blob/main/examples/pa... GPT isn't multi-modal yet (so no images), but that's coming.
- mark_l_watson 4y agoSuch a good idea! A few years ago I tried importing pre-trained Keras models into Racket Scheme, creating a function for each model. I didn't really get the idea right, LangChain looks much better.
- peterth3 4y agoWhat LLMs does LangChain support? Btw I asked chat.langchain.dev and it said: > LangChain uses pre-trained models from Hugging Face, such as BERT, GPT-2, and XLNet. For more information, please see the Getting Started Documentation[0]. That links to a 404, but I did find the correct link[1]. Oddly that doc only mentions an OpenAI API wrapper. I couldn’t find anything about the other models from huggingface. Does LangChain have any tooling around fine tuning pre-trained LLMs like GPTNeoX[2]? [0]https://langchain.readthedocs.io/en/latest/getting_started.html https://langchain.readthedocs.io/en/latest/getting_started.h... [1]https://langchain.readthedocs.io/en/latest/getting_started/getting_started.html https://langchain.readthedocs.io/en/latest/getting_started/g... [2]https://github.com/EleutherAI/gpt-neox https://github.com/EleutherAI/gpt-neox
- lgas 4y agoHere are the docs on the built in models at the moment: https://langchain.readthedocs.io/en/latest/reference/modules/llms.html https://langchain.readthedocs.io/en/latest/reference/modules... One of their examples is from langchain import NLPCloud nlpcloud = NLPCloud(model="gpt-neox-20b") So it looks like you're good to go.
- Ozzie_osman 4y agoLangChain is awesome. For people not sure what it's doing, large language models (LLMs) are very powerful but they're very general. As a common example for this limitation, imagine you want your LLM to answer questions over a large corpus. You can't pass the entire corpus into the prompt. So you might: - preprocess the corpus by iterating over documents, splitting them into chunks, and summarizing them - embed those chunks/summaries in some vector space - when you get a question, search your vector space for similar chunks - pass those chunks to the LLM in the prompt, along with your question This ends up being a very common pattern, where you need to do some preprocessing of some information, some real-time collecting of pieces, and then an interaction with the LLM (in some cases, you might go back and forth with the LLM). For instance, code and semantic search follows a similar pattern (preprocess -> embed -> nearest-neighbors at query time -> LLM). Langchain provides a great abstraction for composing these pieces. IMO, this sort of "prompt plumbing" is far more important than all the slick (but somewhat gimicky) "prompt engineering" examples we see. I suspect this will get more important as the LLMs become more powerful and more integrated, requiring more data to be provided at prompt time.
- Ozzie_osman 4y agoAlso, another library to check out is GPT Index (https://github.com/jerryjliu/gpt_index https://github.com/jerryjliu/gpt_index) which takes a more "data structure" approach (and actually uses langchain for some stuff under the hood).
- fudged71 4y agoI’ve been following both LangChain and GPT Index but to be honest I’m getting super confused which use cases are better suited for each (or a combination of both!). It would be great to see a presentation of both tools together like this :)
- elevenoh 4y ago[dead]
- revskill 4y agoIt's sad that all things now has OpenAI dependency. That OpenAI is political, it's banned in some regions in the world. Academics became a shameful thing.
- ghoshbishakh 4y agoThank god this is not about Blockchains.
- barrenko 4y agoPretty sweet, but are there similar setup more geared towards coding? This would be a great way to get a pair programmer for people who are just coding for hobby or transitioning to SWE. Or to alleviate imposter syndrome.
- swyx 4y agocopilot not enough for you? why?
- camjw 4y agoWe’re using LangChain at my startup to orchestrate all the GPT-3 calls and it’s fantastic! Harrison is so helpful too!
- Animats 4y agoThis looks useful. It should compete with Rasa, which is supposed to do something similar, but in practice is mostly only able to match what you're saying to nodes in a phone tree. Has anyone done a game NPC with this?
- Mizza 4y agoI had a similar idea, but it uses a graph construction and Elixir. Definitely going to poach some ideas from LangChain though. https://github.com/Miserlou/Helix https://github.com/Miserlou/Helix
- mfalcon 4y agoWhat if OpenAI decides to close or charge for access to it's API? Is there a compromise of keeping it open somewhere? What's the plan B? There's a really strong dependency on their service and I sometimes get doubtful about dedicating time to build things on top of it.
- ronsor 4y ago> OpenAI decides to close or charge for access to it's API? They already do: https://openai.com/api/pricing/ https://openai.com/api/pricing/
- mbil 4y agoOpenAI already does charge access to its API. ChatGPT is currently free, but LLM model access has been available through a paid API since before GPT-3.
- monkeydust 4y agoI have a side project at the moment to use LangChain to help build a Q&A system from domain specific documentation on a product I own. Its around 500 different PDFs. I was going to follow this guide* which looks credible and solves for how to deal with large corpus of texts. Curious what others think as to approach. * https://dagster.io/blog/chatgpt-langchain https://dagster.io/blog/chatgpt-langchain
- imranq 4y agoIs there support for Prompt-Tuning? This is also an interesting space to optimize inputs to frozen language models https://ai.googleblog.com/2022/02/guiding-frozen-language-models-with.html https://ai.googleblog.com/2022/02/guiding-frozen-language-mo...