3 ms·

Someone needs to create a “Langchain, but less complicated” framework

riter 3y ago
lololol. i think this opportunity gets bigger post $10m seed round. they'll likely double down and expand footprint vs the inverse. check out llama-index. its purpose-built for document indexing and retrieval and less agents and "everything else"
- data_maan 3y ago
  What do you by post 10m seed round? Do you mean if LlamaIndex starts collecting VC? I'm not sure, are they for-profit?...
  - riter 3y ago
    I was referring to Langchain who raised $10mm from Benchmark https://blog.langchain.dev/announcing-our-10m-seed-round-led-by-benchmark/ https://blog.langchain.dev/announcing-our-10m-seed-round-led... I'm fairly Jerry Liu (LlamaIndex founder) already has angels or will see enough traction to warrant a seed.
- chaxor 3y ago
  That's pretty wild, I've been setting things up like this for about 5 years with just BERT or my own fine tuned encoder only systems. It should be done for free, not millions... Can I get millions for running `ls` too?
krishadi 3y ago
The use-cases and tooling around language models is very premature. So, any framework you build now will either look like bloatware or will remain close to just calling an API. The dust around language models needs to settle a bit, for a useful framework to emerge from it. For our own use-cases, I built a framework from scratch, and it was the best decision we made.
- riter 3y ago
  very much agreed re: dust settling. it makes no sense deploying any of these libraries to prod. as-is. best to understand a configuration / workflow / tuning / etc. that fits your data best and write it from scratch in golang/rust/whatever.
  - Dowwie 3y ago
    Are these computationally expensive operations? If not, Elixir could fit.
    - krishadi 3y ago
      They are not all computationally expensive. The rate limiting step here is the LLM call itself over the API. So, async is definitely needed. The other aspects would be loading the template from filesystem. I would assume this could be something that's needs to be optimized in the application.
- data_maan 3y ago
  > For our own use-cases, I built a framework from scratch, and it was the best decision we made. My thinking precisely. So you just used the "raw" OpenAI (I presume?) API, and no other tech on top?
  - krishadi 3y ago
    Exactly. The most important part was working with Jinja templating. So, openai + jinja2.
bluecoconut 3y ago
I sorta did this, feel free to check it out and let me know your thoughts! On the main langchain post (In January) that got the traction on hackernews, i left this comment: https://news.ycombinator.com/item?id=34422917 https://news.ycombinator.com/item?id=34422917 . It still remains true, a "simpler langchain" > To offer this code-style interface on top of LLMs, I made something similar to LangChain, but scoped what i made to only focus on the bare functional interface and the concept of a "prompt function", and leave the power of the "execution flow" up to the language interpreter itself (in this case python) so the user can make anything with it. I made a really lightweight wrapper over requests and call it lambdaprompt https://github.com/approximatelabs/lambdaprompt https://github.com/approximatelabs/lambdaprompt It has served all of my personal use-cases since making it, including powering `sketch` (copilot for pandas) https://github.com/approximatelabs/sketch https://github.com/approximatelabs/sketch Core things it does: Uses jinja templates, does sync and async, and most importantly treats LLM completion endpoints as "function calls", which you can compose and build structures around just with simple python. I also combined it with fastapi so you can just serve up any templates you want directly as rest endpoints. It also offers callback hooks so you can log & trace execution graphs. All together its only ~600 lines of python. I haven't had a chance to really push all the different examples out there, so I think it hasn't seen much adoption outside of those that give it a try. I hope to get back to it sometime in the next week to introduce local-mode (eg. all the open source smaller models are now available, I want to make those first-class)
opyate 3y ago
This recently via DataMachina substack: https://blog.scottlogic.com/2023/05/04/langchain-mini.html https://blog.scottlogic.com/2023/05/04/langchain-mini.html
- riter 3y ago
  thanks for the share, will check out

lololol. i think this opportunity gets bigger post $10m seed round. they'll likely double down and expand footprint vs the inverse. check out llama-index. its purpose-built for document indexing and retrieval and less agents and "everything else"

data_maan 3y ago
What do you by post 10m seed round? Do you mean if LlamaIndex starts collecting VC? I'm not sure, are they for-profit?...
- riter 3y ago
  I was referring to Langchain who raised $10mm from Benchmark https://blog.langchain.dev/announcing-our-10m-seed-round-led-by-benchmark/ https://blog.langchain.dev/announcing-our-10m-seed-round-led... I'm fairly Jerry Liu (LlamaIndex founder) already has angels or will see enough traction to warrant a seed.
chaxor 3y ago
That's pretty wild, I've been setting things up like this for about 5 years with just BERT or my own fine tuned encoder only systems. It should be done for free, not millions... Can I get millions for running `ls` too?

krishadi 3y ago

The use-cases and tooling around language models is very premature. So, any framework you build now will either look like bloatware or will remain close to just calling an API. The dust around language models needs to settle a bit, for a useful framework to emerge from it. For our own use-cases, I built a framework from scratch, and it was the best decision we made.

riter 3y ago
very much agreed re: dust settling. it makes no sense deploying any of these libraries to prod. as-is. best to understand a configuration / workflow / tuning / etc. that fits your data best and write it from scratch in golang/rust/whatever.
- Dowwie 3y ago
  Are these computationally expensive operations? If not, Elixir could fit.
  - krishadi 3y ago
    They are not all computationally expensive. The rate limiting step here is the LLM call itself over the API. So, async is definitely needed. The other aspects would be loading the template from filesystem. I would assume this could be something that's needs to be optimized in the application.
data_maan 3y ago
> For our own use-cases, I built a framework from scratch, and it was the best decision we made. My thinking precisely. So you just used the "raw" OpenAI (I presume?) API, and no other tech on top?
- krishadi 3y ago
  Exactly. The most important part was working with Jinja templating. So, openai + jinja2.

bluecoconut 3y ago

I sorta did this, feel free to check it out and let me know your thoughts! On the main langchain post (In January) that got the traction on hackernews, i left this comment: https://news.ycombinator.com/item?id=34422917 https://news.ycombinator.com/item?id=34422917 . It still remains true, a "simpler langchain" > To offer this code-style interface on top of LLMs, I made something similar to LangChain, but scoped what i made to only focus on the bare functional interface and the concept of a "prompt function", and leave the power of the "execution flow" up to the language interpreter itself (in this case python) so the user can make anything with it. I made a really lightweight wrapper over requests and call it lambdaprompt https://github.com/approximatelabs/lambdaprompt https://github.com/approximatelabs/lambdaprompt It has served all of my personal use-cases since making it, including powering `sketch` (copilot for pandas) https://github.com/approximatelabs/sketch https://github.com/approximatelabs/sketch Core things it does: Uses jinja templates, does sync and async, and most importantly treats LLM completion endpoints as "function calls", which you can compose and build structures around just with simple python. I also combined it with fastapi so you can just serve up any templates you want directly as rest endpoints. It also offers callback hooks so you can log & trace execution graphs. All together its only ~600 lines of python. I haven't had a chance to really push all the different examples out there, so I think it hasn't seen much adoption outside of those that give it a try. I hope to get back to it sometime in the next week to introduce local-mode (eg. all the open source smaller models are now available, I want to make those first-class)

opyate 3y ago

This recently via DataMachina substack: https://blog.scottlogic.com/2023/05/04/langchain-mini.html https://blog.scottlogic.com/2023/05/04/langchain-mini.html

riter 3y ago
thanks for the share, will check out