8 ms·

FWIW, having written a simple RAG system from "scratch" (meaning not using frameworks or api calls), it's not more complicated than doing it this way with langc

FWIW, having written a simple RAG system from "scratch" (meaning not using frameworks or api calls), it's not more complicated than doing it this way with langchain etc. This post is mostly about plumbing. It's probably the right way to do it if it needs to be scaled. But for learning, it obscures what is essentially simple stuff going on behind the scenes.

iampims 3y ago
Opportunity for you to write a blogpost about your approach :)
- ajhai 3y ago
  We did write one just yesterday that talks about rags and some techniques to improve their performance in production at https://llmstack.ai/blog/retrieval-augmented-generation https://llmstack.ai/blog/retrieval-augmented-generation
  - stringtoint 3y ago
    This is a neat post, thanks for sharing!
jamesblonde 3y ago
I tend to agree - i haven't seen the value in existing "retriever" components in langchain and others.
- haxton 3y ago
  My favorite example is the asana loader[0] for llama-index. It's literally just the most basic wrapper around the Asana SDK to concatenate some strings. [0] - https://github.com/emptycrown/llama-hub/blob/main/llama_hub/asana/base.py https://github.com/emptycrown/llama-hub/blob/main/llama_hub/...
  - themanmaran 3y ago
    Yea that's langchain in a nutshell. Their postgres wrapper was the same last time I used it was pretty much ``` def queryPostgres(client, string) return client.query(string) ```
clharman 3y ago
For serious implementations, frameworks are not very helpful, even LangChain. All the components provide good SDKs/APIs, having a bunch of "integrations" doesn't add any real value. If you know what you want to build, building from scratch is easier than you think. If you're tinkering on the weekend, then maybe the frameworks are helpful.
- lmeyerov 3y ago
  Yeah as soon as we write the word 'thread' or thinking about LLM API concurrency control across many user requests, all frameworks we tried are basically a wall instead of an accelerator. For a single user demo video on Twitter or a low-traffic streamlit POC to get a repo with lots of star gazers, they work quite well, and that's not far from what someone needs for an internal project with a small userbase. Just once this is supposed to be infra for production-grade software, the tools we have tried so far are still prioritizing features over being a foundation.

iampims 3y ago

Opportunity for you to write a blogpost about your approach :)

ajhai 3y ago
We did write one just yesterday that talks about rags and some techniques to improve their performance in production at https://llmstack.ai/blog/retrieval-augmented-generation https://llmstack.ai/blog/retrieval-augmented-generation
- stringtoint 3y ago
  This is a neat post, thanks for sharing!

jamesblonde 3y ago

I tend to agree - i haven't seen the value in existing "retriever" components in langchain and others.

haxton 3y ago
My favorite example is the asana loader[0] for llama-index. It's literally just the most basic wrapper around the Asana SDK to concatenate some strings. [0] - https://github.com/emptycrown/llama-hub/blob/main/llama_hub/asana/base.py https://github.com/emptycrown/llama-hub/blob/main/llama_hub/...
- themanmaran 3y ago
  Yea that's langchain in a nutshell. Their postgres wrapper was the same last time I used it was pretty much ``` def queryPostgres(client, string) return client.query(string) ```

clharman 3y ago

For serious implementations, frameworks are not very helpful, even LangChain. All the components provide good SDKs/APIs, having a bunch of "integrations" doesn't add any real value. If you know what you want to build, building from scratch is easier than you think. If you're tinkering on the weekend, then maybe the frameworks are helpful.

lmeyerov 3y ago
Yeah as soon as we write the word 'thread' or thinking about LLM API concurrency control across many user requests, all frameworks we tried are basically a wall instead of an accelerator. For a single user demo video on Twitter or a low-traffic streamlit POC to get a repo with lots of star gazers, they work quite well, and that's not far from what someone needs for an internal project with a small userbase. Just once this is supposed to be infra for production-grade software, the tools we have tried so far are still prioritizing features over being a foundation.