3 ms·

Could you elaborate? It may not be a replacement, but I can think of a few use cases where RAG could be avoided entirely by having the model be fine tuned on th

Could you elaborate? It may not be a replacement, but I can think of a few use cases where RAG could be avoided entirely by having the model be fine tuned on the right dataset.

minimaxir 3y ago
There is a risk of (greater) hallucination if there aren't explicit sample inputs provided by RAG.
- huac 3y ago
  Finetuning teaches skills. RAG provides 'facts.'
lukev 3y ago
Hallucination, but also the ability to cite a source. A model using RAG can tell you why it answered a certain way, and cite chapter and verse from the underlying data. That's impossible just from model weights.
stavros 3y ago
Speaking of RAG, does anyone know of a Python library that (hopefully) doesn't need a server, that can do embeddings, along with an example? I want to do RAG in Python, and I can wrangle the prompt to where I want it, but I have no idea how to basically search for relevant documents with embeddings.
- lukev 3y ago
  https://www.trychroma.com https://www.trychroma.com It works as a library, you can require the PIP package and get started immediately.
  - stavros 3y ago
    This looks great, thank you!
- J_Shelby_J 3y ago
  Here is my implementation used for my discord and slack bots. The logic for rag is the Shelby_agent module. RAG is straightforward but i improve it by generating extra keywords to help the search. I also add extra metadata filtering to return both technical documentation and more fluffy sales and marketing stuff. IMO, semantic search alone might not be the best, and I want to also add keyword search. I think the thing most people will struggle with is prepping the documents. Chunking them to the best token count for your use case for example. https://github.com/shelby-as-a/shelby-as-a-service https://github.com/shelby-as-a/shelby-as-a-service
  - stavros 3y ago
    That's very helpful, thanks!
  - stavros 3y ago
    I know I left another comment, but I'm replying in a new one to increase the chance that you'll see this: It would probably be very useful to a lot of people if you (or someone) made a standalone library out of this. It looks like Shelby requires Pinecone, but using something like ChromaDB for local embeddings and providing a library that I can just import, add documents to, and then query for answers would enable all kinds of applications. I'm currently trying to build a hobby virtual assistant, and it looks like RAG does need some nuance to implement correctly (from what you said). A library that does all of what you mentioned would take a lot of work out of it.
    - J_Shelby_J 3y ago
      That's actually something that I think is a next step. a) run locally through a web ui b) run with a local vectorstore From my perspective of someone using it as a bot that runs in a container using a cloud based vector store makes a lot of sense, but yeah that's really good feedback and not something I considered. Hrmmm. Ok let me get a look at some local vectorstores.

Could you elaborate? It may not be a replacement, but I can think of a few use cases where RAG could be avoided entirely by having the model be fine tuned on the right dataset.

minimaxir 3y ago

There is a risk of (greater) hallucination if there aren't explicit sample inputs provided by RAG.

huac 3y ago
Finetuning teaches skills. RAG provides 'facts.'

lukev 3y ago

Hallucination, but also the ability to cite a source. A model using RAG can tell you why it answered a certain way, and cite chapter and verse from the underlying data. That's impossible just from model weights.

stavros 3y ago

Speaking of RAG, does anyone know of a Python library that (hopefully) doesn't need a server, that can do embeddings, along with an example? I want to do RAG in Python, and I can wrangle the prompt to where I want it, but I have no idea how to basically search for relevant documents with embeddings.

lukev 3y ago
https://www.trychroma.com https://www.trychroma.com It works as a library, you can require the PIP package and get started immediately.
- stavros 3y ago
  This looks great, thank you!
J_Shelby_J 3y ago
Here is my implementation used for my discord and slack bots. The logic for rag is the Shelby_agent module. RAG is straightforward but i improve it by generating extra keywords to help the search. I also add extra metadata filtering to return both technical documentation and more fluffy sales and marketing stuff. IMO, semantic search alone might not be the best, and I want to also add keyword search. I think the thing most people will struggle with is prepping the documents. Chunking them to the best token count for your use case for example. https://github.com/shelby-as-a/shelby-as-a-service https://github.com/shelby-as-a/shelby-as-a-service
- stavros 3y ago
  That's very helpful, thanks!
- stavros 3y ago
  I know I left another comment, but I'm replying in a new one to increase the chance that you'll see this: It would probably be very useful to a lot of people if you (or someone) made a standalone library out of this. It looks like Shelby requires Pinecone, but using something like ChromaDB for local embeddings and providing a library that I can just import, add documents to, and then query for answers would enable all kinds of applications. I'm currently trying to build a hobby virtual assistant, and it looks like RAG does need some nuance to implement correctly (from what you said). A library that does all of what you mentioned would take a lot of work out of it.
  - J_Shelby_J 3y ago
    That's actually something that I think is a next step. a) run locally through a web ui b) run with a local vectorstore From my perspective of someone using it as a bot that runs in a container using a cloud based vector store makes a lot of sense, but yeah that's really good feedback and not something I considered. Hrmmm. Ok let me get a look at some local vectorstores.