3 ms·

I built this for local RAG https://github.com/kbrisso/byte-vision https://github.com/kbrisso/byte-vision it uses llama.cpp and Elasticsearch. On a laptop with 8

by kbrisso 10mo ago

I built this for local RAG https://github.com/kbrisso/byte-vision https://github.com/kbrisso/byte-vision it uses llama.cpp and Elasticsearch. On a laptop with 8 GB GPU it can handle a 30K token size and summarize a fairly large PDF.

busssard 10mo ago
elasticsearch is the true limitation of rag systems...
- kbrisso 10mo ago
  The vector search works great once you figure it out. I wanted to focus on writing the application and not have to rewrite a document store.