4 ms·
Is this wholly self-hostable? I'd be curious to run something like this on a home server, have some small model via ollama slowly chew through my documents / co
by silversmith 2y ago
Is this wholly self-hostable? I'd be curious to run something like this on a home server, have some small model via ollama slowly chew through my documents / conversations / receipts / .... and provide a chat-like search engine over the whole mess.
- manishsharan 2y agoHere is how I am implementing something close to what you mentioned. In my setup, I make sure to create a readme.md at the root of every folder which is a document for me as well as LLM that tells me what is inside the folder and how it is relevant to my life or project. kind of a drunken brain dump for the folder . I have a cron job that executes every night and iterates through my filesystem looking for changes since the last time it ran. If it finds new files or changes, it creates embeddings and stores them in Milvus. The chat with LLM using Embeddings if not that great yet. To be fair,I have not yet tried to implement the GraphRAG or Claude's contexual RAG approaches. I have a lot of code in different programming languages, text documents, bills pdf, images. Not sure if one RAG can handle it all. I am using AWS Bedrock APIs for LLama and Claude and locally hosted Milvus
- j45 2y agoWondering if you have tried AnythingLLM, and if so what you thought of it.
- manishsharan 2y agoI have not .. but this seems to be something I must try.