9 ms·
Show HN: AnythingLLM – Open-Source, All-in-One Desktop AI Assistant
Hey HN!
This is Tim from AnythingLLM (https://github.com/Mintplex-Labs/anything-llm https://github.com/Mintplex-Labs/anything-llm). AnythingLLM is an open-source desktop assistant that brings together RAG (Retrieval-Augmented Generation), agents, embeddings, vector databases, and more—all in one seamless package.
We built AnythingLLM over the last year iterating and iterating from user feedback. Our primary mission is to enable people with a layperson understanding of AI to be able to use AI with little to no setup for either themselves, their jobs, or just to try out using AI as an assistant but with *privacy by default*.
From these iterations & feedback, we have a couple of key learnings I wanted to share:
- "Chat with your docs" solutions are a dime-a-dozen
- Agent frameworks require knowing how to code or are too isolated from other tools
- Users do not care about benchmarks, only outputs. The magic box needs to be magic to them.
- Asking Consumers to start a docker container or open a terminal is a non-starter for most.
- Privacy by default is non-negotiable. Either by personal preference or legal constraints
- Everything needs to be in one place
From these ideas, we landed on the current state of AnythingLLM:
- Everything in AnythingLLM is private by default, but fully customizable for advanced users.
- Built-in LLM provider, but can swap at any time to the hundreds of other local or cloud LLM providers & models.
- Built-in Vector Database, most users don't even know that it is there.
- Built-in Embedding model, but of course can change if the user wants to.
- Scrape websites, import Github/GitLab repos, YouTube Transcripts, Confluence spaces - all of this is already built in for the user.
- An entire baked-in agent framework that works seamlessly within the app. We even pre-built a handful of agent skills for customers. Custom plugins are in the next update and will be able to be built with code, or a no-code builder.
- All of this just works out of the box in a single installable app that can run on any consumer-grade laptop. Everything a user does, chats, or configures is stored on the user's device. Available for Mac, Windows, and Linux
We have been actively maintaining and working on AnythingLLM via our open-source repo for a while now and welcome contributors as we hopefully launch a Community Hub soon to really proliferate users' abilities to add more niche agent skills, data connectors, and more.
*But there is even more*
We view the desktop app as a hyper-accessible single-player version of AnythingLLM. We publish a Docker image too (https://hub.docker.com/r/mintplexlabs/anythingllm https://hub.docker.com/r/mintplexlabs/anythingllm) that supports multi-user management with permissioning so that you can easily bring AnythingLLM into an organization with all of the same features with minimal headache or lift.
The Docker image is for those more adept with a CLI, but being able to comfortably go from a single-user to a multi-user version of the same familiar app was very important for us.
AnythingLLM aims to be more than a UI for LLMs, we are building a comprehensive tool to leverage LLMs and all that they can do while maintaining user privacy and not needing to be an expert on AI to do it.
https://anythingllm.com/ https://anythingllm.com/
- santamex 2y agoHow does this differ from chatboxai? https://github.com/Bin-Huang/chatbox https://github.com/Bin-Huang/chatbox
- Ringz 2y agoChatboxai is a client for multiple AIs like OpenAi, Cloude, etc. It does not work on your local files and documents.
- ranger_danger 2y agoFinally.
- vednig 2y agoHow do you ensure "privacy by default" if you are also providing cloud models?
- hluska 2y agoGood question - this was an excellent write up and AnythingLLM looks great. But I’m really curious about that too. Regardless of the answer, OP you’ve done a hell of a lot of work and I hope you’re as proud as you should be. Congratulations on getting all the way to a Show HN.
- Tostino 2y agoIt doesn't seem like they are "providing" cloud models. They have their backend able to interface with whatever endpoint you want, given you have access. It's plainly obvious when interacting with 3rd party providers that it depends on their data / privacy policy. When has this ever not been the case? I could just start up a vLLM instance with llama 3.1 and connect their application to it just as easily though. Perfectly secure (as I am able to make it). This seems like such a pedantic thing to complain about.
- tcarambat1010 2y agoIt's not my position to "impose" a preference on a user for an LLM. Privacy by default means "if you use the defaults, it's private". Obviously, if your computer is low-end or you really have been loving GPT-4o or Claude you _can_ use just that external component while still using a local vector db, document storage, etc. So its basically like opt-in for external usage of any particular part of the app, whether that be an LLM, embedder model, vector db, or otherwise.