14 ms·

Ask HN: In-Browser Vector DB?

2 points by ukuina 3y ago

I'm looking to develop a retriever-augmented LLM Q&A app that will call out to an API for the LLM (this will be the only allowed type of API call [other than a fileserver], it's internal and no data can leave the firewall). I plan to ship the chunked, preprocessed document store as embeddings with the app (~100MB, can be read as files from a fileserver). Is there any way to ship a vector DB implementation in-browser? Other research: * I know that Chroma can use DuckDB under the hood, and DuckDB has a WASM port, but Chroma itself doesn't? * Weaviate and Qdrant do not have a WASM target?

andre-z 3y ago
Hi, Qdrant team here. Could you please explain why it has to be in-browser and not just start a docker locally?
- ukuina 3y ago
  Thank you for responding here. For this enterprise use-case, I am looking to ship a web app that will run off of IT-managed fileservers and IT-managed LLM APIs, on user desktops and laptops without privileged access to install software.
  - andre-z 3y ago
    If the dataset isn't extensive, you might be able just to use an ANN library like FAISS.
    - ukuina 3y ago
      That's a great idea, I will look at FAISS for now. Thank you.
dhruv_anand 3y ago
https://docs.trychroma.com/getting-started?lang=js#:~:text=Chroma%27s%20API%20is%20polymorphic%20(it%20can%20run%20in%20the%20browser%20or%20server%2Dside) https://docs.trychroma.com/getting-started?lang=js#:~:text=C...
- ukuina 3y ago
  Thank you for the pointer. This still requires a backend server running Chroma, correct?