7 ms·
Sqlite-vec: Work-in-progress vector search SQLite extension that runs anywhere
- mic47 2y agoNice. Been waiting for this release to try it out.
- bodantogat 2y agoThis sounds useful (I do a lot of throw-away text analysis on my laptop)
- marvel_boy 2y agoCould anybody explain me a simple example how to do text analysis via this vector search. It just searches for the closer vector?
- brylie 2y agoOne use case is for retrieving similar documents, such as when recommending related content. Another use case is retrieving document segments that are similar to a user query and passing them along with the user query to a large language model for improvement in the generated response. Vector search is also better in some ways than keyword search since it can find documents that are semantically similar even when the user may not have used the exact keyword, or even partial keywords like “Postgres” instead of “PostgreSQL.”
- alexgarcia-xyz 2y agoYou can generate "text embeddings" (ie vectors) from your text with an embedding model. Once you have your text represented as vectors, then "closest vector" means "more semantically similar", as defined by the embedding model you use. This can allow you to build semantic search engines, recommendations, and classifiers on top of your text and embeddings. It's kindof like a fancier and fuzzier keyword search. I wouldn't completely replace keyword search with vector search, but it's a great addition that essentially lets you perform calculations on text
- yAak 2y agoThank you! This was an extremely helpful explanation to me, as someone not familiar on the topic. =)
- bodantogat 2y agoNice explanation. One use case where keywords haven't worked well for me , and (at least at first glance) vectors are doing better are longer passages -- finding sentences that are similar rather than just words.
- simonw 2y agoI put together an extensive guide to understanding embeddings and vector search last year: https://simonwillison.net/2023/Oct/23/embeddings/ https://simonwillison.net/2023/Oct/23/embeddings/
- bambax 2y agoYes this is an excellent article, thank you. Do embeddings work across languages? It must depend on the model, but are there models where similar concepts in different languages occupy the same location in vector space? If yes it would allow to search a corpus in one language using another, without translating anything up front (neither the corpus nor the query).
- simonw 2y agoLots more details in Alex's blog post here: https://alexgarcia.xyz/blog/2024/sqlite-vec-stable-release/index.html https://alexgarcia.xyz/blog/2024/sqlite-vec-stable-release/i...
- rsingel 2y agoWould this play nicely with datasette?
- pjot 2y agoI’ve done something similar, but using duckDB as the backend. https://github.com/patricktrainer/duckdb-embedding-search https://github.com/patricktrainer/duckdb-embedding-search
- youngbum 2y agoDuckdb is an excellent choice for this task, and it’s incredibly fast! We’ve also added vector search to our product, which is really useful. OpenAI’s official examples of embedding search use cosine similarity. But here’s the cool part: since OpenAI embeddings are unit vectors, you can just run the dot product instead! DuckDB has a super fast dot product function that you can use with SQL. In our product, we use duckdb-wasm to do vector searches on the client side.
- bhl 2y agoWhat library do you use to compute embedding right now? I'm wondering if it's possible to do both embedding and vector search client-side for a local-first app.
- cxcorp 2y agoI think Transformers.js can do that client-side, e.g. see the documentation for calculating embeddings with Sentence Transformers models: https://huggingface.co/docs/transformers.js/api/pipelines#module_pipelines.FeatureExtractionPipeline https://huggingface.co/docs/transformers.js/api/pipelines#mo...
- rkagerer 2y ago>> In our product, we use duckdb-wasm to do vector searches on the client side. Curious, what is your product? Edit: Nevermind, your recent post explained it quite well: https://news.ycombinator.com/item?id=40520073 https://news.ycombinator.com/item?id=40520073
- pietz 2y agoIs this also what turso uses in their "AI feature"?
- alexgarcia-xyz 2y agoNo, libsql added custom vector search directly into their library, while sqlite-vec is a separate SQLite extension. The libsql vector feature only works in libsql, sqlite-vec works in all SQLite versions. The libsql vector feature works kindof like pgvector, while sqlite-vec works more like the FTS5 full text SQLite extension. I'd say try both and see which one you like more. sqlite-vec will soon be a part of Turso's and SQLite Cloud's products. Turso's version: https://turso.tech/vector https://turso.tech/vector
- alexgarcia-xyz 2y agoAuthor here, happy to answer any questions! Been working on this for a while, so I'm very happy to get this v0.1.0 "stable" release out. sqlite-vec works on MacOS, Linux, Windows, Raspberry Pis, in the browser with WASM, and (theoretically) on mobile devices. I focused a lot on making it as portable as possible. It's also pretty fast - benchmarks are hard to do accurately, but I'd comfortable saying that it's a very very fast brute-force vector search solution. One experimental feature I'm working on: You can directly query vectors that are in-memory as a contiguous block of memory (ie NumPy), without any copying or cloning. You can see the benchmarks for that feature here under "sqlite-vec static", and it's competitive with faiss/usearch/duckdb https://alexgarcia.xyz/blog/2024/sqlite-vec-stable-release/index.html#benchmarks https://alexgarcia.xyz/blog/2024/sqlite-vec-stable-release/i...
- rcarmo 2y agoGreat to see this. Seems simple enough, but I can't wait until ORMs like peewee incorporate support alongside things like FTS, etc. just for the sake of case of use.
- cyanydeez 2y agoWhich wasm sqlite project would it be compatible with?
- ncruces 2y agoIt can be statically compiled into the official SQLite Wasm builds: https://alexgarcia.xyz/sqlite-vec/wasm.html https://alexgarcia.xyz/sqlite-vec/wasm.html It can also be statically compiled into the Wasm used by my Go bindings.
- bambax 2y agoThank you for this, it's really super exciting! The link on See Installing sqlite-vec for more details. https://alexgarcia.xyz/sqlite-vec/installing.html https://alexgarcia.xyz/sqlite-vec/installing.html is a a 404 (the correct link is https://alexgarcia.xyz/sqlite-vec/installation.html https://alexgarcia.xyz/sqlite-vec/installation.html presumably). The datasette link https://datasette.io/plugins/datasette-sqlite-vec https://datasette.io/plugins/datasette-sqlite-vec is an error 500. On the releases page https://github.com/asg017/sqlite-vec/releases/tag/v0.1.0 https://github.com/asg017/sqlite-vec/releases/tag/v0.1.0 can you explain what is vec0.dll vs sqlite-vec-0.1.0-loadable-windows-x86_64.tar.gz, which also contains a similarly named vec0.dll but of a different size?
- Cieric 2y agoI feel like I've touched a lot of things where something like this is useful (hobby projects). In my case I've done a recommendation engine, music matching (I specifically use it for matching anime to their data), and perceptual hash matching.
- alexgarcia-xyz 2y agoReally curious to hear about what kind of music embedding models/tools you used! I've tried finding some good models before but they were all pretty difficult to use
- Cieric 2y agoIn the past I mainly screwed around with [2] musig as it was fairly easy to modify, right off hand I don't know why I stopped using it. [1] seek-tune (not-shazam) was just post here on hn yesterday, I had to modify it quite a bit to actually get something working in a way I was happy with. Overall it was a little slow, but that's not something I'm gonna fault the author for as the project is still quite new. While browsing around yesterday after messing with [2], I found [3] cpuimage/shazam. It's a little easier for me to screw around with as it's in c, and would probably be the one I try and modify to use this project. Everything is based on the shazam algorithm or something similar, so chunking the audio stream, and fingerprinting it. I was focused on that because it allows me to figure out where OP and ED are in some anime and allows me to add a skip button. Also since anime in the same season typically have the same OP/ED I can use that to try and auto categorize them after the first ones been identified. TL;DR: It's all shazam look alikes, everything else was to annoying to use. [1] https://github.com/cgzirim/seek-tune https://github.com/cgzirim/seek-tune [2] https://github.com/sfluor/musig https://github.com/sfluor/musig [3] https://github.com/cpuimage/shazam https://github.com/cpuimage/shazam
- yard2010 2y agoCan you elaborate about your projects please? What tools are you using?
- 2y ago
- haolez 2y agoWhat's the maximum vector size?
- alexgarcia-xyz 2y agovec0 virtual tables have a hard-coded max of 8192 dimensions, but I can raise that very easily (I wanted to reduce resource exhaustion attacks). But if you're comparing vectors manually, then the `vec_distance_ls()` and related functions have no limits (besides SQLite's 1GB blob limit)
- tareqx3 2y ago[dead]
- dang 2y agoRelated: I’m writing a new vector search SQLite Extension - https://news.ycombinator.com/item?id=40243168 https://news.ycombinator.com/item?id=40243168 - May 2024 (85 comments)
- deepsquirrelnet 2y agoI love this. I know how much work addressing the dependencies must be, but you’re really attacking the right problems. Looking forward to trying this out with my project.
- nattaylor 2y agoI have a use case for this that I'm excited to try. I'm glad AlexG has put so much effort into this. Even the docs are pretty good! My pyenv python3.12.2's sqlite won't load extensions even after installing with what I think are the correct command line flags. Argh! My brew installed python3.12's sqlite will load extensions though, so I can proceed.
- fsndz 2y agoI love this. I am currently doing this tutorial of RAG where the vector DB is simply postgreSQL and pgvector. I guess I can try to reproduce that with SQLite and sqlite-vec now ! Awesome: https://www.lycee.ai/courses/91b8b189-729a-471a-8ae1-717033c77eb5/chapters/a8494d55-a5f2-4e99-a0d4-8a79549c82ad https://www.lycee.ai/courses/91b8b189-729a-471a-8ae1-717033c...
- remram 2y agoBe less obvious about your spam for lycee.ai? You only comment links to your own damn site.
- huevosabio 2y agoBeen using this for video games and it's absolutely awesome. Alex, the author, is also great and very approachable. I've been looking for something like this for a while.
- bcjordan 2y agoCurious what applications for vector search you've found interesting
- 1yefuwang1 2y agoHi, nice work. I write a similar vector search extension https://github.com/1yefuwang1/vectorlite https://github.com/1yefuwang1/vectorlite inspired by sqlite-vss using C++17 and hnswlib. I'd like to do a benchmark to compare it with sqlite-vec, but I guess it is not a fair comparison given that sqlite-vec uses brute-force only. One thing I'd recommend is to include recall rate in your benchmark data. Brute force approach is a good starting point but doesn't scale with serious production workload.
- cotega 2y agoI absolutely love this, great work! For those that might find it useful, I created a Python notebook that shows how to extend this to perform Hybrid Search (Vector + BM25 based Full Text search) https://github.com/liamca/sqlite-hybrid-search https://github.com/liamca/sqlite-hybrid-search