7 ms·

I’m writing a new vector search SQLite Extension

526 points by sebg 3y ago

codazoda 3y ago
This is a lot like I imagine “readme driven development” to look like. I’m curious if the author started with docs first.
- alexgarcia-xyz 3y ago
  Thanks for the kind words! I started with code first — the extension itself is already mostly written[0]. But it's one of those "20% effort for 80% of the work," where the last 20% I need to write (error handling, fuzzy tests, correctness testing) will take 80% of the time. But people already have questions about the current status of `sqlite-vss`, so I figured this "work-in-progress" blog post could answer some questions. Though I do like the idea of starting with docs first! Especially with SQLite extensions, where it all really matters what the SQL API looks like (scalar functions, virtual tables, etc.). I definitely did a lot of sketching of what the SQL part of sqlite-vec should look like before writing most of the code. [0] https://github.com/asg017/sqlite-vec/blob/main/sqlite-vec.c https://github.com/asg017/sqlite-vec/blob/main/sqlite-vec.c
  - peter_l_downs 3y ago
    You’ve done a great job communicating the project, in every aspect. Nice work. I’m excited to try this out!
Trufa 3y ago
I love this style of projects, OSS project for a very particular issue. I keep thinking what can I do in the Typescript/Next.js/React ecosystem that's very useful to a technical niche but I haven't had the inspiration yet.
barakm 3y ago
> Thy are binary vectors with 768 dimensions, which takes up 96 bytes (768 / 8 = 96). I guess I’m confused. This is honestly the problem that most vector storage faces (“curse of dimensionality”) let alone the indexing. I assume that you meant 768 dimensions * 8 bytes (for a f64) which is 6144 bytes. Usually, these get shrunk with some (hopefully minor) loss, so like a f32 or f16 (or smaller!). If you can post how you fit 768 dimensions in 96 bytes, even with compression or trie-equivalent amortization, or whatever… I’d love to hear more about that for another post. Ninja edit: Unless you’re treating each dimension as one-bit? But then I still have questions around retrieval quality
- lsb 3y ago
  Binary, quantize each dimension to +1 or -1 You can try out binary vectors, in comparison to quantize every pair of vectors to one of four values, and a lot more, by using a FAISS index on your data, and using Product Quantization (like PQ768x1 for binary features in this case) https://github.com/facebookresearch/faiss/wiki/The-index-factory https://github.com/facebookresearch/faiss/wiki/The-index-fac...
  - barakm 3y ago
    Appreciate the link; but would still like to know how well it works. If you have a link for that, I’d be much obliged
    - lsb 3y ago
      It depends on your data and your embedding model. For example, I was able to quantize embeddings of English Wikipedia from 384-dimensions down to 48 7-bit dimensions, and the search works great: https://www.leebutterman.com/2023/06/01/offline-realtime-embedding-search.html https://www.leebutterman.com/2023/06/01/offline-realtime-emb...
- alexgarcia-xyz 3y ago
  Author here - ya "binary vectors" means quantizing to one bit per dimension. Normally it would be 4 * dimensions bytes of space per vector (where 4=sizeof(float)). Some embedding models, like nomic v1.5[0] and mixedbread's new model[1] are specifically trained to retain quality after binary quantization. Not all models do tho, so results may vary. I think in general for really large vectors, like OpenAI's large embeddings model with 3072 dimensions, it kindof works, even if they didn't specifically train for it. [0] https://twitter.com/nomic_ai/status/1769837800793243687 https://twitter.com/nomic_ai/status/1769837800793243687 [1] https://www.mixedbread.ai/blog/binary-mrl https://www.mixedbread.ai/blog/binary-mrl
alexgarcia-xyz 3y ago
Author here — happy to answer any questions! This is more of a "I'm working on a new project" rather than an official release, the extension itself is still a work-in-progress. A link to the project: https://github.com/asg017/sqlite-vec https://github.com/asg017/sqlite-vec I have a pretty specific vision of what v0.1.0 of this extension will look like, but it'll take a few more weeks to get there. This blog post was more for letting users of sqlite-vss (a previous vector search SQLite extension I wrote) know what will be coming next. There will be a much bigger release when that is ready. But in general, I'm super excited to have an easy embeddable vector search alternative! Especially one that runs on all operating system, in WASM, mobile devices, Raspberry Pis, etc. I personally I'm trying to run a lil' semantic search app on my Beepy[0], which is a ton of fun to play with. [0] https://beepy.sqfmi.com/ https://beepy.sqfmi.com/
- lsb 3y ago
  Would this implement indexing strategies like HNSW? Linear scan is obviously great to start out with, and can definitely be performant, especially if your data is in a reasonable order and under (say) 10MB, so this shouldn't block a beta release. Also do you build with sqlite-httpvfs? This would go together great https://github.com/phiresky/sql.js-httpvfs https://github.com/phiresky/sql.js-httpvfs
  - alexgarcia-xyz 3y ago
    The initial v0.1.0 release will only have linear scans, but I want to support ANN indexes like IVF/HNSW in the future! Wanted to focus on fullscans first to make things easier. In my experiments you can get pretty far with linear scans using sqlite-vec. Depends on the number of dimensions of course, but I'd expect it can handle searching 100's of thousands of vectors to maybe a million with sub-second searches, especially if you tweak some settings (page_size, mmap_size, etc.). And if you quantize your vectors (int8 reduces size 4x, binary 32x) you can get even faster, at the expense of quality. Or use Matryoshka embeddings[0] to shave down dimensions even more. Building sql.js-httpvfs would be cool! Though I'm using the "official" SQLite WASM builds which came out after sql.js, so I don't think it'll be compatible out of the box. Would be very curious to see how much effort it would be to make a new HTTP VFS for SQLite's new WASM builds [0] https://huggingface.co/blog/matryoshka https://huggingface.co/blog/matryoshka
auraham 3y ago
Awesome work! It would be great if you can share a post of how you developed that extension. I am not familiar with SQLite extensions, so I am not sure how to dig into the github repo.
- alexgarcia-xyz 3y ago
  I definitely plan to! I have a much larger list of SQLite extensions I've built here: https://github.com/asg017/sqlite-ecosystem https://github.com/asg017/sqlite-ecosystem Here's a few other references you may enjoy if you wanna learn more about SQLite extensions: - The single source file for sqlite-vec: https://github.com/asg017/sqlite-vec/blob/main/sqlite-vec.c https://github.com/asg017/sqlite-vec/blob/main/sqlite-vec.c - sqlean, a project from Anton Zhiyanov which is good base of great SQLite extensions: https://github.com/nalgeon/sqlean https://github.com/nalgeon/sqlean - The official SQLite docs: https://www.sqlite.org/loadext.html https://www.sqlite.org/loadext.html - The "hello world" SQLite extension example: https://www.sqlite.org/src/file/ext/misc/rot13.c https://www.sqlite.org/src/file/ext/misc/rot13.c
  - auraham 3y ago
    Awesome! Thanks
xrd 3y ago
Very excited about this. When running inside the browser, is sqlite-vec able to persist data into the browser-native indexdb? Or, is this part left to the user? If you can share the thinking there I would appreciate it, even if that is, "no thinking yet!"
- alexgarcia-xyz 3y ago
  It could! It's based on the official SQLite WASM build, so you can use the same persistent options[0] that are offered there. Not sure if IndexedDB is specifically supported, but localStorage/OPFS VFS is available. [0] https://sqlite.org/wasm/doc/trunk/persistence.md#kvvfs https://sqlite.org/wasm/doc/trunk/persistence.md#kvvfs
  - samwillis 3y ago
    OP is correct, the official WASM SQLite build is a sync only build and doesn't support async VFSs (which a IndexedDB VFS needs to be as it's an async api, unless you load the whole file into memory). The best option for IndexedDB backed WASM SQLite is wa-sqlite, it offered both a sync and async build and a bunch of different VFSs, including an IndexedDB one. Note however that the async builds add significant overhead and reduce performance. The most performant VFS is the "OPFS access handle pool" VFS that Roy developed for wa-SQLite and the official build also adopted as an option. That would be my recommendation for now.
ComputerGuru 3y ago
I guess this is an answer (not in answer, mind!) to the GitHub issue I opened against SQLite-vss a couple of months ago? https://github.com/asg017/sqlite-vss/issues/124 https://github.com/asg017/sqlite-vss/issues/124
- alexgarcia-xyz 3y ago
  Yes it is! Sorry for not following up there. Actually, when I first read that ticket, it started me down the rabbit-hole of "how can I make sqlite-vss" better, which eventually turned into "I should make sqlite-vec." So thanks for helping me go down this path! With sqlite-vec's builtin binary quantization, you should be able to do something like: CREATE VIRTUAL TABLE vec_files USING vec0 ( contents_embedding bit[1536] ); INSERT INTO vec_files(rowid,contents_embedding) VALUES ( (1, vec_quantize_binary( /* 1536-dimension float vector here*/)) )
  - ComputerGuru 3y ago
    Hey, you’re welcome! Looking forward to trying this at some point. Sooner if rust bindings are available, later if I have to contribute them. My embedding is already binary, presumably I can bind a blob instead of the call to vec_quantize_binary?
    - alexgarcia-xyz 3y ago
      Yup, you'll be able to bind blobs, you'll currently have to do use vec_bit(?) which is a bit awkward, but it works!
goprams 3y ago
> KNN style query goes brrrr Why do devs write comments like this? What does "goes brrrr" add to anyone's understanding of the code? It's so annoying. Is it supposed to be amusing or cute? It's not. The rest of the article is great but it's so jarring to see this sort of nonsense in amongst a solid piece of work. It's like when you go to a conference and the presentations are peppered with unfunny "meme" pictures. Grow up, people.
- dayjaby 3y ago
  Language evolves. Grow up yourself.
- timmy777 3y ago
  Keep in mind, a presentation (text, or otherwise) is an art. Culture, background, experiences and ideologies influences art. ... and to address your tone, did you know, you can give feedback with empathy?
- efdee 3y ago
  Why do people write comments like this? What does "grow up, people" add to anyone's understanding of the topic? It's so annoying. Is it supposed to make you look better? It doesn't.
zaeger 3y ago
very good
usgroup 3y ago
Shallow insight, but separating the algorithmic/representational and sqlite operational parts into own C projects, is possibly a good idea. I'd expect the rate of evolution to be significantly different between the two parts, and if you are using an algorithm library adopted by others, you may get progress "for free".
- alexgarcia-xyz 3y ago
  Oooh I haven't thought of seperating the SQLite stuff and the vector search stuff. Good idea, will keep that in mind!
elitan 3y ago
Vector search in SQLite is what's keeping me from switching from Postgres.
TheAnkurTyagi 3y ago
This is great. we used Qdrant vector DB for an end to end automation for our AI RAG app at https://github.com/rnadigital/agentcloud https://github.com/rnadigital/agentcloud, but very excited to see you are making a successor. any ETA when it would be ready to use and any quickstart guide? Maybe I can help in writing a blog as well.
- alexgarcia-xyz 3y ago
  I plan to have v0.1.0 in about a month or so! Definitely will include a ton of docs and a quickstart guide. There's an undocumented pip package "sqlite-vec" that you might be able to use now, if you wanted to call it from your Python "Agent Backend" directly.
blizzardman 3y ago
Are you looking for contributor by any chance?
- alexgarcia-xyz 3y ago
  As of now for this beta release not really, but after v0.1.0 is released, will definitely have contributing guides + issues that people could tackle!
babox 3y ago
very cool i like so much sqlite
yard2010 3y ago
Thank you for creating sqlite-vss. It helped me learn how RAG works and implement one in my toy project. It was a bit hard to debug but it worked FLAWLESSLY on ubuntu when done right. I'm still using it. I'm glad you're making a new better version with no limiting deps! You are awesome. Thank you and good luck!
- alexgarcia-xyz 3y ago
  Thanks for the kinds words! Really cool to hear that people were able to use sqlite-vss, lol.
rcarmo 3y ago
Great news. Looking forward to it, as I'm constantly looking for simple solutions that work in constrained environments, and this looks like it's going to be in one of them.
ncruces 3y ago
Do you plan to only use public SQLite APIs, or do you expect to be appended to the amalgamation? I'm definitely interested in something like this, but need to think through how I could distribute this separate from SQLite in my Wasm based Go bindings. So far everything C is bundled, because it's a lot simpler than Wasm "dynamic linking". Also, you've mentioned incremental BLOB I/O, and you probably know this already, but keep in mind that BLOB I/O is never random access, as large BLOBs are stored as a linked list of pages.
- alexgarcia-xyz 3y ago
  Only public SQLite APIs! So no need to append to amalgamation. I'm a big fan of your wazero SQLite bindings! I actually plan on providing 1) CGO bindings to sqlite-vec and 2) a custom WASI build sqlite-vec that can be used in go-sqlite3 directly. My plan was to build a sqlite3.wasm file using the build scripts in your repo. If you did want to support it in your project directly, I think you could just drop the sqlite-vec.c/h file in go-sqlite3/sqlite3 and be good to go Re incremental Blob I/O: I learned that the hard way! It's definitely the limiting factor on query speed for sqlite-vec. I've found that keeping the chunks relatively low in size (low MB's) and increasing the page_size strikes a good balance, but page_size in particular has consequences. PRAGMA mmap_size also helps out a ton, since it seems to keep pages in memory and makes overflow lookups faster, but that of course means a ton more memory usage. It's a difficult balance!
  - ncruces 3y ago
    OK, when you feel it's time, feel free to ping me on my repo, and I'll look into it. A custom Wasm blob definitely works: it's an explicit design goal of my bindings that one can bring their own Wasm (e.g. because one wants to configure the build differently, or bought the SEE extension, or something). And if your extension turns out much like FTS5 that would work. Still, "one big Wasm blob" is less flexible than I'd like, because all connections in an app (currently? maybe this could be lifted) need to use the same Wasm blob. Then (and if you want to import different extensions...) it becomes a fight over who got to initialize the global variable last. So, I've been on-and-off looking into what it would take to "dynamically link" a C extension as a second Wasm, but... I'm not even sure it's possible.
fitzn 3y ago
Cool stuff. I'm probably missing this, but where in the code are you ensuring that all feature vectors have the same number of dimensions (i.e., length)? From what I can tell, for a text value from sqlite, the code converts each char to a float and stores those bits in the vector. This could work if the hamming distance accounts for different length vectors, but that function appears to assume they are the same. Thanks for the clarification.
- alexgarcia-xyz 3y ago
  Internally vectors (at least float vectors) get converted to float * arrays, where the vectors lengths are manually checked. If you provide a vector in JSON form, then it parses that JSON into a float * array. Vectors must be the same length for distance operations.
laurels-marts 3y ago
DuckDB today announced their "Vector Similarity Search in DuckDB" extension. https://duckdb.org/2024/05/03/vector-similarity-search-vss.html https://duckdb.org/2024/05/03/vector-similarity-search-vss.h...
- jasonjmcghee 3y ago
  This is exciting! This could really simplify a little CDN-based HNSW project I did (https://github.com/jasonjmcghee/portable-hnsw https://github.com/jasonjmcghee/portable-hnsw) Seems like with duckdb vss you could just embed, store as duckdb format, then execute sql against it (in the CDN).
jokull 3y ago
libsql (from turso) is also working on this - the more the merrier
michaelsbradley 3y ago
Any thoughts on how your project will compare to CozoDB? https://github.com/cozodb/cozo https://github.com/cozodb/cozo
brainless 3y ago
I have been looking at your extension for the last couple weeks and I think I will end up using it. I am also looking at running qdrant locally. I am creating an open source AI studio (desktop app) (1). It is in very early stages but it is fun to get to know this landscape. I would be happy to contribute to your project in any way I can. And thank you for this project. 1. https://github.com/brainless/dwata https://github.com/brainless/dwata
deleted 2y ago
[deleted]

526 points by sebg 3y ago

This is a lot like I imagine “readme driven development” to look like. I’m curious if the author started with docs first.

alexgarcia-xyz 3y ago
Thanks for the kind words! I started with code first — the extension itself is already mostly written[0]. But it's one of those "20% effort for 80% of the work," where the last 20% I need to write (error handling, fuzzy tests, correctness testing) will take 80% of the time. But people already have questions about the current status of `sqlite-vss`, so I figured this "work-in-progress" blog post could answer some questions. Though I do like the idea of starting with docs first! Especially with SQLite extensions, where it all really matters what the SQL API looks like (scalar functions, virtual tables, etc.). I definitely did a lot of sketching of what the SQL part of sqlite-vec should look like before writing most of the code. [0] https://github.com/asg017/sqlite-vec/blob/main/sqlite-vec.c https://github.com/asg017/sqlite-vec/blob/main/sqlite-vec.c
- peter_l_downs 3y ago
  You’ve done a great job communicating the project, in every aspect. Nice work. I’m excited to try this out!

Trufa 3y ago

I love this style of projects, OSS project for a very particular issue. I keep thinking what can I do in the Typescript/Next.js/React ecosystem that's very useful to a technical niche but I haven't had the inspiration yet.

barakm 3y ago

> Thy are binary vectors with 768 dimensions, which takes up 96 bytes (768 / 8 = 96). I guess I’m confused. This is honestly the problem that most vector storage faces (“curse of dimensionality”) let alone the indexing. I assume that you meant 768 dimensions * 8 bytes (for a f64) which is 6144 bytes. Usually, these get shrunk with some (hopefully minor) loss, so like a f32 or f16 (or smaller!). If you can post how you fit 768 dimensions in 96 bytes, even with compression or trie-equivalent amortization, or whatever… I’d love to hear more about that for another post. Ninja edit: Unless you’re treating each dimension as one-bit? But then I still have questions around retrieval quality

lsb 3y ago
Binary, quantize each dimension to +1 or -1 You can try out binary vectors, in comparison to quantize every pair of vectors to one of four values, and a lot more, by using a FAISS index on your data, and using Product Quantization (like PQ768x1 for binary features in this case) https://github.com/facebookresearch/faiss/wiki/The-index-factory https://github.com/facebookresearch/faiss/wiki/The-index-fac...
- barakm 3y ago
  Appreciate the link; but would still like to know how well it works. If you have a link for that, I’d be much obliged
  - lsb 3y ago
    It depends on your data and your embedding model. For example, I was able to quantize embeddings of English Wikipedia from 384-dimensions down to 48 7-bit dimensions, and the search works great: https://www.leebutterman.com/2023/06/01/offline-realtime-embedding-search.html https://www.leebutterman.com/2023/06/01/offline-realtime-emb...
alexgarcia-xyz 3y ago
Author here - ya "binary vectors" means quantizing to one bit per dimension. Normally it would be 4 * dimensions bytes of space per vector (where 4=sizeof(float)). Some embedding models, like nomic v1.5[0] and mixedbread's new model[1] are specifically trained to retain quality after binary quantization. Not all models do tho, so results may vary. I think in general for really large vectors, like OpenAI's large embeddings model with 3072 dimensions, it kindof works, even if they didn't specifically train for it. [0] https://twitter.com/nomic_ai/status/1769837800793243687 https://twitter.com/nomic_ai/status/1769837800793243687 [1] https://www.mixedbread.ai/blog/binary-mrl https://www.mixedbread.ai/blog/binary-mrl

alexgarcia-xyz 3y ago

Author here — happy to answer any questions! This is more of a "I'm working on a new project" rather than an official release, the extension itself is still a work-in-progress. A link to the project: https://github.com/asg017/sqlite-vec https://github.com/asg017/sqlite-vec I have a pretty specific vision of what v0.1.0 of this extension will look like, but it'll take a few more weeks to get there. This blog post was more for letting users of sqlite-vss (a previous vector search SQLite extension I wrote) know what will be coming next. There will be a much bigger release when that is ready. But in general, I'm super excited to have an easy embeddable vector search alternative! Especially one that runs on all operating system, in WASM, mobile devices, Raspberry Pis, etc. I personally I'm trying to run a lil' semantic search app on my Beepy[0], which is a ton of fun to play with. [0] https://beepy.sqfmi.com/ https://beepy.sqfmi.com/

lsb 3y ago
Would this implement indexing strategies like HNSW? Linear scan is obviously great to start out with, and can definitely be performant, especially if your data is in a reasonable order and under (say) 10MB, so this shouldn't block a beta release. Also do you build with sqlite-httpvfs? This would go together great https://github.com/phiresky/sql.js-httpvfs https://github.com/phiresky/sql.js-httpvfs
- alexgarcia-xyz 3y ago
  The initial v0.1.0 release will only have linear scans, but I want to support ANN indexes like IVF/HNSW in the future! Wanted to focus on fullscans first to make things easier. In my experiments you can get pretty far with linear scans using sqlite-vec. Depends on the number of dimensions of course, but I'd expect it can handle searching 100's of thousands of vectors to maybe a million with sub-second searches, especially if you tweak some settings (page_size, mmap_size, etc.). And if you quantize your vectors (int8 reduces size 4x, binary 32x) you can get even faster, at the expense of quality. Or use Matryoshka embeddings[0] to shave down dimensions even more. Building sql.js-httpvfs would be cool! Though I'm using the "official" SQLite WASM builds which came out after sql.js, so I don't think it'll be compatible out of the box. Would be very curious to see how much effort it would be to make a new HTTP VFS for SQLite's new WASM builds [0] https://huggingface.co/blog/matryoshka https://huggingface.co/blog/matryoshka

auraham 3y ago

Awesome work! It would be great if you can share a post of how you developed that extension. I am not familiar with SQLite extensions, so I am not sure how to dig into the github repo.

alexgarcia-xyz 3y ago
I definitely plan to! I have a much larger list of SQLite extensions I've built here: https://github.com/asg017/sqlite-ecosystem https://github.com/asg017/sqlite-ecosystem Here's a few other references you may enjoy if you wanna learn more about SQLite extensions: - The single source file for sqlite-vec: https://github.com/asg017/sqlite-vec/blob/main/sqlite-vec.c https://github.com/asg017/sqlite-vec/blob/main/sqlite-vec.c - sqlean, a project from Anton Zhiyanov which is good base of great SQLite extensions: https://github.com/nalgeon/sqlean https://github.com/nalgeon/sqlean - The official SQLite docs: https://www.sqlite.org/loadext.html https://www.sqlite.org/loadext.html - The "hello world" SQLite extension example: https://www.sqlite.org/src/file/ext/misc/rot13.c https://www.sqlite.org/src/file/ext/misc/rot13.c
- auraham 3y ago
  Awesome! Thanks

xrd 3y ago

Very excited about this. When running inside the browser, is sqlite-vec able to persist data into the browser-native indexdb? Or, is this part left to the user? If you can share the thinking there I would appreciate it, even if that is, "no thinking yet!"

alexgarcia-xyz 3y ago
It could! It's based on the official SQLite WASM build, so you can use the same persistent options[0] that are offered there. Not sure if IndexedDB is specifically supported, but localStorage/OPFS VFS is available. [0] https://sqlite.org/wasm/doc/trunk/persistence.md#kvvfs https://sqlite.org/wasm/doc/trunk/persistence.md#kvvfs
- samwillis 3y ago
  OP is correct, the official WASM SQLite build is a sync only build and doesn't support async VFSs (which a IndexedDB VFS needs to be as it's an async api, unless you load the whole file into memory). The best option for IndexedDB backed WASM SQLite is wa-sqlite, it offered both a sync and async build and a bunch of different VFSs, including an IndexedDB one. Note however that the async builds add significant overhead and reduce performance. The most performant VFS is the "OPFS access handle pool" VFS that Roy developed for wa-SQLite and the official build also adopted as an option. That would be my recommendation for now.

ComputerGuru 3y ago

I guess this is an answer (not in answer, mind!) to the GitHub issue I opened against SQLite-vss a couple of months ago? https://github.com/asg017/sqlite-vss/issues/124 https://github.com/asg017/sqlite-vss/issues/124

alexgarcia-xyz 3y ago
Yes it is! Sorry for not following up there. Actually, when I first read that ticket, it started me down the rabbit-hole of "how can I make sqlite-vss" better, which eventually turned into "I should make sqlite-vec." So thanks for helping me go down this path! With sqlite-vec's builtin binary quantization, you should be able to do something like: CREATE VIRTUAL TABLE vec_files USING vec0 ( contents_embedding bit[1536] ); INSERT INTO vec_files(rowid,contents_embedding) VALUES ( (1, vec_quantize_binary( /* 1536-dimension float vector here*/)) )
- ComputerGuru 3y ago
  Hey, you’re welcome! Looking forward to trying this at some point. Sooner if rust bindings are available, later if I have to contribute them. My embedding is already binary, presumably I can bind a blob instead of the call to vec_quantize_binary?
  - alexgarcia-xyz 3y ago
    Yup, you'll be able to bind blobs, you'll currently have to do use vec_bit(?) which is a bit awkward, but it works!

goprams 3y ago

> KNN style query goes brrrr Why do devs write comments like this? What does "goes brrrr" add to anyone's understanding of the code? It's so annoying. Is it supposed to be amusing or cute? It's not. The rest of the article is great but it's so jarring to see this sort of nonsense in amongst a solid piece of work. It's like when you go to a conference and the presentations are peppered with unfunny "meme" pictures. Grow up, people.

dayjaby 3y ago
Language evolves. Grow up yourself.
timmy777 3y ago
Keep in mind, a presentation (text, or otherwise) is an art. Culture, background, experiences and ideologies influences art. ... and to address your tone, did you know, you can give feedback with empathy?
efdee 3y ago
Why do people write comments like this? What does "grow up, people" add to anyone's understanding of the topic? It's so annoying. Is it supposed to make you look better? It doesn't.

zaeger 3y ago

very good

usgroup 3y ago

Shallow insight, but separating the algorithmic/representational and sqlite operational parts into own C projects, is possibly a good idea. I'd expect the rate of evolution to be significantly different between the two parts, and if you are using an algorithm library adopted by others, you may get progress "for free".

alexgarcia-xyz 3y ago
Oooh I haven't thought of seperating the SQLite stuff and the vector search stuff. Good idea, will keep that in mind!

elitan 3y ago

Vector search in SQLite is what's keeping me from switching from Postgres.

TheAnkurTyagi 3y ago

This is great. we used Qdrant vector DB for an end to end automation for our AI RAG app at https://github.com/rnadigital/agentcloud https://github.com/rnadigital/agentcloud, but very excited to see you are making a successor. any ETA when it would be ready to use and any quickstart guide? Maybe I can help in writing a blog as well.

alexgarcia-xyz 3y ago
I plan to have v0.1.0 in about a month or so! Definitely will include a ton of docs and a quickstart guide. There's an undocumented pip package "sqlite-vec" that you might be able to use now, if you wanted to call it from your Python "Agent Backend" directly.

blizzardman 3y ago

Are you looking for contributor by any chance?

alexgarcia-xyz 3y ago
As of now for this beta release not really, but after v0.1.0 is released, will definitely have contributing guides + issues that people could tackle!

babox 3y ago

very cool i like so much sqlite

yard2010 3y ago

Thank you for creating sqlite-vss. It helped me learn how RAG works and implement one in my toy project. It was a bit hard to debug but it worked FLAWLESSLY on ubuntu when done right. I'm still using it. I'm glad you're making a new better version with no limiting deps! You are awesome. Thank you and good luck!

alexgarcia-xyz 3y ago
Thanks for the kinds words! Really cool to hear that people were able to use sqlite-vss, lol.

rcarmo 3y ago

Great news. Looking forward to it, as I'm constantly looking for simple solutions that work in constrained environments, and this looks like it's going to be in one of them.

ncruces 3y ago

Do you plan to only use public SQLite APIs, or do you expect to be appended to the amalgamation? I'm definitely interested in something like this, but need to think through how I could distribute this separate from SQLite in my Wasm based Go bindings. So far everything C is bundled, because it's a lot simpler than Wasm "dynamic linking". Also, you've mentioned incremental BLOB I/O, and you probably know this already, but keep in mind that BLOB I/O is never random access, as large BLOBs are stored as a linked list of pages.

alexgarcia-xyz 3y ago
Only public SQLite APIs! So no need to append to amalgamation. I'm a big fan of your wazero SQLite bindings! I actually plan on providing 1) CGO bindings to sqlite-vec and 2) a custom WASI build sqlite-vec that can be used in go-sqlite3 directly. My plan was to build a sqlite3.wasm file using the build scripts in your repo. If you did want to support it in your project directly, I think you could just drop the sqlite-vec.c/h file in go-sqlite3/sqlite3 and be good to go Re incremental Blob I/O: I learned that the hard way! It's definitely the limiting factor on query speed for sqlite-vec. I've found that keeping the chunks relatively low in size (low MB's) and increasing the page_size strikes a good balance, but page_size in particular has consequences. PRAGMA mmap_size also helps out a ton, since it seems to keep pages in memory and makes overflow lookups faster, but that of course means a ton more memory usage. It's a difficult balance!
- ncruces 3y ago
  OK, when you feel it's time, feel free to ping me on my repo, and I'll look into it. A custom Wasm blob definitely works: it's an explicit design goal of my bindings that one can bring their own Wasm (e.g. because one wants to configure the build differently, or bought the SEE extension, or something). And if your extension turns out much like FTS5 that would work. Still, "one big Wasm blob" is less flexible than I'd like, because all connections in an app (currently? maybe this could be lifted) need to use the same Wasm blob. Then (and if you want to import different extensions...) it becomes a fight over who got to initialize the global variable last. So, I've been on-and-off looking into what it would take to "dynamically link" a C extension as a second Wasm, but... I'm not even sure it's possible.

fitzn 3y ago

Cool stuff. I'm probably missing this, but where in the code are you ensuring that all feature vectors have the same number of dimensions (i.e., length)? From what I can tell, for a text value from sqlite, the code converts each char to a float and stores those bits in the vector. This could work if the hamming distance accounts for different length vectors, but that function appears to assume they are the same. Thanks for the clarification.

alexgarcia-xyz 3y ago
Internally vectors (at least float vectors) get converted to float * arrays, where the vectors lengths are manually checked. If you provide a vector in JSON form, then it parses that JSON into a float * array. Vectors must be the same length for distance operations.

laurels-marts 3y ago

DuckDB today announced their "Vector Similarity Search in DuckDB" extension. https://duckdb.org/2024/05/03/vector-similarity-search-vss.html https://duckdb.org/2024/05/03/vector-similarity-search-vss.h...

jasonjmcghee 3y ago
This is exciting! This could really simplify a little CDN-based HNSW project I did (https://github.com/jasonjmcghee/portable-hnsw https://github.com/jasonjmcghee/portable-hnsw) Seems like with duckdb vss you could just embed, store as duckdb format, then execute sql against it (in the CDN).

jokull 3y ago

libsql (from turso) is also working on this - the more the merrier

michaelsbradley 3y ago

Any thoughts on how your project will compare to CozoDB? https://github.com/cozodb/cozo https://github.com/cozodb/cozo

brainless 3y ago

I have been looking at your extension for the last couple weeks and I think I will end up using it. I am also looking at running qdrant locally. I am creating an open source AI studio (desktop app) (1). It is in very early stages but it is fun to get to know this landscape. I would be happy to contribute to your project in any way I can. And thank you for this project. 1. https://github.com/brainless/dwata https://github.com/brainless/dwata

deleted 2y ago

[deleted]