6 ms·
Ultra efficient vector extension for SQLite
- simonw 1y agoThis is a neat project - good API design, performance looks impressive. Note that this one isn't open source: https://github.com/sqliteai/sqlite-vector/blob/main/LICENSE.md https://github.com/sqliteai/sqlite-vector/blob/main/LICENSE.... The announcement says: > We believe the community could benefit from sqlite-vector, which is why we’ve made it entirely free for open-source projects. But that's not really how this works. If my open source projects incorporate aspects of this project they're no longer open source!
- rudedogg 1y agoDang, I was really excited about this too. I guess I'll either stick with sqlite-vec or give turso another look. I'm not fond of the idea of a SQLite fork though. Do you know if anything else I should take a look at? I know you use a lot of this stuff for your open-source AI/ML stuff. I'd like something I can use on device.
- Rendello 1y agoYou can point DuckDB at a SQLite file and it will read it using its special columnar format. I'm not sure if that's what you need, though.
- OutOfHere 1y agoIn contrast, https://github.com/asg017/sqlite-vec https://github.com/asg017/sqlite-vec is dual-licensed under Apache and MIT, which makes it open source.
- thorsson12 1y agoAh, yes, this is a "source available" project, not what you would normally call an "open source" project. Still cool!
- tanvach 1y agoThere is the 'Additional Grant for Open-Source Projects' section that seems to permit inclusion in open source project. Do you mind explaining why you think this is not enough? I'm not an expert in licenses so genuinely interested in your take.
- matharmin 1y agoLet's say I have an open-source project licensed under Apache 2. The grant allows me to include the extension in my project. But it doesn't allow me to relicense it under Apache 2 or any other compatible license. So if I include it, my project can't be Apache 2-licensed anymore. Apache 2 is just an example here - the same would apply for practically any open source license. The one place I imagine it could still work is if the open-source project, say a sqlite browser, includes it as an optional plugin. So the project itself stays open-source, but the grant allows using the proprietary plugin with it.
- stavros 1y agoI don't see why this would infect your project, though. You aren't using the code directly, you're using it as a tool dependency, no? Same way as if your OSS project used an Oracle DB to store data.
- nextaccountic 1y agoUnlike Oracle DB, sqlite gets embedded in your program binary. It's a library, not an external service, and this matters for OSS licenses
- stavros 1y agoAh true, I forgot because I always use it in Python, where it's built in.
- simonw 1y agoThe reason I choose to apply open source licenses to my project as I want other people to be able to use them without any limitations (beyond those set out in the open source license I selected, which are extremely permissive.) If they depend on software that carries limitations, I can no longer make that promise to my own users. Or does their extra license term mean I can ship my own project which is the thinnest possible wrapper around theirs but makes it fully open source? That seems unlikely.
- dmezzetti 1y agoOdd licensing strategy here. It's like someone that wants the cachet of saying they are open source without being it.
- MangoToupe 1y agoI guess "free software" is well and truly dead as a term with any general cultural weight.
- theamk 1y agoAs long as threads like this appear, not yet. But not for the lack of trying - people keep trying to redefine it...
- F3nd0 1y agoEven worse, it seems like it’s not Free Software, either.
- froh 1y agofrom the way you say this, it seems you confuse cost free with freedom, free software being about the latter, just implying the former.
- F3nd0 1y agoI was talking about freedom, hence the capitalisation to make that even clearer. The parent only talked about ‘open source’, which has a huge overlap with Free Software, but the two still have different formal definitions (not to mention the completely different ideas behind them). This still left the (admittedly unlikely) possibility of the software in question being Free (as in freedom), so I felt it worth pointing out it wasn’t that, either. A common way to talk about software which is explicitly both Free and open-source at the same time is to call it Free and Open-Source Software.
- Dylan16807 1y agoI think the confusion is that "even worse" sounds like something meaningful but any license split between those two would be quite a fine hair and people tend to treat them as the same. I mean, can you name any licenses that are one or the other but not both? And I explicitly don't mean whether one of OSI or FSF approved a license when the other rejected it, because sometimes they make that decision based on nitpicks and not because of differences in principles.
- fifilura 1y agoHonest question, I just want to learn, what are vector databases used for?
- aaronblohowiak 1y agofinding similar things quickly, where the "shape" of a thing can be defined by a vector (like embeddings for instance). this can be used in lots of machine learning applications
- fifilura 1y agoI figured it would be something like this. And vectors as rows in a regular table would be too slow then?
- sandyarmstrong 1y agosqlite does not have native support for a vector-like column type. Extensions like this and sqlite-vec build on the BLOB column type, and provide additional functions to let you efficiently search and manipulate this vector data.
- simonw 1y agoThey're useful for embeddings, which let you turn articles (and images and other content) into a huge array of floating point numbers that capture the semantics of the content. Then you can use a vector database to find similar items to each other - or similar items to the user's own search query. I wrote a big tutorial about embeddings a couple of years ago which still holds up today: https://simonwillison.net/2023/Oct/23/embeddings/ https://simonwillison.net/2023/Oct/23/embeddings/
- bawolff 1y agoEveryone is saying AI, but also things like image similarity search.
- kamranjon 1y agoI'd be interested to understand the query performance when compared to the HNSW implementation (Turso?) they mentioned. In general search performance is more important to me, and I don't mind having an increase in insert overhead to have very fast vector search.
- dandanua 1y agoHNSW is not accurate. I guess brute-force means that sqlite-vector returns the best match.
- kamranjon 1y agoRight but libsql(Turso) uses HNSW - so i'd be curious to know how performance of sqlite-vector compares - they do say they "Use a brute-force-like approach, but highly optimized." - which to me, would be very interesting to see compared with a HNSW approach.
- simonw 1y agoI believe a common approach with this kind of inaccurate index is to use the index to get the top 100 and then calculate the exact distance against those 100 matches to get the top 10.
- ashvardanian 1y agoMy usual line of feedback would be to start with a more aggressive benchmark. Indexing 100K dense vector (100ish MB here) is not generally a good idea. Brute-force search at that scale is already trivial at 10 GB/s/core.
- sdenton4 1y agoThey say int he post that they're doing optimized brute-force search, which honestly makes a lot of sense for the local-scaled context. Vector databases are often over-optimized for getting results into papers (where the be-all-end-all measure is recall@latency benchmarking). Indexed search is very rigid - filtered search is a pain, and you're stuck with the metric you used for indexing. At smaller data scales, you get a lot more flexibility, and shoving things into a indexed search mindlessly will lead to a lot of pain. Providing optimized flexible search at smaller scales is quite valuable, imo.
- ashvardanian 1y agoAh, I see the article does mention "brute-force-like" — I must have skimmed past that. I'd be curious what exactly is meant by it in practice. A small note: since the project seems to include @maratyszcza's fp16 library (MIT), it might be nice to add a line of attribution: https://github.com/maratyszcza/fp16 https://github.com/maratyszcza/fp16 And if you ever feel like broadening the benchmarks, it could be interesting to compare with USearch. It has had an SQLite connector for a while and covers a wide set of hardware backends and similarity metrics, though it's not yet as straightforward to use on some OSes: https://unum-cloud.github.io/usearch/sqlite https://unum-cloud.github.io/usearch/sqlite
- sdenton4 1y agoTo be clear, I'm not the author of the post. But I do maintain a library for folks working with large audio datasets, built on a combination of SQLite and usearch. :)
- 1y ago
- aabhay 1y agoThe repo mentions approximate NN search but the article implies this is mainly brute force. Is there any indexing at all then? If not, is the approximate part an app-space thing e.g. storing binary vectors alongside the real ones? In addition, if things are brute forced, wouldn’t a columnar db perform better than a row-based one? E.G. DuckDB?
- OutOfHere 1y agoA columnar database is completely irrelevant to vector search. Vectors aren't stored in columns. Traditional indexing too is altogether irrelevant because brute force means a full pass through the data. Specialized indexes can be relevant, but then the search is generally approximate, not exact.
- roflcopter69 1y agoHow's a database being columnar irrelevant to vector search? This very vector search extension shows that brute force search can work surprisingly well up to a certain dataset size and at this point columnar storage is great because it gives a perfect memory access pattern for the vector search instead of iterating over all the rows of a table and only accessing the vector of a row.
- OutOfHere 1y agoThat makes sense. I withdraw my comment.
- mholt 1y ago> [sqlite-vec] works via virtual tables, which means vectors must live in separate tables and queries become more complex. Not really, you can just call the distance function directly and your vector blob can be in any regular field in any regular table, like what I do. Works great. More info: https://github.com/asg017/sqlite-vec/issues/196 https://github.com/asg017/sqlite-vec/issues/196
- roflcopter69 1y agoSorry for not being on topic, just wanted to say hi @mholt and for making and maintaining Caddy! Happy Caddy user here.
- mholt 1y agoThank you, that's always nice to read! I will pass this along to our maintainer team.
- roflcopter69 1y agoIt's unfortunate that this one is not really open source, it has Elastic License 2.0 license. But it's still a wonderful example for how far you can get with brute-force vector similarity search that has been optimized like crazy by making use of SIMD. Not having to use a fancy index is such a blessing. Think of all the complexities that you don't have when not using an index. You don't have these additional worries about memory and/or disk usage or insert/delete/update costs and you can make full use of SQL filters. It's crazy to me what kind of vector DBs people put up with. They use custom query languages, have to hold huge indices in memory or write humongous indices to disk for something that's not necessarily faster than brute-force search for tables with <5M rows. And let's be honest who has those gigantic tables with more than 5M rows? Not too many I'd assume.
- benjiro 1y ago> And let's be honest who has those gigantic tables with more than 5M rows? Not too many I'd assume. /Looks around all innocent ... does 57 billion count? Hate to tell ya but plenty of use cases when you deal with large datasets and normal table design will get out of hand. And row overhead will bite! We ended up writing a specialized encoder / decoder to store the information in bytea fields to reduce it to a measly 3 billion (row packing is the better term) but we also lost some of the advantages of having the database btree index on some of the date fields. Thing is, the moment you deal with big data, things starts to collaps fast if you want to deal with brute force, vs storage vs index handeling. I can think of a lot more projects that can expand to crazy numbers, if you follow basic database normalization.
- roflcopter69 1y ago> /Looks around all innocent ... does 57 billion count? That surely counts as a case where brute-force search will not do :) I'm intrigued though, do you really need to make searches over all those vectors or could you filter the candidates down to something <5M ? As I wrote, this is one of the nice advantages of no-index brute-force search, you can use good 'ol SQL WHERE clauses to limit the amount of candidates in many cases and then the brute-force search is not as expensive. Complex indices like HNSW or DiskANN don't play as nice with filters.
- rcarmo 1y agoSeems to work fine, even if the license is a bit of a put off. I am however still looking for a fast CPU-bound embedding model (fastembed is quite slow on small ARM chips).
- hbcondo714 1y agoThe author replied to one of my comments[1] here on HN a few months ago when I asked about doing ANN on the edge; nice to see it arrive! [1] https://news.ycombinator.com/item?id=44063950 https://news.ycombinator.com/item?id=44063950
- mijoharas 1y agoJust for anyone looking into this, while I haven't tried this solution yet, I can say that sqlite-vec has been very pleasant to work with and I'd recommend it to people who are looking.
- bawolff 1y agoWait they are literally using the domain name https://sqlite.ai https://sqlite.ai while not having any association with the sqlite authors? I know that open source projects rarely protect their trademarks so maybe they are legally in the clear, but this still feels like bordering on fraud. Seriously, look at their logo. This seems like a clear attempt to mislead consumers.
- kg 1y agoTheir company name is apparently SQLite Cloud, inc. and they offer multiple products with SQLite in the name. I guess maybe the sqlite people just don't care about companies using the trademark?
- bawolff 1y agoEven if they don't (or maybe lawyers are too expensive to make it worth it) it still seems pretty scummy to me.
- marcobambini 1y agoWe are backed by the SQLite author (Dr. Richard Hipp), and we have full rights to use the SQLite name.
- deleted 1y ago[deleted]
- marcobambini 1y agoI understand your concerns about the license, but our goal was simply to prevent large corporations from taking our work, forking it, and offering it to their customers while we struggle to sustain development. We need to monetize our work in order to survive, though we do offer very generous commercial licenses for those who are interested.
- misswaterfairy 1y agoWhy not release the source under AGPL (where 'network use' counts as distribution, unlike GPL), and offer commercial licences for those who want more favourable terms? https://fossa.com/blog/dual-licensing-models-explained/ https://fossa.com/blog/dual-licensing-models-explained/ The maintainer of libxml2, Nick Wellnhofer, will be moving all his future contributions over to an AGPL fork as corporate users of libxml2 were unwilling to contribute financially. https://news.ycombinator.com/item?id=45288488 https://news.ycombinator.com/item?id=45288488 https://gitlab.gnome.org/GNOME/libxml2/-/issues/976#note_2538944 https://gitlab.gnome.org/GNOME/libxml2/-/issues/976#note_253...