3 ms·
SIMD-accelerated search over binary vectors is very fast and you can fit a lot in memory. Then rerank over f32 from disk. I tried a few alternatives and found
by intalentive 2y ago
SIMD-accelerated search over binary vectors is very fast and you can fit a lot in memory. Then rerank over f32 from disk.
I tried a few alternatives and found that SQLite + usearch extension wins for my use cases (< 1 million records), as measured by latency and simplicity. I believe this approach can scale up to hundreds of millions of records.
- wild_egg 2y agoI've been using DuckDB similarly and loving the simplicity. What sort of latency are you seeing with your setup? I'm hitting 300ms to query against 10M vectors and could see that becoming a bottleneck if going into the hundreds of millions.
- VoVAllen 2y agoHi, I'm the author of the article. Please check out our vector search extension in postgres, VectorChord [1]. It achieves 10ms-level latency for top-10 searches on a 100M dataset and 60ms-level latency when using SSD with limited memory. [1] https://blog.pgvecto.rs/vectorchord-store-400k-vectors-for-1-in-postgresql https://blog.pgvecto.rs/vectorchord-store-400k-vectors-for-1...
- wild_egg 2y agoLook like very nice tech but unfortunately the operational overhead of postgres is too high for my use case. Will keep it in mind for future postgres-backed projects though
- ashvardanian 2y agoDuckDB also uses USearch and the underlying SimSIMD library for kernels, but both were originally designed for a Billion+ scale. Depending on how the wrapper is implemented you can get different numbers. With the raw libraries, on larger machines, I'd expect 200'000 requests per second for 1 Billion entries.