4 ms·
Are vector databases like Chroma already doing some of this? If not, it could be a huge win for RAG workflows!
by turnsout 3y ago
Are vector databases like Chroma already doing some of this? If not, it could be a huge win for RAG workflows!
- ashvardanian 3y agoSimSIMD is already used in USearch, ClickHouse, and Lantern. In other solutions SIMD is generally limited to pre-2015 Assembly instructions. So yes, there a lot of RAG workflows to accelerate :)
- behnamoh 3y agothe sad part is that they are the ones ripping the benefits of this tech by capturing that sweet VC money
- teaearlgraycold 3y agoI’m curious if pgvector is using it. It’s actually pretty fast for us without even indexing.
- sakras 3y agoAs far as I'm aware pg_vector just uses the compiler's autovectorization on float32. I think specifically for Euclidean distance you won't really beat GCC (the README even admits it: "GCC handles single-precision float but might not be the best choice for int8 and _Float16 arrays, which has been part of the C language since 2011.")
- ashvardanian 3y agoYes, the improvements for float32 aren't very dramatic, but it can be 3x NumPy/SciPy: https://ashvardanian.com/posts/simsimd-faster-scipy/ https://ashvardanian.com/posts/simsimd-faster-scipy/ C11 is also a bit tricky, as its support is optional, as far as I remember.
- sakras 3y agoI also notice you use FMA in your AVX2 L2 distance calculation. I don't think pg_vector enables that, so SimSIMD might be slightly faster. Also interesting that it beats NumPy/SciPy by so much! I wonder what they're doing..
- menaerus 3y agoNot using SIMD at all as it seems. Thus relying on auto-vectorization of I'm not reading it indirectly. https://github.com/scipy/scipy/blob/main/scipy/spatial/src/distance_impl.h https://github.com/scipy/scipy/blob/main/scipy/spatial/src/d...
- teaearlgraycold 3y agoIt doesn’t support f16 or i8 at all yet (at least not in the released version).