4 ms·
I have gotten 10x speedups with SIMD on modern hardware - goes from fast to blazing. https://github.com/stillmatic/gollum/blob/main/vectorstore.go https://gith
by huac 3y ago
I have gotten 10x speedups with SIMD on modern hardware - goes from fast to blazing.
https://github.com/stillmatic/gollum/blob/main/vectorstore.go https://github.com/stillmatic/gollum/blob/main/vectorstore.g...
FWIW - Weaviate's pure Go implementation of cosine similarity is probably as fast as you can possibly get in pure go, without SIMD -- https://github.com/weaviate/weaviate/blob/2f44dbee52196656bd6c0eacb588b5f8b926e8e7/usecases/vectorizer/distance.go#L30 https://github.com/weaviate/weaviate/blob/2f44dbee52196656bd...
- phillipcarter 3y agoYep, we'd go with what they did, except OpenAI embedding vectors are already normalized, so you can just do the dot product.
- nearestn 3y agoWeaviate actually does use SIMD for AMD64 with Go assembly (cosine distance is just normalized dot product) https://github.com/weaviate/weaviate/blob/master/adapters/repos/db/vector/hnsw/distancer/asm/dot_amd64.s https://github.com/weaviate/weaviate/blob/master/adapters/re...
- AA-BA-94-2A-56 3y agoDo you need anything special to run a Rust program with SIMD enabled?