2 ms·
I've recently presented the approach with exhaustive search and heap, then iteratively adding simple indices, using ClickHouse. It works pretty well despite the
by zX41ZdbW 3y ago
I've recently presented the approach with exhaustive search and heap, then iteratively adding simple indices, using ClickHouse. It works pretty well despite the overall usage simplicity.
Video: https://www.youtube.com/watch?v=hGRNcftpqAk https://www.youtube.com/watch?v=hGRNcftpqAk
Presentation: https://presentations.clickhouse.com/meetup74/ai/ https://presentations.clickhouse.com/meetup74/ai/
PS. It should work at the same speed as "bruteforce-blas" in the ANN benchmarks.
- huac 3y agonice, if I'm reading this right, you do random projection as a first stage filter, then exact search over the filtered subset? that's clever, did you test precision/recall at all?
- zX41ZdbW 3y agoYes, this is the idea. I set the threshold for approximate search with a margin so that on a few test queries, I get 100% recall, and then it is filtered by precise search. But I didn't evaluate it thoroughly. The threshold for Hamming distance over bit masks was set almost arbitrarily, and it will be interesting to test further.