A practical tip: try hard not to need this sort of thing, and you'd be surprised how often you can avoid it. Before you start looking at approximate nearest ne

A practical tip: try hard not to need this sort of thing, and you'd be surprised how often you can avoid it. Before you start looking at approximate nearest neighbors, ask yourself the following: 1. Do I know all of the vectors ahead of time? (i.e. query vectors are not constructed at runtime, it's a fixed inventory of items being embedded) 2. Do I need to return an unlimited number of neighbors per query? (i.e. do you ever need more than like, a few thousand neighbors) If the answer to both of these is "no", then you should probably just precompute the neighbors exactly on one or more GPUs. It's feasible for millions of terms, and may actually be faster than the index-building steps for many ANN algorithms. Example code: https://github.com/explosion/sense2vec/blob/master/scripts/06_precompute_cache.py https://github.com/explosion/sense2vec/blob/master/scripts/0... A cache of k neighbors for vectors of dimension d will be k/d the size of the original vector data, because one integer and one float both occupy 32-bits of data. So if your vectors are 300 dimensions, you can save 150 neighbors and serve the results with half the space of the original data. The responses are then instant and exact, and you'll never have problems with load management because the runtime is so cheap. Example demo: https://explosion.ai/demos/sense2vec https://explosion.ai/demos/sense2vec Again, this doesn't work for queries you have to construct on the fly, for instance it won't work if you need to construct a vector that incorporates information from the user's session or something. It only works if the process of constructing the vectors is already a batch job to e.g. embed the user's history. It also won't work well if you have to page through arbitrary amounts of history, as storing more than a few thousand neighbors per query would get impractical. Generally the similarity accuracy degrades to uselessness long before that point though (especially if you're using an approximation technique).