I wrote something similar a few years ago (finding photos with the most-similar tag vectors among a set of ~500,000). The core lookup was done by a few hundred

I wrote something similar a few years ago (finding photos with the most-similar tag vectors among a set of ~500,000). The core lookup was done by a few hundred lines of C code returning answers in ~0.1s, IIRC. This used a standard info-retrieval method, not a linear-algebra library -- all vectors pre-loaded into RAM in a sparse representation, then for each query scan them all, keeping a min-heap of the best vectors so far. (This could be sped up by indexing the vectors and skipping the ones that with no tags in common with the query, but I didn't need to bother.) Email if you'd like to discuss my experience with this, though it looks like you already get decently fast answers. You can do queries besides "sites most like site X", you know -- "What sites are like a cross of X and Y?"