I've scaled large transformer based models that supplement a lucene-based search engine. The architecture supports an ensemble approach where Lucene results are

I've scaled large transformer based models that supplement a lucene-based search engine. The architecture supports an ensemble approach where Lucene results are first-class and then we tailor similarity rankings with the models. It looks a lot like this: https://huggingface.co/blog/bert-cpu-scaling-part-1 https://huggingface.co/blog/bert-cpu-scaling-part-1 We have to store large "index" embeddings on SSDs and use leveldb for value retrievals of the lucene results.