Somewhat related, if someone want to build something awesome, I haven't seen anything that merges lucene with BPE/SentencePiece. SentencePiece has to make it s

Somewhat related, if someone want to build something awesome, I haven't seen anything that merges lucene with BPE/SentencePiece. SentencePiece has to make it so you can shrink the memory requirements of your indexes for search and typeahead stuff.