This is an area that's dear to me, I'm the cofounder of Vectara and have been working with embedding-based semantic search, aka neural search or neural IR, sinc

This is an area that's dear to me, I'm the cofounder of Vectara and have been working with embedding-based semantic search, aka neural search or neural IR, since 2017. To whether Google uses semantic search, the answer is yes, very heavily [1][2]. Not only that, but they have led, and continue to lead, much of the pioneering research in NLP and neural IR for the past decade [3][4][5]. Technical challenges lie along a few primary dimensions. The first has been search quality, because, while early neural systems like Google Talk to Books [5][6] demonstrated the potential of these techniques, benchmarks like BEIR [7], released a few years later, in 2020, showed that the best keyword retrieval algorithms still outperformed neural techniques in general settings. The landscape since then has shifted very rapidly: In 2022, for the first time, neural search methods outperformed BM25 on BEIR. This includes late interaction [8], sparse encoding [9], and, most challengingly, dense encoding [10] systems. The second technical challenge is scalability. After decades of infrastructure optimization, keyword systems scale well to very large corpora, while semantic systems struggle to achieve the same scale. The k-d tree approach presented in the article, for example, while good for experimentation, would be difficult to productionize, as-is, in a large-scale system. However, research into scaling dense vector retrieval has received a lot of focus recently [11], so I'm confident this will change. I'll close by saying your observation about being stuck with keyword search in a lot of apps is accurate, but I expect that to change soon. It's becoming easier to embed neural models everywhere, and I think that distilled models in the 5-50mb size range can feasibly power semantic search everywhere you press Ctrl-F today. [1] https://blog.google/products/search/search-language-understanding-bert/ https://blog.google/products/search/search-language-understa... [2] https://blog.google/products/search/introducing-mum/ https://blog.google/products/search/introducing-mum/ [2] https://arxiv.org/abs/1706.03762 https://arxiv.org/abs/1706.03762 [3] https://arxiv.org/abs/1810.04805 https://arxiv.org/abs/1810.04805 [4] https://arxiv.org/abs/1907.04307 https://arxiv.org/abs/1907.04307 [5] https://books.google.com/talktobooks/ https://books.google.com/talktobooks/ [6] https://ai.googleblog.com/2018/04/introducing-semantic-experiences-with.html https://ai.googleblog.com/2018/04/introducing-semantic-exper... [7] https://arxiv.org/abs/2104.08663 https://arxiv.org/abs/2104.08663 [8] https://arxiv.org/abs/2112.01488 https://arxiv.org/abs/2112.01488 [9] https://arxiv.org/abs/2109.10086 https://arxiv.org/abs/2109.10086 [10] https://arxiv.org/pdf/2112.09118.pdf https://arxiv.org/pdf/2112.09118.pdf [11] https://www.microsoft.com/en-us/research/uploads/prod/2021/11/SPANN_finalversion1.pdf https://www.microsoft.com/en-us/research/uploads/prod/2021/1...