4 ms·
Being from 2017 the article misses some of the coolest advances in semantic search, which is now pretty easy and lets you search in (almost) the same way they w
by jamesbriggs 5y ago
Being from 2017 the article misses some of the coolest advances in semantic search, which is now pretty easy and lets you search in (almost) the same way they would ask a shop assistant when looking for something specific - "do you know where the thing with the cool circles and pointy bits is?" (maybe being a little more specific...)
Google do this and they're very good at it, but a lot of companies need their own search capabilities - think about those internal help pages. They usually seem super outdated compared to the semantic search capabilities of Google.
In the end there's only a few components to it, you use some NLP model to create what are called 'dense vectors'. Then you put all these dense vectors into an 'index' which is optimized for fast search (that comes under the umbrella of ANN search). Then given a new query you just compare that to the items in the index and return the most similar results.
I covered the search part of it (https://www.pinecone.io/learn/ https://www.pinecone.io/learn/) with Pinecone, who provide managed-search - although here we mainly focus on Faiss (a great engine from Facebook AI). However, we're also looking to create some content covering the first half too, which is 'how to build dense vectors' using models like BERT, we have one post so far on that (https://www.pinecone.io/learn/dense-vector-embeddings-nlp/ https://www.pinecone.io/learn/dense-vector-embeddings-nlp/)