4 ms·
>Particular in the corporate space, the problem of finding relevant knowledge I'm eagerly waiting for the day that I don't get frustrated with Notion's search.
by Hamuko 4y ago
>Particular in the corporate space, the problem of finding relevant knowledge
I'm eagerly waiting for the day that I don't get frustrated with Notion's search. I'm not sure if I actually even bother with the search that much these days. I just try to drill down the page structure to find what looks relevant.
- visarga 4y agoThey could have used HuggingFace models to embed texts, put them in a vector search engine and serve results. - models here: https://sbert.net/docs/pretrained_models.html https://sbert.net/docs/pretrained_models.html - fast vector search here: annoy, faiss, Milvus, Elastic, Pinecone It's like the hello world of current day NLP.
- fzliu 4y agoThanks for the shout-out! For folks interested in playing around with vector and/or hybrid search: Milvus is open-source (https://github.com/milvus-io/milvus https://github.com/milvus-io/milvus).
- gk1 4y agoThis is exactly what Mem does using Pinecone and OpenAI: https://www.pinecone.io/learn/mem-semantic-search/ https://www.pinecone.io/learn/mem-semantic-search/ Almost any self-hosted ANN library will do if you're dealing with <1M vectors, but the indexes at these note-based applications tend to balloon very quickly into the hundreds of millions and billions. Given the popularity of Notion I expect their number of (would-be) vectors to be monstrously big. That's where rolling your own gets painful (or downright not feasible) and where Pinecone shines.[1] [1] https://www.pinecone.io https://www.pinecone.io - Disclosure: I work there.