3 ms·
This depends mostly on what kind of search engine you're trying to build. I unfortunately won't be able to point you towards courses, but there are tons of grea
by fzliu 4y ago
This depends mostly on what kind of search engine you're trying to build. I unfortunately won't be able to point you towards courses, but there are tons of great resources online to help you get started.
Search engines are a fairly broad topic, and a lot of it depends on the _type of data_ that you want to build a search engine for. If you're looking towards more traditional, Google/Yahoo-like search, Elasticsearch's learning center (https://www.elastic.co/learn https://www.elastic.co/learn) has quite a few good resources that can point you in the right direction. Many enterprise search solutions are built on top of Apache Lucene (including Elasticsearch), and videos/blogs discussing Lucene's architecture (https://www.endava.com/en/blog/Engineering/2021/Elasticsearch-and-apache-lucene-fundamentals-behind-the-relevance-score https://www.endava.com/en/blog/Engineering/2021/Elasticsearc...) is a great starting point as well.
Opposite text/web search is _unstructured data_ search, i.e. searching across images, video, audio, etc... based on their semantics (https://www.deepset.ai/blog/semantic-search-with-milvus-knowledge-graph-qa-web-crawlers-and-more https://www.deepset.ai/blog/semantic-search-with-milvus-know...). Work in this space has been ongoing for decades, but an emerging way of doing this is via a _vector database_ (https://frankzliu.com/blog/a-gentle-introduction-to-vector-databases https://frankzliu.com/blog/a-gentle-introduction-to-vector-d...) such as Zilliz Cloud (https://zilliz.com/cloud https://zilliz.com/cloud) or Milvus (https://milvus.io/ https://milvus.io/). The idea here is to turn the type of data that you're searching for into an high-dimensional vector called an embedding, and to perform nearest-neighbor search on the embeddings themselves.
Disclaimer: I'm a part of the Zilliz team and Milvus community.