4 ms·
The goal of ElasticSearch, I always thought, was that it scales horizontally and can handle the loss of multiple nodes without availability- or data-loss. It's
by ghayes 7y ago
The goal of ElasticSearch, I always thought, was that it scales horizontally and can handle the loss of multiple nodes without availability- or data-loss. It's interesting to build a single-server replacement, and this can likely work for many use-cases, but it's definitely a different approach from ElasticSearch itself.
- tpayet 7y agoReplication for MeiliSearch is on its way :) The main differentiator is that MeiliSearch algorithms are made for end-user search not for complex queries. MeiliSearch focus on site search or app search, not analytics on hyper large datasets
- rjammala 7y agowhat is the size of the largest dataset that you have indexed with MeiliSearch?
- tpayet 7y agoWe are currently working with this dataset: https://data.discogs.com/?prefix=data/2020/ https://data.discogs.com/?prefix=data/2020/ It's a dataset of 107M songs, 7.6 Gb of compressed files which represents 250 Gb of disk usage by MeiliSearch. We are indexing the release, song and artists names. We also work with a dataset of 2M cities that we can index in less than 2 minutes when the db uses 3 shards.
- nodesocket 7y agoIs it just replication (can sustain node failures) or also sharding the data?
- tpayet 7y agoWe are working on both replication (for high availability and we may use the Raft consensus) and distribution (sharding to scale horizontally and keeping low latency)