6 ms·
Manticore Search: Fast, efficient, drop-in replacement for Elasticsearch
- sandstrom 1y agoFor anyone who's interested, two other popular contenders for replacing Elasticsearch are Typesense (https://typesense.org/ https://typesense.org/) and Meilisearch (https://www.meilisearch.com/ https://www.meilisearch.com/). (both are also trying to replace Algolia, because both have cloud offerings)
- robertlagrant 1y agoDon't forget OpenSearch[0]. [0] https://opensearch.org https://opensearch.org
- smarx007 1y agoIsn't that just an old fork of Elasticsearch?
- mdaniel 1y agoIt is a fork, but not old; they have ongoing commits: https://github.com/opensearch-project/OpenSearch/commits/main https://github.com/opensearch-project/OpenSearch/commits/mai... Plus, given that AWS is currently hosting Open Search, they are not incentivized to sit on their laurels when it comes to modern features or stability
- Keyframe 1y agoWent from ES and sharding hell to less of a sharding hell with OS on AWS. I've been looking for a replacement ever since first friday evening sharding party with infrastructure team.
- cbsmith 1y ago"sharding party" Man, that made me laugh. I'm using that.
- Keyframe 1y agoyeah, you haven't lived until you curled in blind rage _cluster/allocation/explain and _cat/shards?h=index,shard,prirep,state,unassigned.reason | grep UNASSIGNED every few seconds; In production party, with tarantulas on your back asking for status, of course.
- sontek 1y agoI don't believe meilisearch or typesense are API compatible with Elasticsearch. I think the best part of this new tool is its a drop-in replacement. Edit: Nevermind, in another part of this thread the maintainer said: We do get compared to Elasticsearch a lot. While we support some of its APIs, Manticore isn't a drop-in replacement Which conflicts with the README: "Drop-in replacement for E in the ELK stack"
- snikolaev 1y agoIt's not a drop-in replacement in general, but it can be seen as a drop-in replacement for Elasticsearch in the ELK stack because: - You can send data to Manticore using Logstash (L) - You can visualize the data using Kibana (K) Sorry for the confusion :)
- merb 1y agoThat is the most bullshit thing I’ve read in a while. Send data to manticore via logstash does not make you an elasticsearch replacement. And a lot of elasticsearch use cases are not using kibana. (Logstash can basically ingest and output to everything…)
- snikolaev 1y agoBut not everything works with Kibana. What are some well known alternatives besides Manticore?
- nchmy 1y agoFYI, manticore is not "new". It has been around many years, and is based on Sphinx search which has been around longer than, I think, Lucene (which elasticseaech is built on)
- entropyie 1y agoHonourable mention to ZincSearch, if you are looking for a lightweight single binary (golang) alternative: https://github.com/zincsearch/zincsearch https://github.com/zincsearch/zincsearch I have no affiliation.
- mdaniel 1y agoOne should not use "drop-in" when they have their own query language and seemingly input shape for the /search endpoint (which is also different from Elastic, of course) https://manual.manticoresearch.com/Searching/Full_text_matching/Basic_usage#query_string https://manual.manticoresearch.com/Searching/Full_text_match... It sounds like they're really targeting the logging search store part of ELK, which can be a perfectly fine objective, but no need to mislead audiences since they will find out and then you've made an enemy
- tbarbugli 1y agoI agree, only reason I read the project readme was to see the drop-in explainer. Very misleading title
- andygeorge 1y agocc klaussilveira
- klaussilveira 1y agoWell, it says right there in the GitHub About section: "Drop-in replacement for E in the ELK stack".
- snikolaev 1y agoThe Manticore Search github repository calls it a "drop-in replacement for E in the ELK stack," not just a replacement for Elasticsearch. On https://manticoresearch.com/ https://manticoresearch.com/, it's described as an "Elasticsearch alternative," so the confusion is probably just here on HN :)
- snikolaev 1y agoHi everyone — I'm one of the maintainers of Manticore Search. Huge thanks to @klaussilveira for submitting this — we really appreciate the interest and the thoughtful discussion here. A few points that came up in the thread and are worth clarifying: - We do get compared to Elasticsearch a lot. While we support some of its APIs, Manticore isn't a drop-in replacement. We've focused on performance, simplicity, and keeping things open-source without vendor lock-in. Our own SQL-based query language and REST endpoints are part of that philosophy. - @mdaniel was right to question the "drop-in" wording — that's not our goal. - As @sandstrom pointed out, tools like Typesense and Meilisearch are part of this evolving search space. We see Manticore fitting in where users want powerful full-text and vector search capabilities with lower resource overhead and SQL capabilities (we support JSON too though) We'd love to hear from you: - What are your main use cases for search or log indexing? - Which Elasticsearch features (if any) are absolutely essential for you? - Are there performance comparisons or scaling challenges you'd like to see addressed? Happy to answer any questions or dive deeper.
- aaroninsf 1y agoSince you asked! I don't see mention of Lucene on the repo landing page, could you ELI5 the query language and TD-IDF? (Being lazy, I am happy to look into this myself lol.)
- snikolaev 1y agoWe made a blogpost "TF-IDF in a nutshell" - https://manticoresearch.com/blog/tf-idf-in-a-nutshell/ https://manticoresearch.com/blog/tf-idf-in-a-nutshell/ Manticore Search's query language is more expressive than Lucene's. While Lucene supports basic boolean logic, field search, phrase queries, wildcards, and proximity, Manticore adds many powerful operators. These include quorum matching (/N), strict word order (<<), NEAR/NOTNEAR proximity, MAYBE (soft OR), exact form (=word), position anchors (^word, word$), full regular expressions, and more. Manticore uses SQL-style syntax with a MATCH() clause for full-text search, making it easier to combine text search with filters and ranking.
- Semaphor 1y ago
- another_twist 1y agoCurious about the architecture here. Where does the 20x speedup come from ? Recently had a look at Tantivy as well, although compared to raw lucene, their perf is actually inferior. Wonder if there are specific benchmarks here which measure performace and if they compared tail latencies as opposed to averages.
- snikolaev 1y agoThe speedup comes from a number of architectural and low-level performance optimizations in Manticore Search. Manticore has a modern multithreading architecture with efficient query parallelization that fully utilizes all CPU cores. It supports real-time indexing - documents are searchable immediately after insertion, with no need to wait for flushes or refreshes. It uses row-wise storage optimized for small to large datasets, and for even larger datasets that don’t fit into memory, there's support for columnar storage through the Manticore Columnar Library. Secondary indexes are built automatically using the PGM-index (Piecewise Geometric Model index), which enables efficient filtering and sorting by mapping keys to their memory locations. The cost-based query optimizer uses statistics about the data to choose the most efficient execution plan for each query. Manticore is SQL-first: SQL is its native syntax, and it speaks the MySQL protocol, so it works out of the box with MySQL clients. It's written in C++, starts quickly, uses minimal RAM, and avoids garbage collection — which helps keep latencies low and stable even under load. As for benchmarks, there's a growing collection of them at https://db-benchmarks.com https://db-benchmarks.com, where Manticore is compared to Elasticsearch, MySQL, PostgreSQL, Meilisearch, Typesense, and others. The results are open and reproducible.
- 9dev 1y agoIf I had to guess, I would say it’s the 20x smaller feature set compared to Elasticsearch. We built a custom search engine on top of Elasticsearch. Our query builder regularly constructs optimised queries that would be impossible to implement in any of the touted alternatives or replacements, which almost always focus on simple full text search, because that’s everything the developers ever used ES for. There’s a mindboggingly huge number of additional features that you need for serious search engines though, and any contender will have to support at least a subset of these to deserve that title in the first place. I’m keeping an eye on the space, but so far, I’m less than impressed with everything I’ve seen.
- wavemode 1y ago> Manticore Search was forked from Sphinx 2.3.2 in 2017. What was the reason for the fork, and in what ways does Manticore Search differ from Sphinx today?
- aleksi 1y agoSee https://manticoresearch.com/comparison/vs-sphinx/ https://manticoresearch.com/comparison/vs-sphinx/. Sphinx is no longer open-source.
- pQd 1y agosphinx project went half-dead around that time, few years later it got revived but with closed source license. as far as i understand apparent death of sphinx and demand for continued development/support from big users of it led to creation of manticore.
- cess11 1y agoI like Manticore. It's easy to setup, lean on resources and quite fast. I use it when I want to quickly pour a lot of semi-structured text into a database for exploratory browsing and prototype web applications. The auto-bolding of query terms in responses is quite convenient and has allowed me to skip annoying little regexes many times. Maybe other engines have it too and I never noticed?
- snikolaev 1y agoThank you!
- aitchnyu 1y agoI used to auto bold with Solr with <b> tags in 2011. In hindsight, I should have whitelisted only <b> tags for user inputs, resumes in this case. Rambling on, Solr is an Apache product, does tandem releases with Apache Lucene, big sister of Elasticsearch. ES used/uses Apache Lucene too as underlying database.
- cess11 1y agoOK, didn't notice that functionality when I've spent time with those, and to me they feel like more serious commitments, you know? There's more work involved and more maintenance and so on. Especially ElasticSearch is rather fiddly and demanding, and I've seen exactly one production ES-system that impressed me, the rest were log-stores that could have used just about any database engine. With Manticore I just apt it in and start pouring data into it immediately. Similar to how MySQL/MariaDB is almost frictionless when you run it locally with a root account, and neither ever starts to be annoying in resource consumption unless I actively mess up. I typically chose either (or DuckDB) when I have an idea or some ETL task I'd like to implement, over Postgres and ElasticSearch.
- tonyhart7 1y agoI thought we already have elastic search alternative called meilisearch
- mdaniel 1y agorelevant: https://www.meilisearch.com/docs/learn/resources/comparison_to_alternatives#meilisearch-vs-elasticsearch https://www.meilisearch.com/docs/learn/resources/comparison_... MIT: https://github.com/meilisearch/meilisearch/blob/v1.15.2/LICENSE https://github.com/meilisearch/meilisearch/blob/v1.15.2/LICE... from a few months ago: https://news.ycombinator.com/item?id=43680699 https://news.ycombinator.com/item?id=43680699 and the .com has quite a few submissions, but without any obvious commentary upon them https://news.ycombinator.com/from?site=meilisearch.com https://news.ycombinator.com/from?site=meilisearch.com
- pQd 1y agomanticore, earlier sphinx search, has been rock solid for us for the past 16 years. now serving searches across nearly 300M short documents. we're using it in the old mode - where full index is re-created every 24h. it's great to see that the project is alive and adding embeddings-related functions needed for semantic search.
- snikolaev 1y agoGreat to hear! 16 years is impressive. Glad to see the new semantic features caught your eye — we're excited to keep improving the project.