4 ms·

Every time someone comes up with an alternative to a software behemoth like Elasticsearch, what they actually mean is: "An alternative to the 10% of functionali

by 9dev 4y ago

Every time someone comes up with an alternative to a software behemoth like Elasticsearch, what they actually mean is: "An alternative to the 10% of functionality of $tool that are interesting to me". This is surely an impressive engineering feat, but hardly a replacement for the myriad of query possibilities Elasticsearch offers.

marginalia_nu 4y ago
To be fair, there is often a better reasons to only replace a portion of ES' functionality, since doing so can save a lot of computation and space; than to replace ES itself, since it already exists and does a good job if what you need is the full kit. I found myself last week reimplementing 10% of RoaringBitmap's functionality as a homebrew replacement, because doing so was 500% faster. Not that RB isn't great, but it's designed for a general problem space, and not my particular problem.
- unrealhoang 4y ago
  can you explain more of your problem and solution? Or is your solution open/published anywhere?
jamil7 4y ago
Agreed, although to be fair to the actual author (assuming they didn't post this here) the readme is a lot more upfront about it's capabilities. > Sonic can be used as a simple alternative to super-heavy and full-featured search backends such as Elasticsearch in some use-cases.
RicoElectrico 4y ago
Honestly most of the "alternative to" programs do not meet expectations they set by dropping a big known name. So much so, that I think people are doing FOSS disservice by comparing to those who they can't meaningfully overtake. The only exceptions could be small single feature utilities.
- graftak 4y ago
  To me it seems the “alternative to” part is more damaging in that sense than dropping a big name. The name is used to put a complicated piece of software in a context many people are familiar with. The same thing happens with the “Tinder/Uber/Airbnb for <x>…” type of services. The friction is introduced where it’s not made crystal clear how it’s similar, and which concept are different or missing altogether. Then it will cause unmet expectations. Perhaps it’s better to say “inspired by …” or “similar to …” to make a more precise statement.
rlex 4y ago
and almost all of them won't offer ES-compatible API. Out of my head i can think about manticore (https://manticoresearch.com/ https://manticoresearch.com/) that offers at least subset of elasticsearch API
- sanxiyn 4y ago
  Quickwit does too (a subset anyway): https://github.com/quickwit-oss/quickwit https://github.com/quickwit-oss/quickwit
ianbutler 4y ago
I don't think your opinion is wrong, but I do think ElasticSearch has a lot of features that many people consider bloat depending on their work, and scaling and doing general dev ops for ES can be an absolute slog. Light weight alternatives that cut down to a set of core features for some niche seem like a good idea to me.
- 9dev 4y ago
  It's totally fine that many people consider stuff bloat, but other people don't. I've built a highly specialised search engine for manufacturing companies on top of Elasticsearch, and I decidedly need vector queries, TF-IDF queries, geospatial range queries, and heaps of other, niche features you probably never used before. Having a lightweight search engine is fine, but calling it an alternative to Elasticsearch is not doing either justice.
  - ianbutler 4y ago
    That's very assumptive of you. I have in fact used most of those features, and note I said their opinion was not wrong. In their readme they said it's a replacement for some use cases which is upfront and fine. Vector queries aren't niche, Elastic however only tacked on a proper (non HNSW) implementation in the last year and a half. Geospatial isn't niche, anyone working with location data will work with those queries. TF-IDF is a basic ranking algo / signal. Maybe Elasticsearch is good for you because they have all their features in aggregate. But I can name a tool that focuses specifically on each area and query type and is better for that specific subset of functionality. So my point still stands, if all you need are specific features Elastic is too much. You need all of it and that's fine too.
    - pbowyer 4y ago
      > But I can name a tool that focuses specifically on each area and query type and is better for that specific subset of functionality. Please do name them, because I for one would like to never run ElasticSearch again for faceted, full-text and specialised search.
    - 4y ago
osigurdson 4y ago
I agree, but ES should re-write their core engine to be more lightweight, otherwise a viable competitor will emerge.
- snorremd 4y ago
  Projects like Meili Search are already coming for Elastic Search's lunch: https://www.meilisearch.com https://www.meilisearch.com. I think there is a market for fast, light weight alternatives like Meili that offers up a fully featured open source experience. With Elastic Search many of the features, security being one, are locked away behind commercial licenses. With Meili it seems they are, for the time being anyway, going with a proper open source version. I understand Elastic needs to earn money, and I get their licensing model to accomplish this. But Meili will probably steal away a good portion of customers interested in self hosting their search solution.
  - osigurdson 4y ago
    I’m not sure what this competitor will be but > ES will have the following properties: - written in rust or maybe just C - extremely lightweight and high performance - single small binary that runs anywhere - designed to run in Kubernetes from the ground up - scales dynamically up/down - zero downtime upgrades - rigorous security built into the core offering - fully open source - wire compatibility with ES I hope that ES themselves do this. There are pretty significant barriers to creating a serious competitor to ES (unlike something like MongoDB for example which seems to have a very limited role in the future).
coldtea 4y ago
"ative to a software behemoth like Elasticsearch, what they actually mean is: "An alternative to the 10% of functionality of $tool that are interesting to me" Which is perfectly fine. A lot of tools become so general and bloated, that there are large groups that would be fine with many different 10% subsets of their features... Kind of like how I don't need MS Word or OpenOffice Write, any simple text editing program with a few basic features (like printing, bold/italics, and word count) will do for my needs...
- 9dev 4y ago
  I'm not opposed to that, however, the chance of their 10% and my 10% overlapping is rather slim. Just like you only need basic formatting, and I require footnotes in my documents. Nothing wrong with either, but I'd be upset if you tried to sell me GEdit as a replacement for OpenOffice Write.
GrinningFool 4y ago
From the opening line of the README: "Sonic can be used as a simple alternative to super-heavy and full-featured search backends such as Elasticsearch in some use-cases." Seem pretty up-front about it, and doesn't claim to be a full-featured alternative.
- lolinder 4y ago
  Agreed, they do a good job of hedging it. I think OP was probably pre-empting the usual comments along the lines of "yep, $tool is super bloated, $smallerTool proves that those other guys building $tool are bad engineers."
tensor 4y ago
My guess is that the majority of people using ES could actually use something simpler like this.
jethro_tell 4y ago
well, and how do you solve excessive ram usage for a search engine? Generally you write the indexes/search trees to disk which, may or may not be ideal.
felipellrocha 4y ago
That is exactly what they are, and I don't think they hide it?! So, I don't know what the issue it. This is the kind of innovation that keeps us moving forward.
manigandham 4y ago
True, but most deployments are also just generic searching of records like Algolia rather than using all the low-level functionality. Tyoesense is probably the most compete competitor in that regard: https://typesense.org/ https://typesense.org/ Other alternatives here: https://gist.github.com/manigandham/58320ddb24fed654b57b4ba22aceae25 https://gist.github.com/manigandham/58320ddb24fed654b57b4ba2...
papruapap 4y ago
tbh "Text Search" is a vague description for these kind of softwares, so I guess everyone go with elasticsearch-like.

by 9dev 4y ago

marginalia_nu 4y ago

To be fair, there is often a better reasons to only replace a portion of ES' functionality, since doing so can save a lot of computation and space; than to replace ES itself, since it already exists and does a good job if what you need is the full kit. I found myself last week reimplementing 10% of RoaringBitmap's functionality as a homebrew replacement, because doing so was 500% faster. Not that RB isn't great, but it's designed for a general problem space, and not my particular problem.

unrealhoang 4y ago
can you explain more of your problem and solution? Or is your solution open/published anywhere?

jamil7 4y ago

Agreed, although to be fair to the actual author (assuming they didn't post this here) the readme is a lot more upfront about it's capabilities. > Sonic can be used as a simple alternative to super-heavy and full-featured search backends such as Elasticsearch in some use-cases.

RicoElectrico 4y ago

Honestly most of the "alternative to" programs do not meet expectations they set by dropping a big known name. So much so, that I think people are doing FOSS disservice by comparing to those who they can't meaningfully overtake. The only exceptions could be small single feature utilities.

graftak 4y ago
To me it seems the “alternative to” part is more damaging in that sense than dropping a big name. The name is used to put a complicated piece of software in a context many people are familiar with. The same thing happens with the “Tinder/Uber/Airbnb for <x>…” type of services. The friction is introduced where it’s not made crystal clear how it’s similar, and which concept are different or missing altogether. Then it will cause unmet expectations. Perhaps it’s better to say “inspired by …” or “similar to …” to make a more precise statement.

rlex 4y ago

and almost all of them won't offer ES-compatible API. Out of my head i can think about manticore (https://manticoresearch.com/ https://manticoresearch.com/) that offers at least subset of elasticsearch API

sanxiyn 4y ago
Quickwit does too (a subset anyway): https://github.com/quickwit-oss/quickwit https://github.com/quickwit-oss/quickwit

ianbutler 4y ago

I don't think your opinion is wrong, but I do think ElasticSearch has a lot of features that many people consider bloat depending on their work, and scaling and doing general dev ops for ES can be an absolute slog. Light weight alternatives that cut down to a set of core features for some niche seem like a good idea to me.

9dev 4y ago
It's totally fine that many people consider stuff bloat, but other people don't. I've built a highly specialised search engine for manufacturing companies on top of Elasticsearch, and I decidedly need vector queries, TF-IDF queries, geospatial range queries, and heaps of other, niche features you probably never used before. Having a lightweight search engine is fine, but calling it an alternative to Elasticsearch is not doing either justice.
- ianbutler 4y ago
  That's very assumptive of you. I have in fact used most of those features, and note I said their opinion was not wrong. In their readme they said it's a replacement for some use cases which is upfront and fine. Vector queries aren't niche, Elastic however only tacked on a proper (non HNSW) implementation in the last year and a half. Geospatial isn't niche, anyone working with location data will work with those queries. TF-IDF is a basic ranking algo / signal. Maybe Elasticsearch is good for you because they have all their features in aggregate. But I can name a tool that focuses specifically on each area and query type and is better for that specific subset of functionality. So my point still stands, if all you need are specific features Elastic is too much. You need all of it and that's fine too.
  - pbowyer 4y ago
    > But I can name a tool that focuses specifically on each area and query type and is better for that specific subset of functionality. Please do name them, because I for one would like to never run ElasticSearch again for faceted, full-text and specialised search.
  - 4y ago

osigurdson 4y ago

I agree, but ES should re-write their core engine to be more lightweight, otherwise a viable competitor will emerge.

snorremd 4y ago
Projects like Meili Search are already coming for Elastic Search's lunch: https://www.meilisearch.com https://www.meilisearch.com. I think there is a market for fast, light weight alternatives like Meili that offers up a fully featured open source experience. With Elastic Search many of the features, security being one, are locked away behind commercial licenses. With Meili it seems they are, for the time being anyway, going with a proper open source version. I understand Elastic needs to earn money, and I get their licensing model to accomplish this. But Meili will probably steal away a good portion of customers interested in self hosting their search solution.
- osigurdson 4y ago
  I’m not sure what this competitor will be but > ES will have the following properties: - written in rust or maybe just C - extremely lightweight and high performance - single small binary that runs anywhere - designed to run in Kubernetes from the ground up - scales dynamically up/down - zero downtime upgrades - rigorous security built into the core offering - fully open source - wire compatibility with ES I hope that ES themselves do this. There are pretty significant barriers to creating a serious competitor to ES (unlike something like MongoDB for example which seems to have a very limited role in the future).

coldtea 4y ago

"ative to a software behemoth like Elasticsearch, what they actually mean is: "An alternative to the 10% of functionality of $tool that are interesting to me" Which is perfectly fine. A lot of tools become so general and bloated, that there are large groups that would be fine with many different 10% subsets of their features... Kind of like how I don't need MS Word or OpenOffice Write, any simple text editing program with a few basic features (like printing, bold/italics, and word count) will do for my needs...

9dev 4y ago
I'm not opposed to that, however, the chance of their 10% and my 10% overlapping is rather slim. Just like you only need basic formatting, and I require footnotes in my documents. Nothing wrong with either, but I'd be upset if you tried to sell me GEdit as a replacement for OpenOffice Write.

GrinningFool 4y ago

From the opening line of the README: "Sonic can be used as a simple alternative to super-heavy and full-featured search backends such as Elasticsearch in some use-cases." Seem pretty up-front about it, and doesn't claim to be a full-featured alternative.

lolinder 4y ago
Agreed, they do a good job of hedging it. I think OP was probably pre-empting the usual comments along the lines of "yep, $tool is super bloated, $smallerTool proves that those other guys building $tool are bad engineers."

tensor 4y ago

My guess is that the majority of people using ES could actually use something simpler like this.

jethro_tell 4y ago

well, and how do you solve excessive ram usage for a search engine? Generally you write the indexes/search trees to disk which, may or may not be ideal.

felipellrocha 4y ago

That is exactly what they are, and I don't think they hide it?! So, I don't know what the issue it. This is the kind of innovation that keeps us moving forward.

manigandham 4y ago

True, but most deployments are also just generic searching of records like Algolia rather than using all the low-level functionality. Tyoesense is probably the most compete competitor in that regard: https://typesense.org/ https://typesense.org/ Other alternatives here: https://gist.github.com/manigandham/58320ddb24fed654b57b4ba22aceae25 https://gist.github.com/manigandham/58320ddb24fed654b57b4ba2...

papruapap 4y ago

tbh "Text Search" is a vague description for these kind of softwares, so I guess everyone go with elasticsearch-like.