4 ms·

After spending a week evaluating if we are to pick Solr or Elastic, here are some of the more interesting conclusions: - Solr comes with optional security conf

After spending a week evaluating if we are to pick Solr or Elastic, here are some of the more interesting conclusions: - Solr comes with optional security config; Elastic charges for Shield. At the time of the evaluation, Shield pricing wasn't online (and there were hints online it starts at 1600/year and goes upwards from there). - Elastic can have a 'split brain' from time to time, and some data can be lost. - Elastic is more user friendly, but that comes at a price of installing Kibana + other software. Solr is all-in-one. - Both will let you shoot yourself in the foot with dynamic fields and schema, but if you go the hardcoded schema way, solr is easier to setup. - Solr has DataImportHandler, which makes configuring the entire DB import from an existing database, 2 minutes job once you know what you are doing. The import seemed to be doing 50k records / second, and it can also do partial syncs if things have changed. - Performance wise, they are both overkill if one is staying away from managed schemas and provide decently beefed-up VMs. But Solr+DataImportHandler, gives you one more option (aside from zookeeper) for cheap scaling. If you have mostly read heavy instances, you can put separate VMs behind a load balancer to ensure near 100% uptime and robustness, with the minimal fuss of clicking a button once a month to add the deltas to the separate VMs.

ksec 10y ago
I read this as basically a Win to Solr?
- quickben 10y ago
  They are very very close. The differences are minute at this point. Unless you are doing some heavy relational algebra, it will be hard to distinguish real world results for simple queries. In the end I chose Solr because it comes with 'ecosystem' of semi-related projects that you can pull on (spark, ambari, etc etc). On a sidenote, I'm personally more comfortable with forkable sourcebases, than singular companies with paid plugins. I've been too long in this industry and have seen all kinds of dirty tricks pulled off.
  - emmelaich 10y ago
    For us the authentication options might tip it. Also the ecosystem point is important; we're using Spark/Avro/..
- sheeshkebab 10y ago
  I'm using both Solr and Elastic right now (on different projects) - second on those items above. One other plus for Solr is it has a flexible replication and consistency model, out of the box - much better fit for large clusters and multi data center deployments (or multi region high availability AWS deploys). Elastic lacks in replication area (although something appears to be coming soon, via again their paid plugins).
  - IndianAstronaut 10y ago
    How do you handle the data for Solr with AWS rehydration? We are constantly taking backups of the data into S3 to then use to put data back into Solr.
- janhoy 10y ago
  I think what made ES particularly attractive in the beginning was - Easier (linux-pks) install and setup a small cluster - "Just throw some data at it" - JSON and pure REST While ES is still more compact and faster/easier to install, Solr has of course grown up with robust SolrCloud scaling, start scripts, schemaless mode and all the config APIs. And I think the change of mindset from files to APIs is crucial. Your search engine should be like a DB, just install it and then start using it from a client. Your schema stays close to your data in your app where it belongs. A few next important steps for Solr which are being worked on are - New set of APIs /v2/ that gives a more unified experience - More analyzers/tokenfilters/components configurable via APIs - Easier getting-started with basic auth and SSL, add LDAP and more - Mature the streaming/SQL stuff All in all, the competition drives both products to new hights every year! Disclaimer. I'm a Lucene/Solr committer and consult on both products
emmelaich 10y ago
Thanks, excellent review.
aaroninsf 10y ago
Having inherited an ES cluster shortly after it replaced Solr, and built out many clusters since then, I can comment that the 'split brain' problem which attracted a lot of attention 2-3 years ago thanks to the infamous Jepsen 'Call Me Maybe' torture test is effectively a non-issue in a well-configured environment, today. Theoretically possible, perhaps; in practice, just not an issue. Our initial cluster suffered from it thanks to improperly having been run across data centers with default settings of various kinds. Once I got that under control, zero recurrence despite our clusters living in what I like to call a 'hostile network environment.' I am not a fan of the 'Zen' cluster management system they adopted; but it does work and Elastic has put a ton of work into cleaning out the corner cases. For my purposes, One of the distinctions I would say is most interesting today is that Elastic the company has been aggressively chasing the analytics market, at the expense of developing the text/freeform search capabilities of ES. At Elasticon last year there were some interesting talks that touched on this in various ways; and I was part of a group of people who traded chagrin that Elastic does not have a strong user community in this area. Instead they have put a lot of work on their 'stack' and data shippers and cloud deployment. Solr on the other hand has continued to move forward and for an environment in which the primary application is going to be free-form text search (vs. say structured metadata, or analytics) it is probably stronger. Fwiw I find deployment, configuration, inspection, etc., all refreshingly clean. Imperfect, sure, but the ES team did have the advantage of starting with a blank page in a lot of respects. It shows. (Currently running 6+ clusters, 100+ nodes, doing metadata and full-text and a small amount of analytics)
StreamBright 10y ago
Just by experience, be careful with DataImportHandler, if you have many joined tables it can surprise you sometimes with weird behaviour. We ended up building views for just having a simple table to index with Solr.

ksec 10y ago

I read this as basically a Win to Solr?

quickben 10y ago
They are very very close. The differences are minute at this point. Unless you are doing some heavy relational algebra, it will be hard to distinguish real world results for simple queries. In the end I chose Solr because it comes with 'ecosystem' of semi-related projects that you can pull on (spark, ambari, etc etc). On a sidenote, I'm personally more comfortable with forkable sourcebases, than singular companies with paid plugins. I've been too long in this industry and have seen all kinds of dirty tricks pulled off.
- emmelaich 10y ago
  For us the authentication options might tip it. Also the ecosystem point is important; we're using Spark/Avro/..
sheeshkebab 10y ago
I'm using both Solr and Elastic right now (on different projects) - second on those items above. One other plus for Solr is it has a flexible replication and consistency model, out of the box - much better fit for large clusters and multi data center deployments (or multi region high availability AWS deploys). Elastic lacks in replication area (although something appears to be coming soon, via again their paid plugins).
- IndianAstronaut 10y ago
  How do you handle the data for Solr with AWS rehydration? We are constantly taking backups of the data into S3 to then use to put data back into Solr.
janhoy 10y ago
I think what made ES particularly attractive in the beginning was - Easier (linux-pks) install and setup a small cluster - "Just throw some data at it" - JSON and pure REST While ES is still more compact and faster/easier to install, Solr has of course grown up with robust SolrCloud scaling, start scripts, schemaless mode and all the config APIs. And I think the change of mindset from files to APIs is crucial. Your search engine should be like a DB, just install it and then start using it from a client. Your schema stays close to your data in your app where it belongs. A few next important steps for Solr which are being worked on are - New set of APIs /v2/ that gives a more unified experience - More analyzers/tokenfilters/components configurable via APIs - Easier getting-started with basic auth and SSL, add LDAP and more - Mature the streaming/SQL stuff All in all, the competition drives both products to new hights every year! Disclaimer. I'm a Lucene/Solr committer and consult on both products

emmelaich 10y ago

Thanks, excellent review.

aaroninsf 10y ago

Having inherited an ES cluster shortly after it replaced Solr, and built out many clusters since then, I can comment that the 'split brain' problem which attracted a lot of attention 2-3 years ago thanks to the infamous Jepsen 'Call Me Maybe' torture test is effectively a non-issue in a well-configured environment, today. Theoretically possible, perhaps; in practice, just not an issue. Our initial cluster suffered from it thanks to improperly having been run across data centers with default settings of various kinds. Once I got that under control, zero recurrence despite our clusters living in what I like to call a 'hostile network environment.' I am not a fan of the 'Zen' cluster management system they adopted; but it does work and Elastic has put a ton of work into cleaning out the corner cases. For my purposes, One of the distinctions I would say is most interesting today is that Elastic the company has been aggressively chasing the analytics market, at the expense of developing the text/freeform search capabilities of ES. At Elasticon last year there were some interesting talks that touched on this in various ways; and I was part of a group of people who traded chagrin that Elastic does not have a strong user community in this area. Instead they have put a lot of work on their 'stack' and data shippers and cloud deployment. Solr on the other hand has continued to move forward and for an environment in which the primary application is going to be free-form text search (vs. say structured metadata, or analytics) it is probably stronger. Fwiw I find deployment, configuration, inspection, etc., all refreshingly clean. Imperfect, sure, but the ES team did have the advantage of starting with a blank page in a lot of respects. It shows. (Currently running 6+ clusters, 100+ nodes, doing metadata and full-text and a small amount of analytics)

StreamBright 10y ago

Just by experience, be careful with DataImportHandler, if you have many joined tables it can surprise you sometimes with weird behaviour. We ended up building views for just having a simple table to index with Solr.