4 ms·
Elasticsearch, definitely. I always recommend using it in hosted form and not running your own cluster. That allows you to focus on getting data in and out of y
by jillesvangurp 7y ago
Elasticsearch, definitely. I always recommend using it in hosted form and not running your own cluster. That allows you to focus on getting data in and out of your cluster instead of sinking time into doing devops.
While I have not used Solr recently, it has evolved along with ES as a solid product with a solid community. Nothing against it; it's a solid choice and there are probably people offering to host that as well. Either way you are using Lucene. Using Lucene directly makes no sense unless you know what you are doing and have a need to do that. If you have to ask that means it's not for you.
If search is important to your use case and things like relevance, precision, and recall have real impact on your business, you should get some specialists involved and not reinvent the wheel or make all the rookie mistakes. Somebody like me basically ;-). However, that can be expensive and if search is not that critical, just sign up for one of the search as a plug in solution type products out there and don't bother running a lot of infrastructure. E.g. Elastic offers a thing called App Search and it probably covers most of the simple needs and is stupidly easy to get started with. There are several competing products probably that I can't vouch for.
You can always upgrade to something proper later. Things like mongo and postgres also have some limited capabilities here and you can get away with doing some simplistic stuff with sql even. However, there's a point where you hit a brick wall and there's something you need that is simply hard/impossible using that or where you end up reinventing a lot of stuff that things like Elasticsearch probably do better.
- busterarm 7y ago> Elasticsearch, definitely. I always recommend using it in hosted form and not running your own cluster. That allows you to focus on getting data in and out of your cluster instead of sinking time into doing devops. I think everyone doing Elasticsearch well has to bring it in-house eventually. AWS's hosted solution is poor, Logz.io and ElasticCloud are expensive. There's a 7-figure/yr Elastic Cloud customer I work with who is so tired of Elastic just randomly killing their clusters out of nowhere and having to spend basically triple to deal with it that they're bringing it all in house.
- showerst 7y agoI think it depends massively on how much data you have. A great many companies and websites only need a few hundred megs of text data indexed, which is easier to outsource. Once you grow larger than that though, the hosted service prices get astronomical compared to standing up a cluster, assuming you have someone who can admin it.
- devonkim 7y agoThings are not quite cut and dry operationally. Some places may have only a few hundred MB of data but they need really high availability and performance (probably picked the wrong solution IMO but still...) which is better guaranteed hosting it yourself and perhaps even outside the cloud. Most places the availability of whatever a hosted solution provides is a better value than spending engineering hours to maintain these things. Devops / SRE folks are expensive compared to most other engineers that deliver features primarily. My point is that a ton of companies (more for cultural reasons than business requirements IME) are so freaked out by any downtime in any way that they’re going to pay for an engineer to just maintain these things themselves.
- busterarm 7y agoThe only issue I have with this suggestion is that all of the situational awareness needed to _use_ Elasticsearch effectively is the same as is needed to _operate_ Elasticsearch effectively. If you're just spinning up something's defaults and throwing data in it, that bill is going to due eventually and it's probably going to be ugly.
- amelius 7y agoAlso, some clients may prohibit uploading of data to third parties. In those cases you simply have no other choice than to run your own cluster.
- ragle 7y ago> AWS's hosted solution is poor What has been poor about it in your experience?
- ageyfman 7y agoWe host a smallish (12 large node) cluster ourselves on AWS, and it's been nothing but SUPER SOLID for us. Literally 0 issues in the year. We use it for analytics, aggregations and the like, as well as domain-specific search, for which it's a great fit.
- fredsted 7y agoI agree, if ES is scaled well for your usage it just keeps going. However, a big time sink for me has been reindexing, lost data due to bad mapping, and version incompatibilities.
- anaphor 7y agoI've found https://www.npmjs.com/package/elasticdump https://www.npmjs.com/package/elasticdump to be VERY useful for doing things like copying indices and recovering data. Much more than the built in stuff.