3 ms·

Elasticsearch is one of the easiest clustered databases to administrate, and it's a lot cheaper to just run your own EC2 instances with ES than it is to use the

by cheald 10y ago

Elasticsearch is one of the easiest clustered databases to administrate, and it's a lot cheaper to just run your own EC2 instances with ES than it is to use the managed ES service. A roll-your own deployment is basically "install Elasticsearch, install cloud-aws, configure cloud-aws with a security group ID to identify other machines in the cluster, add that security group to all machines you want to cluster together, start the service". Totally braindead-easy.

Xylakant 10y ago
I do among other things elasticsearch consulting and a significant part of the urgent jobs I do is fixing braindead-easy ES setups. It's easy to install and configure, it will probably even run fine with a small dataset or few queries, but it will start to fall apart if you're putting load on it and haven't thought at least a little about your setup. That's not a surprising thing in itself, practically no piece of software will just eat any garbage you throw at it, so ES is at least on par with a lot of other options out there. It's often just a small thing, tweaking shard counts and index sizes, using the appropriate instance type, something like that. But it's still easy to get wrong and a lot of people do.
fizx 10y ago
> Elasticsearch is one of the easiest clustered databases to administrate You seem to have installation confused with administration. Off the top of my head you forgot security, monitoring, logging config, backups, handling common production issues such as splitbrains, write multiplication, garbage collection snafus, upgrades between versions with questionably compatible internal apis. Running any database in the cloud is non-trivial, and, running elasticsearch effectively in the cloud is harder than running Zookeeper or Cassandra, for example. Source: I founded and sold an ElasticSearch hosting company.
- cheald 10y ago
  Much of that isn't specific to Elasticsearch; security, backups, logging config, etc is all going to be extremely common across just about every similar product. I didn't say that Elasticsearch was "trivial", I said that it's one of the easiest clustered DBs to administrate. Anything distributed has a whole set of caveats that you have to be aware of and know how to handle, but using managed ES vs hosting your own doesn't really change any of that. My point is that the value-add from AWS' managed ES product is much, much smaller than it might be for other similar products because of the relative ease of administrating ES.
  - AznHisoka 10y ago
    "easiest clustered DBs to administrate" easy + clustered should never be in the same sentence.
notyourwork 10y ago
The part you forget with this is the people cost. A company moving to an AWS (or other cloud provider solution) may be able to drop a human from their staff. This is worth a lot more than saving a few dollars rolling your own ES. Also when you roll your own you usually maintain, if not increase staff count. Cloud providers afford businesses the ability to look at decreasing staffing while increasing capacity.

by cheald 10y ago

Xylakant 10y ago

I do among other things elasticsearch consulting and a significant part of the urgent jobs I do is fixing braindead-easy ES setups. It's easy to install and configure, it will probably even run fine with a small dataset or few queries, but it will start to fall apart if you're putting load on it and haven't thought at least a little about your setup. That's not a surprising thing in itself, practically no piece of software will just eat any garbage you throw at it, so ES is at least on par with a lot of other options out there. It's often just a small thing, tweaking shard counts and index sizes, using the appropriate instance type, something like that. But it's still easy to get wrong and a lot of people do.

fizx 10y ago

> Elasticsearch is one of the easiest clustered databases to administrate You seem to have installation confused with administration. Off the top of my head you forgot security, monitoring, logging config, backups, handling common production issues such as splitbrains, write multiplication, garbage collection snafus, upgrades between versions with questionably compatible internal apis. Running any database in the cloud is non-trivial, and, running elasticsearch effectively in the cloud is harder than running Zookeeper or Cassandra, for example. Source: I founded and sold an ElasticSearch hosting company.

cheald 10y ago
Much of that isn't specific to Elasticsearch; security, backups, logging config, etc is all going to be extremely common across just about every similar product. I didn't say that Elasticsearch was "trivial", I said that it's one of the easiest clustered DBs to administrate. Anything distributed has a whole set of caveats that you have to be aware of and know how to handle, but using managed ES vs hosting your own doesn't really change any of that. My point is that the value-add from AWS' managed ES product is much, much smaller than it might be for other similar products because of the relative ease of administrating ES.
- AznHisoka 10y ago
  "easiest clustered DBs to administrate" easy + clustered should never be in the same sentence.

notyourwork 10y ago

The part you forget with this is the people cost. A company moving to an AWS (or other cloud provider solution) may be able to drop a human from their staff. This is worth a lot more than saving a few dollars rolling your own ES. Also when you roll your own you usually maintain, if not increase staff count. Cloud providers afford businesses the ability to look at decreasing staffing while increasing capacity.