5 ms·

Exactly - how can I run a database server without EBS?

by atonse 13y ago

robszumski 13y ago
I think Cassandra is what all the big guys use. Apple, Netflix, etc.
scott_w 13y ago
We use RDS (they now support Postgres) and Elasticache. The pricing seems reasonable for the performance (I've not done benchmarks if I'm being honest), and you get to treat all your EC2 instances as expendable.
- atonse 13y ago
  Sure - but I'm assuming RDS uses EBS under the covers... I meant rather that is there any other option for persistent data on AWS?
- mikebabineau 13y ago
  If you're eschewing EBS because of past problems (like the Oct 2012 outage), you should be aware that RDS is EBS-backed [1]. Also, on EC2, EBS-backed instances boot faster [2]. Of course, you can (and should) still treat EBS-backed instances as expendable. [1] http://aws.amazon.com/rds/faqs/#hardware-scaling http://aws.amazon.com/rds/faqs/#hardware-scaling [2] http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ComponentsAMIs.html#storage-for-the-root-device http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Component...
  - scott_w 13y ago
    Just to clarify: I don't eschew RDS because of outage issues, I use RDS because it's much easier to set up and manage compared to doing it yourself. To be honest, I didn't know RDS was backed by EBS but it makes little real difference to me as long as the backup procedures are rock-solid and the performance is acceptable.
- copergi 13y ago
  As others pointed out, you use EBS then.
jedberg 13y ago
By having good replication, either hand rolled or built in. At Netflix we use Cassandra and store all data on local instance storage. We don't use EBS for databases.
- atonse 13y ago
  Jedberg... this single post blew my mind. :) It's one of those "huh, never ever thought of it from that angle" sort of ideas. The very idea of putting your absolutely valuable data on ephemeral storage and just duplicating it. Thanks for that.
grosskur 13y ago
Run multiple replicas with synchronous replication. For Postgres see: https://wiki.postgresql.org/wiki/Synchronous_replication https://wiki.postgresql.org/wiki/Synchronous_replication This way you should be able to tolerate failure of a single instance without losing data.
- joevandyk 13y ago
  we do async replication with good monitoring of the slaves to make sure the slaves don't get too far behind. couple that with wal-e. of course, if a comet hits the master db server, you might lose a tiny bit data (typically 0.1 seconds) if not using sync replication. you can use sync replication for the important things that you absolutely can't lose, and async replication for everything else.
- copergi 13y ago
  But the thing people are needlessly concerned about with EBS is full datacenter failure. Which is just as much a problem for those EC2 instances. Synchronous replication to another data center is a massive performance killer.

Exactly - how can I run a database server without EBS?

by atonse 13y ago

Exactly - how can I run a database server without EBS?

robszumski 13y ago

I think Cassandra is what all the big guys use. Apple, Netflix, etc.

scott_w 13y ago

We use RDS (they now support Postgres) and Elasticache. The pricing seems reasonable for the performance (I've not done benchmarks if I'm being honest), and you get to treat all your EC2 instances as expendable.

atonse 13y ago
Sure - but I'm assuming RDS uses EBS under the covers... I meant rather that is there any other option for persistent data on AWS?
mikebabineau 13y ago
If you're eschewing EBS because of past problems (like the Oct 2012 outage), you should be aware that RDS is EBS-backed [1]. Also, on EC2, EBS-backed instances boot faster [2]. Of course, you can (and should) still treat EBS-backed instances as expendable. [1] http://aws.amazon.com/rds/faqs/#hardware-scaling http://aws.amazon.com/rds/faqs/#hardware-scaling [2] http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ComponentsAMIs.html#storage-for-the-root-device http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Component...
- scott_w 13y ago
  Just to clarify: I don't eschew RDS because of outage issues, I use RDS because it's much easier to set up and manage compared to doing it yourself. To be honest, I didn't know RDS was backed by EBS but it makes little real difference to me as long as the backup procedures are rock-solid and the performance is acceptable.
copergi 13y ago
As others pointed out, you use EBS then.

jedberg 13y ago

By having good replication, either hand rolled or built in. At Netflix we use Cassandra and store all data on local instance storage. We don't use EBS for databases.

atonse 13y ago
Jedberg... this single post blew my mind. :) It's one of those "huh, never ever thought of it from that angle" sort of ideas. The very idea of putting your absolutely valuable data on ephemeral storage and just duplicating it. Thanks for that.

grosskur 13y ago

Run multiple replicas with synchronous replication. For Postgres see: https://wiki.postgresql.org/wiki/Synchronous_replication https://wiki.postgresql.org/wiki/Synchronous_replication This way you should be able to tolerate failure of a single instance without losing data.

joevandyk 13y ago
we do async replication with good monitoring of the slaves to make sure the slaves don't get too far behind. couple that with wal-e. of course, if a comet hits the master db server, you might lose a tiny bit data (typically 0.1 seconds) if not using sync replication. you can use sync replication for the important things that you absolutely can't lose, and async replication for everything else.
copergi 13y ago
But the thing people are needlessly concerned about with EBS is full datacenter failure. Which is just as much a problem for those EC2 instances. Synchronous replication to another data center is a massive performance killer.