3 ms·

So I'm kinda confused. The instance goes down, you lose your data. That's the main reason why Amazon build EBS (granted it has issues of its own) I'm not qui

by dotBen 15y ago

So I'm kinda confused. The instance goes down, you lose your data. That's the main reason why Amazon build EBS (granted it has issues of its own) I'm not quite sure how, in production, this setup really helps you because you KNOW that you will loose your data.

ghotli 15y ago
Some examples where you might not care: HDFS has every block on three DataNodes around the cluster. Cassandra is similar and both will auto heal if a node goes down. Projects like drbd will do block level mirroring of a partition between a set of machines with only one master elected at a time. Some usage patterns don't care if the data is lost, they just care that the I/O is fast. Consider a group of identical machines serving up read-only data round-robin'd via a load balancer. If one goes down, who cares? Only the load balancer does. A new machine gets provisioned and gets added back to the group. DuckDuckGo probably has a data lifecycle that pushes out a new read-only index set on a regular schedule. If the data is kept in S3 and pushed out to read-only nodes then this is a very good use case for only using the ephemeral drives. Servers where writes occur are really the ones that need to be durable. Best case would be for the application to wait until the write was fsync'd on multiple machines if you're faced with node failure at any moment. This topic is kind of a rabbit hole. If you're using ephemeral drives and can't stand to lose data since your last backup then you have to use some sort of multiple machine architecture. Once you do that you're at the mercy of the network. A set of postgres servers in master/slave replication will always have the possibility of losing writes since the slave replication is asynchronous. Cassandra and other quorum-write based systems can make it so that if you write to a single node, a quorum of nodes must agree upon the write before returning the "block written" signal to the client. Really the CAP theorem[1] explains it all. Read the paxos wikipedia article[2] and the chubby paper[3] if you want to see how to keep a set of nodes in total agreement as to the current state of the system. [1] http://en.wikipedia.org/wiki/CAP_theorem http://en.wikipedia.org/wiki/CAP_theorem [2] http://en.wikipedia.org/wiki/Paxos_(computer_science) http://en.wikipedia.org/wiki/Paxos_(computer_science) [3] http://labs.google.com/papers/chubby.html http://labs.google.com/papers/chubby.html

by dotBen 15y ago

ghotli 15y ago

Some examples where you might not care: HDFS has every block on three DataNodes around the cluster. Cassandra is similar and both will auto heal if a node goes down. Projects like drbd will do block level mirroring of a partition between a set of machines with only one master elected at a time. Some usage patterns don't care if the data is lost, they just care that the I/O is fast. Consider a group of identical machines serving up read-only data round-robin'd via a load balancer. If one goes down, who cares? Only the load balancer does. A new machine gets provisioned and gets added back to the group. DuckDuckGo probably has a data lifecycle that pushes out a new read-only index set on a regular schedule. If the data is kept in S3 and pushed out to read-only nodes then this is a very good use case for only using the ephemeral drives. Servers where writes occur are really the ones that need to be durable. Best case would be for the application to wait until the write was fsync'd on multiple machines if you're faced with node failure at any moment. This topic is kind of a rabbit hole. If you're using ephemeral drives and can't stand to lose data since your last backup then you have to use some sort of multiple machine architecture. Once you do that you're at the mercy of the network. A set of postgres servers in master/slave replication will always have the possibility of losing writes since the slave replication is asynchronous. Cassandra and other quorum-write based systems can make it so that if you write to a single node, a quorum of nodes must agree upon the write before returning the "block written" signal to the client. Really the CAP theorem[1] explains it all. Read the paxos wikipedia article[2] and the chubby paper[3] if you want to see how to keep a set of nodes in total agreement as to the current state of the system. [1] http://en.wikipedia.org/wiki/CAP_theorem http://en.wikipedia.org/wiki/CAP_theorem [2] http://en.wikipedia.org/wiki/Paxos_(computer_science) http://en.wikipedia.org/wiki/Paxos_(computer_science) [3] http://labs.google.com/papers/chubby.html http://labs.google.com/papers/chubby.html