3 ms·

Forgive my ignorance, but could someone explain in layman's terms in which situation this would be helpful? E.g. if I have 1TB of data would I use this? If I ha

by tabeth 10y ago

Forgive my ignorance, but could someone explain in layman's terms in which situation this would be helpful? E.g. if I have 1TB of data would I use this? If I have 1GB with a growth rate of 25GB/daily would I use this?

biot 10y ago
You may benefit from something like this if you do an analysis of your DB usage/growth patterns and determine that server/cluster scalability is going to be a non-trivial concern in the near future. Volume of data doesn't matter (to a certain point) if you seldom query that data. 1000TB of data that is neither queried nor updated doesn't require a database at all, for example. So, as with all things engineering, it depends. You know you'll benefit from it if you can demonstrate that you'll benefit from it an that the effort/risk/etc. is lower than some other solution you could choose. If you're unable to determine this, you should bring in an expert who can.
mdolan 10y ago
The rule of thumb I have always been told is that you can push MySql to ~200 GB total and between 99.9% and 99.99% availability (between 1 and 9 hours of downtime per year). Your milage might vary, but these are probably the right orders of magnitude. There is also an iops limit but that's harder to put a clear limit on because it's workload dependent. If you need more storage, availability, or iops, the current recommendations are shared relational stored or NoSql. Sharding relational databases means instead of turning on 1 MySql machine, you turn on 16 MySql machines and store 1/16th of the data on each machine. It only allow queries patterns that can be responded to by a single shard (so if you are only doing queries on a per user basis, your golden, otherwise, not so much) and require that your hottest shards (e.g. most active users) should only be a few orders of magnitude larger than your coldest (Justin Bieber probably needs his own machines at Twitter because he has so many users). There are ways to get around this, but they are tricky and will require a lot of developer time. With NoSql (riak, cassandra, dynamo, etc.), you have to give up many features that make developers lives easier in order to scale beyond a single machine. Each flavor has it's own set of tradeoffs, but the biggest problem is that you loose (1) committing multiple rows at the same time (atomic commit) and (2) ensuring queries don't overlap (isolation). Spanner offers an alternative where the developer (for the most part) doesn't have to know that her data lives on multiple different machines. It gives you 99.999% availability and theoretically unlimited scaling to any size, with an API that is still relatively developer friendly (compared with NoSql or Shared Sql). One cost here is latency / performance. You need to write to multiple computers across the world so write speed will be bounded at a minimum by the speed of light between those datacenters. Short answer, there are a lot of companies that could benefit from a system like this. It could give higher availability to a small Sql dataset or stronger guarantees to a current large NoSql database. There is even an argument to be made that the majority of problems companies face today are best solved with this type of system and relational or NoSql optimized workloads are the exception, not the rule.
- ec109685 10y ago
  1 to 9 hours of downtime a year for MySQL seems on the high side. With Aura, the failover times are much less and outside of the cloud, if there's a "noc", they can failover quicker than that.

by tabeth 10y ago

biot 10y ago

You may benefit from something like this if you do an analysis of your DB usage/growth patterns and determine that server/cluster scalability is going to be a non-trivial concern in the near future. Volume of data doesn't matter (to a certain point) if you seldom query that data. 1000TB of data that is neither queried nor updated doesn't require a database at all, for example. So, as with all things engineering, it depends. You know you'll benefit from it if you can demonstrate that you'll benefit from it an that the effort/risk/etc. is lower than some other solution you could choose. If you're unable to determine this, you should bring in an expert who can.

mdolan 10y ago

The rule of thumb I have always been told is that you can push MySql to ~200 GB total and between 99.9% and 99.99% availability (between 1 and 9 hours of downtime per year). Your milage might vary, but these are probably the right orders of magnitude. There is also an iops limit but that's harder to put a clear limit on because it's workload dependent. If you need more storage, availability, or iops, the current recommendations are shared relational stored or NoSql. Sharding relational databases means instead of turning on 1 MySql machine, you turn on 16 MySql machines and store 1/16th of the data on each machine. It only allow queries patterns that can be responded to by a single shard (so if you are only doing queries on a per user basis, your golden, otherwise, not so much) and require that your hottest shards (e.g. most active users) should only be a few orders of magnitude larger than your coldest (Justin Bieber probably needs his own machines at Twitter because he has so many users). There are ways to get around this, but they are tricky and will require a lot of developer time. With NoSql (riak, cassandra, dynamo, etc.), you have to give up many features that make developers lives easier in order to scale beyond a single machine. Each flavor has it's own set of tradeoffs, but the biggest problem is that you loose (1) committing multiple rows at the same time (atomic commit) and (2) ensuring queries don't overlap (isolation). Spanner offers an alternative where the developer (for the most part) doesn't have to know that her data lives on multiple different machines. It gives you 99.999% availability and theoretically unlimited scaling to any size, with an API that is still relatively developer friendly (compared with NoSql or Shared Sql). One cost here is latency / performance. You need to write to multiple computers across the world so write speed will be bounded at a minimum by the speed of light between those datacenters. Short answer, there are a lot of companies that could benefit from a system like this. It could give higher availability to a small Sql dataset or stronger guarantees to a current large NoSql database. There is even an argument to be made that the majority of problems companies face today are best solved with this type of system and relational or NoSql optimized workloads are the exception, not the rule.

ec109685 10y ago
1 to 9 hours of downtime a year for MySQL seems on the high side. With Aura, the failover times are much less and outside of the cloud, if there's a "noc", they can failover quicker than that.