5 ms·

How high is your write load? We considered using it recently but used ndbcluster instead since XtraDB Cluster doesn't actually shard writes (iirc it only return

by jdjb 13y ago

How high is your write load? We considered using it recently but used ndbcluster instead since XtraDB Cluster doesn't actually shard writes (iirc it only returns a write as "complete" after enough nodes report the write as being finished so it's actually slower than writing to a single master). Is there a way to use it to linearly partition writes?

druiid 13y ago
Essentially, no. This is perhaps the one drawback with current Galera implementation. We run about 32 slave threads per node though and it has never been a real problem for us. It WILL be a problem for you though if you have 32 of those filled with write requests that take 20-30 seconds to complete. You'll end up with waiting sessions. I'd say that's not really an issue for Galera to directly resolve though. If this is a highly transactional web application you really don't want writes taking more than a few seconds in the majority of cases. It's an entirely different scenario for a data-warehouse type situation obviously, and having 30+ second writes with enough threads won't be an issue then either. Basically though, this is one area where I'd like to see Galera change in one way and it's kind of the 'one' valid criticism to have for the platform. If your workload falls within the good use-cases for it though, I'd recommend it well before NDB/MySQL Cluster due to if nothing else, the sheer complexity of scaling Cluster.
- jdjb 13y ago
  Thanks for the reply. We have a high-write application and while individual writes aren't really that slow we have A LOT of them (we receive our writes off the network from a 20+ hadoop-machine cluster so they can send quite a bit of junk our way). What's wrong with the complexity of scaling Cluster? From what I've experienced it's been pretty easy to add machines, take individual data nodes offline while cluster is running, etc..
jmngomes 13y ago
Did you consider Gizzard for sharding writes? Can you share why not?
- druiid 13y ago
  Well while this question doesn't really apply to me.. I can at least say one reason not to use Gizzard is that it's deprecated now.
  - jmngomes 13y ago
    Didn't know it had been deprecated, it seemed like a great tool for scaling databases. Anyone know why?
- jdjb 13y ago
  We've used master/master shards in the past and they've been very difficult to manage. I'm not fully aware of Gizzard's capabilities so I guess the answer is no, because we weren't aware of it. We also needed something that would speed up writes much more than just 2x (and the way ndbcluster does writes was the key).

by jdjb 13y ago

druiid 13y ago

Essentially, no. This is perhaps the one drawback with current Galera implementation. We run about 32 slave threads per node though and it has never been a real problem for us. It WILL be a problem for you though if you have 32 of those filled with write requests that take 20-30 seconds to complete. You'll end up with waiting sessions. I'd say that's not really an issue for Galera to directly resolve though. If this is a highly transactional web application you really don't want writes taking more than a few seconds in the majority of cases. It's an entirely different scenario for a data-warehouse type situation obviously, and having 30+ second writes with enough threads won't be an issue then either. Basically though, this is one area where I'd like to see Galera change in one way and it's kind of the 'one' valid criticism to have for the platform. If your workload falls within the good use-cases for it though, I'd recommend it well before NDB/MySQL Cluster due to if nothing else, the sheer complexity of scaling Cluster.

jdjb 13y ago
Thanks for the reply. We have a high-write application and while individual writes aren't really that slow we have A LOT of them (we receive our writes off the network from a 20+ hadoop-machine cluster so they can send quite a bit of junk our way). What's wrong with the complexity of scaling Cluster? From what I've experienced it's been pretty easy to add machines, take individual data nodes offline while cluster is running, etc..

jmngomes 13y ago

Did you consider Gizzard for sharding writes? Can you share why not?

druiid 13y ago
Well while this question doesn't really apply to me.. I can at least say one reason not to use Gizzard is that it's deprecated now.
- jmngomes 13y ago
  Didn't know it had been deprecated, it seemed like a great tool for scaling databases. Anyone know why?
jdjb 13y ago
We've used master/master shards in the past and they've been very difficult to manage. I'm not fully aware of Gizzard's capabilities so I guess the answer is no, because we weren't aware of it. We also needed something that would speed up writes much more than just 2x (and the way ndbcluster does writes was the key).