4 ms·

If you dislike joins and love data inconsistencies, go for it.

by duaneb 10y ago

kod 10y ago
Joins are still possible in sharded relational models. I wouldn't want to roll my own on mysql, but there are plenty of data stores that use this model (vertica, citus, etc)
- deleted 10y ago
  [deleted]
- duaneb 10y ago
  > Joins are still possible in sharded relational models. For some definition of joins, this is true. For a SQL-oriented version, I don't think it's possible, let alone easy, to run arbitrary joins across sharded data. Can you write subqueries that cross shard boundaries? How about aggregate functions? I would be (very pleasantly) surprised if this were possible outside of directly calling the aggregation/join. > I wouldn't want to roll my own on mysql, but there are plenty of data stores that use this model (vertica, citus, etc) Yes, and none of them have succeed at scaling mysql without sacrificing heavily in functionality. I don't believe foreign keys are supported cross-shard anywhere. I'm also pretty sure the shard schemes heavily dictate which operations are efficient and even possible, and which are not, so you really need to design around scaling to begin with anyway. So—yes, you can scale MySQL, but avoid using any of the features that make MySQL worth using, except intrashard.
  - kod 10y ago
    None of the stores I mentioned use mysql. I'm not talking about scaling mysql, I'm talking about scaling relational databases, specifically the comment about joins. Joins arent a big deal, just join on (at least) the shard key, or for smaller hash joins ship the small side of the join. If you have multiple kinds of joins keep multiple copies with different shard keys. Yes, (monoidal) aggregations are possible with this, yes subqueries are possible with this.
Thaxll 10y ago
It's the method that largest companies are using, it's not as bad as it sounds.
- duaneb 10y ago
  > It's the method that largest companies are using ....in addition to other data stores. I don't know of any company exclusively using mysql; I hear that billing was migrated onto a sharded mysql cluster much more frequently due to the inherently transactional nature of billing operations and queries.

If you dislike joins and love data inconsistencies, go for it.

by duaneb 10y ago

If you dislike joins and love data inconsistencies, go for it.

kod 10y ago

Joins are still possible in sharded relational models. I wouldn't want to roll my own on mysql, but there are plenty of data stores that use this model (vertica, citus, etc)

deleted 10y ago
[deleted]
duaneb 10y ago
> Joins are still possible in sharded relational models. For some definition of joins, this is true. For a SQL-oriented version, I don't think it's possible, let alone easy, to run arbitrary joins across sharded data. Can you write subqueries that cross shard boundaries? How about aggregate functions? I would be (very pleasantly) surprised if this were possible outside of directly calling the aggregation/join. > I wouldn't want to roll my own on mysql, but there are plenty of data stores that use this model (vertica, citus, etc) Yes, and none of them have succeed at scaling mysql without sacrificing heavily in functionality. I don't believe foreign keys are supported cross-shard anywhere. I'm also pretty sure the shard schemes heavily dictate which operations are efficient and even possible, and which are not, so you really need to design around scaling to begin with anyway. So—yes, you can scale MySQL, but avoid using any of the features that make MySQL worth using, except intrashard.
- kod 10y ago
  None of the stores I mentioned use mysql. I'm not talking about scaling mysql, I'm talking about scaling relational databases, specifically the comment about joins. Joins arent a big deal, just join on (at least) the shard key, or for smaller hash joins ship the small side of the join. If you have multiple kinds of joins keep multiple copies with different shard keys. Yes, (monoidal) aggregations are possible with this, yes subqueries are possible with this.

Thaxll 10y ago

It's the method that largest companies are using, it's not as bad as it sounds.

duaneb 10y ago
> It's the method that largest companies are using ....in addition to other data stores. I don't know of any company exclusively using mysql; I hear that billing was migrated onto a sharded mysql cluster much more frequently due to the inherently transactional nature of billing operations and queries.