3 ms·

> The result is that we do very few joins in MySQL I realize depending on your data model some of this is unavoidable once you shard, but it sounds like they d

by fovc 6y ago

> The result is that we do very few joins in MySQL I realize depending on your data model some of this is unavoidable once you shard, but it sounds like they did this even before sharding. What's the reason to do that? Seems like you either end up with long transactions or doing MVCC in app code

tobyjsullivan 6y ago
The preceding sentence is > Joins inside MySQL were strongly discouraged in the codebase so that we would have more freedom in choosing which tables to move for scale out. The result is that we do very few joins in MySQL. And that comes at the end of the section describing their “vertical scaling” strategy and its motivations. As for your hypothesis about long transactions, this is not actually a problem in my experience (on larger scale CRUDy apps, specifically). In this scenario, you’re choosing to trade consistency for the ability to break out arbitrary tables to discrete databases. You’re going to lose that consistency after splitting data between multiple databases anyway (we’ll gloss over distributed transactions here) so why not be proactive about it? And on a “simple” platform like Quora, data is going to have a predominantly hierarchical relationship (users have posts which have comments which have likes, etc.) so internal consistency between content is rarely a significant concern.
toast0 6y ago
As a sibling has noted, joins across database hosts with SQL is somewhere between hard and impossible. So it made sense to discourage and remove them on that basis, given that the datasize was growing/has grown beyond what can be managed reasonably on a single host. Another reason to avoid joins is that they it can be difficult to consistently write joins that execute quickly. Often, a poorly performing join can be written as a sequence of queries which each perform well, and joined with reasonable performance on the client side. Usually, it's easier to scale the client (webserver) tier than the database tier, so moving work to the client makes sense for that reason too. Of course, this has to be considered in the context of the service. These days it's pretty easy to get a database server with 64 cores and a couple terrabytes of ram, and several terrabytes of fast SSD storage. You can do a lot with that, and you may never need to shard.
rb808 6y ago
> The result is that we do very few joins in MySQL What is the point of using a RDBMS if you never do queries?

by fovc 6y ago

tobyjsullivan 6y ago

The preceding sentence is > Joins inside MySQL were strongly discouraged in the codebase so that we would have more freedom in choosing which tables to move for scale out. The result is that we do very few joins in MySQL. And that comes at the end of the section describing their “vertical scaling” strategy and its motivations. As for your hypothesis about long transactions, this is not actually a problem in my experience (on larger scale CRUDy apps, specifically). In this scenario, you’re choosing to trade consistency for the ability to break out arbitrary tables to discrete databases. You’re going to lose that consistency after splitting data between multiple databases anyway (we’ll gloss over distributed transactions here) so why not be proactive about it? And on a “simple” platform like Quora, data is going to have a predominantly hierarchical relationship (users have posts which have comments which have likes, etc.) so internal consistency between content is rarely a significant concern.

toast0 6y ago

As a sibling has noted, joins across database hosts with SQL is somewhere between hard and impossible. So it made sense to discourage and remove them on that basis, given that the datasize was growing/has grown beyond what can be managed reasonably on a single host. Another reason to avoid joins is that they it can be difficult to consistently write joins that execute quickly. Often, a poorly performing join can be written as a sequence of queries which each perform well, and joined with reasonable performance on the client side. Usually, it's easier to scale the client (webserver) tier than the database tier, so moving work to the client makes sense for that reason too. Of course, this has to be considered in the context of the service. These days it's pretty easy to get a database server with 64 cores and a couple terrabytes of ram, and several terrabytes of fast SSD storage. You can do a lot with that, and you may never need to shard.

rb808 6y ago

> The result is that we do very few joins in MySQL What is the point of using a RDBMS if you never do queries?