8 ms·

Can someone please explain how Vitess works, in plain English? How does it magically make MySQL scale? And then what does PlanetScale add on top of Vitess host

Can someone please explain how Vitess works, in plain English? How does it magically make MySQL scale? And then what does PlanetScale add on top of Vitess hosted anywhere else? Sorry, the linked blog post is both very abstract and assumes a high level of preexisting knowledge about database scaling.

moshmosh 5y ago
IME the answer to "how did they make [hard to scale thing] easily scalable?" is usually that they introduce limitations in how you can use [hard to scale thing] so you can't use it in ways that are hard to scale, then automate scaling it in well-known ways for use cases that are so-limited. Vitesse's site mentions that it relies on horizontal sharding, so right off the bat, my guess is that you can't use it in ways that are sharding-unfriendly, or if you can then you'll be met with restrictions on much of the "magic" of it if you do. Rarely is it the case that someone's actually discovered e.g. novel math or something to make the hard part easier. Better tools (to do well-understood things more easily for this use case) and restrictions (so you don't use it in ways the tools can't handle) are the usual way.
- lizztheblizz 5y ago
  The "ease" we used to refer to in Vitess primarily relates to its interaction with the application side, where it basically presents itself as "one big MySQL datastore". It uses a standard MySQL connector, and in general, once you have the infrastructure up and running with a compatible schema design, there's not too much to worry about from a coding standpoint. Sharding happens transparently to the application code, which generally translates to fewer code changes required. Admittedly, that view left out the considerable challenge of actually deploying and running the infrastructure, designing and optimizing that schema, along with all the joys of managing large cluster environments. That's what PlanetScale, the product, aims to solve. Dealing with clustering infrastructure IS a hurdle for most teams to overcome, and though Vitess' feature set and compatibility has expanded greatly to accommodate some of the most demanding use cases on the web today, a lot of its functionality can still be out of reach for a developer just trying to merge some code and a schema change. Abstracting as much of that complexity away from the end user is the goal, as well as making their lives easier with a ton of the functionality we've always wanted to see built with Vitess. I can confirm that that is not an "easy" job on our end. :)
  - Aeolun 5y ago
    > once you have the infrastructure up and running with a compatible schema design, there's not too much to worry about from a coding standpoint Isn’t that the same for a normal sharded MySQL database though?
    - lizztheblizz 5y ago
      Depends. Have any examples of "normal" sharded MySQL databases? EDIT: To clarify, sharding is not a standard feature included in Community Edition MySQL. Over the years, there have been various Oracle-initiated attempts at providing it as an enterprise scaling strategy through MySQL (NDB) Cluster, MySQL Fabric, etc., but these have either ended up having limited applicability outside very specific use cases and are not widely in use. Most large MySQL users (e.g. Facebook or YouTube) ended up rolling their own frameworks, like Vitess, which has since been open sourced and adapted to more diverse environments. Until that became more accessible, though, the rest of the world mostly made do with wobbly multi-master setups relying on circular replication, behind some kind of proxy, or had to implement the sharding logic itself into their application code.
      - Aeolun 5y ago
        Damn, I never knew this. I think the first time I ever needed sharding it was just available in whatever version of MySQL we were using at the time (through some plugin, presumably). I never needed it again and ever since just assumed it was the default. Thanks for the correction!
paxys 5y ago
Vitess is an additional layer on top of MySQL which all queries pass though. Among other things, it implements its own query parser which can then do stuff like split a query across shards and join results etc. I wouldn't say there's too much "magic" in there, but it does a lot of known difficult things (schema management, sharding/resharding, connection pooling, query optimization, DB administration, monitoring, backup/failover) which are generally painful and expensive to do yourself.
- Aeolun 5y ago
  I do think it’s funny that whenever these ‘magic’ products appear. It always turns out that they’re just packaging the accepted best method in a way that’s easy to consume.
  - hodgesrm 5y ago
    There are not many new things under the sun in the DBMS field. Ease of use is a killer feature.
- unknown_error 5y ago
  Hmm, so in a way, it's kinda like a CDN for a database?
motives 5y ago
As I understand it, Vitess is basically a really powerful sharding system, which goes a step further than typical sharding solutions by basically making the shards one or more unique databases. In the case of someone like slack, because your tenant (e.g your company slack), is completely isolated from other tenants, you can treat that basically as its own database, and have a master for just that DB, allowing much better scaling. The big limitation on Vitess is cross-shard transactions, and the fact you have to make sure your schema has a clear cut sharding key (like your tenant ID) that works nicely with your application needs. The alternative for scaling transactional SQL DBs in a multi-master fashion are the "NewSQL" DBs like yugabyte and cockroachdb which are basically document DBs with a partially implemented postgres frontend, so don't have the full feature set of your SQL engine like Vitesse does but don't require so much attention to sharding. These are oversimplifications of the actual mechanisms, but give a basic overview of the tradeoffs involved, please feel free to correct me on any inaccuracies as I'm not an expert in DBs.
- kmavm 5y ago
  I was Chief Architect at Slack from 2016 to 2020, and was privileged to work with the engineers who were doing the work of migrating to Vitess in that timeframe. The assumption that tenants are perfectly isolated is actually the original sin of early Slack infrastructure that we adopted Vitess to migrate away from. From some earlier features in the Enterprise product (which joins lots of "little Slacks" into a corporate-wide entity) to more post-modern features like Slack Connect (https://slack.com/help/articles/1500001422062-Start-a-direct-message-with-someone-outside-your-company https://slack.com/help/articles/1500001422062-Start-a-direct...) or Network Shared Channels (https://slack.com/blog/news/shared-channels-growth-innovation https://slack.com/blog/news/shared-channels-growth-innovatio...), the idea that each tenant is fully isolated was increasingly false. Vitess is a meta-layer on top of MySQL shards that asks, per table, which key to shard on. It then uses that information to maintain some distributed indexes of its own, and to plan the occasional scatter/gather query appropriately. In practice, simply migrating code from our application-sharded, per-tenant old way into the differently-sharded Vitess storage system was not a simple matter of pointing to a new database; we had to change data access patterns to avoid large fan-out reads and writes. The team did a great write-up about it here: https://slack.engineering/scaling-datastores-at-slack-with-vitess/ https://slack.engineering/scaling-datastores-at-slack-with-v...
manigandham 5y ago
Ultimately all databases scale the same way, by splitting up data into shards/partitions/segments and spreading them out over several servers, along with replication for durability. The partitioning is done by a primary/sorting/distribution key on the data for each table. Implementations vary but there are the 2 major architectures: systems like Vitess/Proxy SQL/Citus/Timescale that act as a proxy layer on top of existing RDBMS running on multiple servers to make them look like a single database, and entirely custom projects like CockroachDB/TiDB/Yugabyte/Cloud Spanner which have their own native processing and data layers. OLAP relational data warehouses like Vertica/Greenplum/MemSQL/Redshift/Bigquery are also natively distributed but focus on large-scale analytics with features like column-oriented storage and vectorized processing.

moshmosh 5y ago

IME the answer to "how did they make [hard to scale thing] easily scalable?" is usually that they introduce limitations in how you can use [hard to scale thing] so you can't use it in ways that are hard to scale, then automate scaling it in well-known ways for use cases that are so-limited. Vitesse's site mentions that it relies on horizontal sharding, so right off the bat, my guess is that you can't use it in ways that are sharding-unfriendly, or if you can then you'll be met with restrictions on much of the "magic" of it if you do. Rarely is it the case that someone's actually discovered e.g. novel math or something to make the hard part easier. Better tools (to do well-understood things more easily for this use case) and restrictions (so you don't use it in ways the tools can't handle) are the usual way.

lizztheblizz 5y ago
The "ease" we used to refer to in Vitess primarily relates to its interaction with the application side, where it basically presents itself as "one big MySQL datastore". It uses a standard MySQL connector, and in general, once you have the infrastructure up and running with a compatible schema design, there's not too much to worry about from a coding standpoint. Sharding happens transparently to the application code, which generally translates to fewer code changes required. Admittedly, that view left out the considerable challenge of actually deploying and running the infrastructure, designing and optimizing that schema, along with all the joys of managing large cluster environments. That's what PlanetScale, the product, aims to solve. Dealing with clustering infrastructure IS a hurdle for most teams to overcome, and though Vitess' feature set and compatibility has expanded greatly to accommodate some of the most demanding use cases on the web today, a lot of its functionality can still be out of reach for a developer just trying to merge some code and a schema change. Abstracting as much of that complexity away from the end user is the goal, as well as making their lives easier with a ton of the functionality we've always wanted to see built with Vitess. I can confirm that that is not an "easy" job on our end. :)
- Aeolun 5y ago
  > once you have the infrastructure up and running with a compatible schema design, there's not too much to worry about from a coding standpoint Isn’t that the same for a normal sharded MySQL database though?
  - lizztheblizz 5y ago
    Depends. Have any examples of "normal" sharded MySQL databases? EDIT: To clarify, sharding is not a standard feature included in Community Edition MySQL. Over the years, there have been various Oracle-initiated attempts at providing it as an enterprise scaling strategy through MySQL (NDB) Cluster, MySQL Fabric, etc., but these have either ended up having limited applicability outside very specific use cases and are not widely in use. Most large MySQL users (e.g. Facebook or YouTube) ended up rolling their own frameworks, like Vitess, which has since been open sourced and adapted to more diverse environments. Until that became more accessible, though, the rest of the world mostly made do with wobbly multi-master setups relying on circular replication, behind some kind of proxy, or had to implement the sharding logic itself into their application code.
    - Aeolun 5y ago
      Damn, I never knew this. I think the first time I ever needed sharding it was just available in whatever version of MySQL we were using at the time (through some plugin, presumably). I never needed it again and ever since just assumed it was the default. Thanks for the correction!

paxys 5y ago

Vitess is an additional layer on top of MySQL which all queries pass though. Among other things, it implements its own query parser which can then do stuff like split a query across shards and join results etc. I wouldn't say there's too much "magic" in there, but it does a lot of known difficult things (schema management, sharding/resharding, connection pooling, query optimization, DB administration, monitoring, backup/failover) which are generally painful and expensive to do yourself.

Aeolun 5y ago
I do think it’s funny that whenever these ‘magic’ products appear. It always turns out that they’re just packaging the accepted best method in a way that’s easy to consume.
- hodgesrm 5y ago
  There are not many new things under the sun in the DBMS field. Ease of use is a killer feature.
unknown_error 5y ago
Hmm, so in a way, it's kinda like a CDN for a database?

motives 5y ago

As I understand it, Vitess is basically a really powerful sharding system, which goes a step further than typical sharding solutions by basically making the shards one or more unique databases. In the case of someone like slack, because your tenant (e.g your company slack), is completely isolated from other tenants, you can treat that basically as its own database, and have a master for just that DB, allowing much better scaling. The big limitation on Vitess is cross-shard transactions, and the fact you have to make sure your schema has a clear cut sharding key (like your tenant ID) that works nicely with your application needs. The alternative for scaling transactional SQL DBs in a multi-master fashion are the "NewSQL" DBs like yugabyte and cockroachdb which are basically document DBs with a partially implemented postgres frontend, so don't have the full feature set of your SQL engine like Vitesse does but don't require so much attention to sharding. These are oversimplifications of the actual mechanisms, but give a basic overview of the tradeoffs involved, please feel free to correct me on any inaccuracies as I'm not an expert in DBs.

kmavm 5y ago
I was Chief Architect at Slack from 2016 to 2020, and was privileged to work with the engineers who were doing the work of migrating to Vitess in that timeframe. The assumption that tenants are perfectly isolated is actually the original sin of early Slack infrastructure that we adopted Vitess to migrate away from. From some earlier features in the Enterprise product (which joins lots of "little Slacks" into a corporate-wide entity) to more post-modern features like Slack Connect (https://slack.com/help/articles/1500001422062-Start-a-direct-message-with-someone-outside-your-company https://slack.com/help/articles/1500001422062-Start-a-direct...) or Network Shared Channels (https://slack.com/blog/news/shared-channels-growth-innovation https://slack.com/blog/news/shared-channels-growth-innovatio...), the idea that each tenant is fully isolated was increasingly false. Vitess is a meta-layer on top of MySQL shards that asks, per table, which key to shard on. It then uses that information to maintain some distributed indexes of its own, and to plan the occasional scatter/gather query appropriately. In practice, simply migrating code from our application-sharded, per-tenant old way into the differently-sharded Vitess storage system was not a simple matter of pointing to a new database; we had to change data access patterns to avoid large fan-out reads and writes. The team did a great write-up about it here: https://slack.engineering/scaling-datastores-at-slack-with-vitess/ https://slack.engineering/scaling-datastores-at-slack-with-v...

manigandham 5y ago

Ultimately all databases scale the same way, by splitting up data into shards/partitions/segments and spreading them out over several servers, along with replication for durability. The partitioning is done by a primary/sorting/distribution key on the data for each table. Implementations vary but there are the 2 major architectures: systems like Vitess/Proxy SQL/Citus/Timescale that act as a proxy layer on top of existing RDBMS running on multiple servers to make them look like a single database, and entirely custom projects like CockroachDB/TiDB/Yugabyte/Cloud Spanner which have their own native processing and data layers. OLAP relational data warehouses like Vertica/Greenplum/MemSQL/Redshift/Bigquery are also natively distributed but focus on large-scale analytics with features like column-oriented storage and vectorized processing.