5 ms·

YouTube built Vitess, which lets you scale out MySQL writers. If I can scale out both my writers and my readers then what could be "vastly more scalable" than t

by vp8989 5y ago

YouTube built Vitess, which lets you scale out MySQL writers. If I can scale out both my writers and my readers then what could be "vastly more scalable" than that?

WJW 5y ago
As far as I can tell, Vitess still has trouble with cross-server transactions and distributed locking. Presumably something "vastly more scalable" could work around this. The Spanner team noticeably put out https://cloud.google.com/blog/products/databases/inside-cloud-spanner-and-the-cap-theorem https://cloud.google.com/blog/products/databases/inside-clou... a few years back, which tl;drs that they managed to get a reliable enough network that partitions don't really happen anymore and so you can get both C _and_ A from the CAP theorem. I don't think that Vitess is quite at that level, especially since you can self-host it and thus the devs don't have such network-level uptime guarantees.
- vp8989 5y ago
  Are cross-server transactions scalable? Would anyone architect a system that did them en masse unless they absolutely 100% had to? I am just speculating here but I doubt that YouTube migrated to Spanner so that they could do cross-server transactions. I would think consolidating the Ops burden into the GCP org and also serving as a trophy "customer" were higher on the list of reasons. But again, just speculating...
  - derekperkins 5y ago
    Yeah, Google policy was to migrate everything to Spanner, not a technical decision. > Are cross-server transactions scalable? Would anyone architect a system that did them en masse unless they absolutely 100% had to? As a long time Vitess user and contributor/maintainer, this perfectly describes the issue. Minimal effort has been put into solving cross shard / 2PC transactions because a well designed system rarely needs it.
- ksec 5y ago
  I guess people from PlanetScale will know. Not sure if they are reading the thread.
ansible 5y ago
Spanner paper: https://static.googleusercontent.com/media/research.google.com/en//archive/spanner-osdi2012.pdf https://static.googleusercontent.com/media/research.google.c... It is designed from the ground-up to be globally distributed. Some other databases like FoundationDB (now owned and killed? by Apple) and CockroachDB were designed to scale across multiple datacenters. Correction: FoundationDB is alive as open-source again, yay!
- svieira 5y ago
  FoundationDB was re-released as open source a couple of years ago. https://www.foundationdb.org/ https://www.foundationdb.org/
- imoverclocked 5y ago
  AFAIK, FoundationDB is alive and well. What makes you think it's killed?
  - capableweb 5y ago
    It was acquired by a huge company that sometimes kills products. Shortly after the acquisition, the downloads were removed from the website. It did go open source "recently" though which made it longer lasting, but I'd understand if someone would think it was dead by now. - https://news.ycombinator.com/item?id=9259986 https://news.ycombinator.com/item?id=9259986 - Apple Acquires FoundationDB - March 24, 2015 - Comparison between March 25 and March 14, 2015 - https://web.archive.org/web/20150314231702/https://foundationdb.com/ https://web.archive.org/web/20150314231702/https://foundatio... (14th March) > https://web.archive.org/web/20150325003252/https://foundationdb.com/ https://web.archive.org/web/20150325003252/https://foundatio... (25th March)

by vp8989 5y ago

YouTube built Vitess, which lets you scale out MySQL writers. If I can scale out both my writers and my readers then what could be "vastly more scalable" than that?

WJW 5y ago

As far as I can tell, Vitess still has trouble with cross-server transactions and distributed locking. Presumably something "vastly more scalable" could work around this. The Spanner team noticeably put out https://cloud.google.com/blog/products/databases/inside-cloud-spanner-and-the-cap-theorem https://cloud.google.com/blog/products/databases/inside-clou... a few years back, which tl;drs that they managed to get a reliable enough network that partitions don't really happen anymore and so you can get both C _and_ A from the CAP theorem. I don't think that Vitess is quite at that level, especially since you can self-host it and thus the devs don't have such network-level uptime guarantees.

vp8989 5y ago
Are cross-server transactions scalable? Would anyone architect a system that did them en masse unless they absolutely 100% had to? I am just speculating here but I doubt that YouTube migrated to Spanner so that they could do cross-server transactions. I would think consolidating the Ops burden into the GCP org and also serving as a trophy "customer" were higher on the list of reasons. But again, just speculating...
- derekperkins 5y ago
  Yeah, Google policy was to migrate everything to Spanner, not a technical decision. > Are cross-server transactions scalable? Would anyone architect a system that did them en masse unless they absolutely 100% had to? As a long time Vitess user and contributor/maintainer, this perfectly describes the issue. Minimal effort has been put into solving cross shard / 2PC transactions because a well designed system rarely needs it.
ksec 5y ago
I guess people from PlanetScale will know. Not sure if they are reading the thread.

ansible 5y ago

Spanner paper: https://static.googleusercontent.com/media/research.google.com/en//archive/spanner-osdi2012.pdf https://static.googleusercontent.com/media/research.google.c... It is designed from the ground-up to be globally distributed. Some other databases like FoundationDB (now owned and killed? by Apple) and CockroachDB were designed to scale across multiple datacenters. Correction: FoundationDB is alive as open-source again, yay!

svieira 5y ago
FoundationDB was re-released as open source a couple of years ago. https://www.foundationdb.org/ https://www.foundationdb.org/
imoverclocked 5y ago
AFAIK, FoundationDB is alive and well. What makes you think it's killed?
- capableweb 5y ago
  It was acquired by a huge company that sometimes kills products. Shortly after the acquisition, the downloads were removed from the website. It did go open source "recently" though which made it longer lasting, but I'd understand if someone would think it was dead by now. - https://news.ycombinator.com/item?id=9259986 https://news.ycombinator.com/item?id=9259986 - Apple Acquires FoundationDB - March 24, 2015 - Comparison between March 25 and March 14, 2015 - https://web.archive.org/web/20150314231702/https://foundationdb.com/ https://web.archive.org/web/20150314231702/https://foundatio... (14th March) > https://web.archive.org/web/20150325003252/https://foundationdb.com/ https://web.archive.org/web/20150325003252/https://foundatio... (25th March)