3 ms·
Horizontally scalable data storage is generally available these days (cockroach, tidb, vitess, etc.) Rearchitecting from unscalable to scalable data storage is
by dub 4y ago
Horizontally scalable data storage is generally available these days (cockroach, tidb, vitess, etc.)
Rearchitecting from unscalable to scalable data storage is notoriously difficult and expensive. Even the most famously competent companies and teams have struggled with that transition and invested millions and millions of dollars on it. Building on unscalable data storage when scalable data storage is readily available feels like planning for failure and making a huge bet that your product or system will never have widespread use.
- dominotw 4y agoI am suprised how niche these newsql databases are. Is it because they are expensive? why wouldn't everyone use those.
- cersa8 4y agoI've experimented with migrating from PostgreSQL to CockroachDB and YugabyteDB. It makes sense that a newsql DB is slower but compared to to a local PostgreSQL cluster it was unacceptably slow. You lose features and are now dependent on a heavily VC funded startup that may not make it.
- dub 4y agoScalable relational storage is relatively new to industry consciousness (outside of Google, where Spanner is used for nearly everything). Spanner and Vitess existed at Google in the early 2010s. It was mid-to-late 2010s before CockroachDB, TiDB, and Vitess were available as open source. Battle hardened database administrators who've been working with traditional unscalable databases for decades often aren't up-to-date with what's possible today unless they frequently talk shop with peers at other companies or go to conferences. Personally I hope old databases like MySQL and PostgreSQL die out within the next decade. They served us well, but they're 25+ years old and the weight of legacy code plus the need for backwards compatibility is a huge drag on their ability to evolve.