4 ms·

This question is similar to asking on a car forum when using a 40 foot lorry will be the default starter car for everyone. The answer is "probably never" becaus

by WJW 6y ago

This question is similar to asking on a car forum when using a 40 foot lorry will be the default starter car for everyone. The answer is "probably never" because while it does offer superior cargo transport scalability, the tradeoffs are not worth it for the vast majority of users. The question is posed like distributed databases have no disadvantages over non-distributed databases, but that is simply not the case. Clustering introduces all sort of problems, from network splits to slow(er) joins across nodes.

__s 6y ago
To back up how far one server can go, Stack Overflow used a single database for a long time https://nickcraver.com/blog/2016/02/17/stack-overflow-the-architecture-2016-edition https://nickcraver.com/blog/2016/02/17/stack-overflow-the-ar... (2013 post had much less redundancy, but even their 2016 architecture is pretty undistributed in terms of being able to recreate everything from the single source of truth database)
- bcrosby95 6y ago
  Yeah, it's been a while since I've been in a high traffic situation, but back in the late 00s we had a couple sites with 3-5 million daily unique users. Each site had a single master, a couple read replicas, and some queries were cached in memcached. The problem in MySQL we eventually ran into was the read replicas fell behind master during peak load (by 30-60 minutes depending upon the day). So we still had to hit the master for certain queries. I left before we took the next step to fix this issue.
kevincox 6y ago
I don't think it is quite the same. - Switching from a car to a van to a lorry is fairly low cost. You don't need to recreate your product (probably). - You don't need to run distributed databases in a cluster to start. But I think most importantly the decrease in dev speed and performance is an investment in future scalability. And I only imagine that this different will shrink over time to where for example a 1 node "distributed" DB isn't that different to work with than a 1 node "traditional" DB. And that small difference pays off because adding a second node doesn't have huge tradeoffs. I agree that right now it doesn't make sense. If I was starting a new product I would still fire up a PostgreSQL instance. However I think that the day will come where the difference is small enough and software support is complete enough that we will start with something distributed, much like people don't often start with SQLite today, even though it is quicker to get going (also not a great comparison because it has less features).
- yuribro 6y ago
  > But I think most importantly the decrease in dev speed and performance is an investment in future scalability. It makes sense only if you'll ever need this scalability. And you take a hit on other fronts too: Infra cost, Deployment complexity. And both deployment complexity and code complexity also increase QA cost, instability, product and company reputation. >> much like people don't often start with SQLite today Maybe they should start with SQLite by default
  - lrem 6y ago
    For personal projects, I start with SQLite by default. Never came close to its scalability limit too ;)
- WJW 6y ago
  Making investments in future scalability at the cost of dev speed and performance is exactly the wrong tradeoff given that 90% of startups fail. At the start, when you have very little income, you want to allow for as much speed and flexibility as possible so that you can get to product/market fit ASAP. By the time your company gets big enough that plain MySQL/Postgres can't handle the load any longer, you will have more than enough money to afford a few experts that can help you migrate.
- andreimatei1 6y ago
  > I agree that right now it doesn't make sense. This CRDB engineer respectfully disagrees. This thread takes it as a given that a non-distributed DB is better if you don't need to scale up (i.e. if you run a single "node"). Let me offer a couterpoint: it's easier to embed CRDB into some software you're distributing than it is to embed Postgres. This is to say, we do try to compete at every scale (well, perhaps not at SQLite scale). CRDB doesn't have all the SQL features of PG, but it does have its pluses: CRDB does online schema changes, it is generally simpler to run, comes with a UI, comes with more observability, can backup to the cloud, can be more easily embedded into tests. Online schema changes are a big deal; the other thing that I hope will help us win small-scale hearts and minds is the ever-improving observability story. I hope CRDB will develop world-class capabilities here. Other open-source databases traditionally have not had many capabilities out of the box.

by WJW 6y ago

__s 6y ago

To back up how far one server can go, Stack Overflow used a single database for a long time https://nickcraver.com/blog/2016/02/17/stack-overflow-the-architecture-2016-edition https://nickcraver.com/blog/2016/02/17/stack-overflow-the-ar... (2013 post had much less redundancy, but even their 2016 architecture is pretty undistributed in terms of being able to recreate everything from the single source of truth database)

bcrosby95 6y ago
Yeah, it's been a while since I've been in a high traffic situation, but back in the late 00s we had a couple sites with 3-5 million daily unique users. Each site had a single master, a couple read replicas, and some queries were cached in memcached. The problem in MySQL we eventually ran into was the read replicas fell behind master during peak load (by 30-60 minutes depending upon the day). So we still had to hit the master for certain queries. I left before we took the next step to fix this issue.

kevincox 6y ago

I don't think it is quite the same. - Switching from a car to a van to a lorry is fairly low cost. You don't need to recreate your product (probably). - You don't need to run distributed databases in a cluster to start. But I think most importantly the decrease in dev speed and performance is an investment in future scalability. And I only imagine that this different will shrink over time to where for example a 1 node "distributed" DB isn't that different to work with than a 1 node "traditional" DB. And that small difference pays off because adding a second node doesn't have huge tradeoffs. I agree that right now it doesn't make sense. If I was starting a new product I would still fire up a PostgreSQL instance. However I think that the day will come where the difference is small enough and software support is complete enough that we will start with something distributed, much like people don't often start with SQLite today, even though it is quicker to get going (also not a great comparison because it has less features).

yuribro 6y ago
> But I think most importantly the decrease in dev speed and performance is an investment in future scalability. It makes sense only if you'll ever need this scalability. And you take a hit on other fronts too: Infra cost, Deployment complexity. And both deployment complexity and code complexity also increase QA cost, instability, product and company reputation. >> much like people don't often start with SQLite today Maybe they should start with SQLite by default
- lrem 6y ago
  For personal projects, I start with SQLite by default. Never came close to its scalability limit too ;)
WJW 6y ago
Making investments in future scalability at the cost of dev speed and performance is exactly the wrong tradeoff given that 90% of startups fail. At the start, when you have very little income, you want to allow for as much speed and flexibility as possible so that you can get to product/market fit ASAP. By the time your company gets big enough that plain MySQL/Postgres can't handle the load any longer, you will have more than enough money to afford a few experts that can help you migrate.
andreimatei1 6y ago
> I agree that right now it doesn't make sense. This CRDB engineer respectfully disagrees. This thread takes it as a given that a non-distributed DB is better if you don't need to scale up (i.e. if you run a single "node"). Let me offer a couterpoint: it's easier to embed CRDB into some software you're distributing than it is to embed Postgres. This is to say, we do try to compete at every scale (well, perhaps not at SQLite scale). CRDB doesn't have all the SQL features of PG, but it does have its pluses: CRDB does online schema changes, it is generally simpler to run, comes with a UI, comes with more observability, can backup to the cloud, can be more easily embedded into tests. Online schema changes are a big deal; the other thing that I hope will help us win small-scale hearts and minds is the ever-improving observability story. I hope CRDB will develop world-class capabilities here. Other open-source databases traditionally have not had many capabilities out of the box.