3 ms·

I love these new distributed DBs. CockroachDB is one of them. Still I think a managed Postgres/MySQL is a better choice. My primary concern is how challenging w

I love these new distributed DBs. CockroachDB is one of them. Still I think a managed Postgres/MySQL is a better choice. My primary concern is how challenging will it be if you have to eventually move your data out to a RDBMS for cost or other reasons. Does anyone have any experience ? I am not talking enterprise scale but data about size of 50 - 100GB scale.

convolvatron 2y ago
why do you think that would be harder? assume for the moment that the reader here is going to run at the effective rate of a single node and not that we're going to try to parallelize that. Assuming we have transaction isolation, that reader is going to get a consistent snapshot. a distributed database is potentially more complicated to operate, and optimize, and because its new and potentially has more sharp edges maybe less reliable (?) but the extraction of moderate sized datasets doesn't really seem to be an obvious failing.
- std_reply 2y ago
  I would argue the opposite, distributed databases are much easier to operate at large scale. Truly online distributed DDL, at least in TIDB, strong consistency etc. People who bang on about Postgres replication have rarely setup replication in Postgres themselves and that too in the 100a of Pb scale. MySQL replication works well and can be scaled more easily (relative to Postgres) but has its own problems. eg., DDL is still a nightmare, lag is a real problem, usually masked by async replication. But then eventual consistency makes the application developers life more complicated.
sgarland 2y ago
Distributed DBs and traditional RDBMS serve different purposes. Most – by which I mean the overwhelming majority – companies do not need a distributed DB, they need better schema and queries. My fear is companies without in-house RDBMS expertise see these products as a way to continue to avoid getting that expertise.
debarshri 2y ago
I think it is mostly three folds. Availability, to be cloud agnostic and scale of few tbs to few hundred TBs. I have been seeing cockroach and yugabyte in an on prem setup where you don't get the benefits of a cloud provider but you can get the availability guarantees and fault tolerance.

convolvatron 2y ago

why do you think that would be harder? assume for the moment that the reader here is going to run at the effective rate of a single node and not that we're going to try to parallelize that. Assuming we have transaction isolation, that reader is going to get a consistent snapshot. a distributed database is potentially more complicated to operate, and optimize, and because its new and potentially has more sharp edges maybe less reliable (?) but the extraction of moderate sized datasets doesn't really seem to be an obvious failing.

std_reply 2y ago
I would argue the opposite, distributed databases are much easier to operate at large scale. Truly online distributed DDL, at least in TIDB, strong consistency etc. People who bang on about Postgres replication have rarely setup replication in Postgres themselves and that too in the 100a of Pb scale. MySQL replication works well and can be scaled more easily (relative to Postgres) but has its own problems. eg., DDL is still a nightmare, lag is a real problem, usually masked by async replication. But then eventual consistency makes the application developers life more complicated.

sgarland 2y ago

Distributed DBs and traditional RDBMS serve different purposes. Most – by which I mean the overwhelming majority – companies do not need a distributed DB, they need better schema and queries. My fear is companies without in-house RDBMS expertise see these products as a way to continue to avoid getting that expertise.

debarshri 2y ago

I think it is mostly three folds. Availability, to be cloud agnostic and scale of few tbs to few hundred TBs. I have been seeing cockroach and yugabyte in an on prem setup where you don't get the benefits of a cloud provider but you can get the availability guarantees and fault tolerance.