4 ms·

I brought the performance argument (among others) because the article does. When I read 100k I thought it was rps, not rpm. It would have been an achievement.

I brought the performance argument (among others) because the article does. When I read 100k I thought it was rps, not rpm. It would have been an achievement. The achievable performance of dbs is linked to their horizontal scalability, and SQL does not scale horizontally because of the relational model. Subsets of SQL can be made to scale horizontally, like in Cassandra, without locks, transactions, joins, aggregations, etc... or with a homemade cluster like the one described in the article. But it's not easy to shard a mysql or sqlite system. It's very hard to rebalance a cluster. It's very hard to make it work during partitions. Paxos or Raft are difficult. Most nosql dbs do that for you. MySQL doesn't. So the argument of a "battle tested db" is weak if a mysql system must use custom built cluster management (custom built != battle tested) Online games are a good use case where you need very high "50% write/50% read" throughput, but it's only one among others. Logging/timeseries is another use case, with 99% write. As for the sqlite cluster: will you implement all the sharding/rebalancing/partition tolerance/CP database too? If not, you're comparing apples and oranges again. MySQL has merits, but nosql dbs have different design goals: judging by the "battle tested" argument totally missed the point. BTW, even a modest (2dbs + 1arbiter) mongodb cluster would handle 1600rps easily, and you'd get automatic failover and replication for free, with sharding baked-in if you need it tomorrow.

lotyrin 11y ago
That's the thing though, I can do 100k transactions per second with an RDBMS, it's not an achievement. Whether a DBMS allows relations or not or uses SQL or not is an independent property of whether it has built in dynamic schema/graph/replication/failover/query distribution/sharding/rebalancing/distributed consistency solutions. And whether those solutions are built-in has little to do with whether solving them is possible. If it's just that we are disappointed that the traditional database systems (which happen to be relational and SQL because these are elements of that tradition) feel like they don't need to solve these problems, then I absolutely agree. The performance benefits of flattening your data model, avoiding indexing things you don't query against, sharding, using a distributed map reduce, etc. can all be had with SQL and an RDBMS, so speed is a poor argument for straying from the tradition, IMO. If it's really about wanting a new generation of comprehensive platforms for solving distributed data management (all those features above) that's a great argument, but somehow it's always about speed. Of course distributing asynchronous writes across a cluster of cheap cloud VMs starved for disk IO is faster than single-point synchronous writes on one of those VMs, but is the operations overhead of orchestrating and monitoring that cluster really cheaper than provisioning the hardware it'd take to do the same throughput with a simpler traditional system? Not as often as I'd be had to believe.

lotyrin 11y ago

That's the thing though, I can do 100k transactions per second with an RDBMS, it's not an achievement. Whether a DBMS allows relations or not or uses SQL or not is an independent property of whether it has built in dynamic schema/graph/replication/failover/query distribution/sharding/rebalancing/distributed consistency solutions. And whether those solutions are built-in has little to do with whether solving them is possible. If it's just that we are disappointed that the traditional database systems (which happen to be relational and SQL because these are elements of that tradition) feel like they don't need to solve these problems, then I absolutely agree. The performance benefits of flattening your data model, avoiding indexing things you don't query against, sharding, using a distributed map reduce, etc. can all be had with SQL and an RDBMS, so speed is a poor argument for straying from the tradition, IMO. If it's really about wanting a new generation of comprehensive platforms for solving distributed data management (all those features above) that's a great argument, but somehow it's always about speed. Of course distributing asynchronous writes across a cluster of cheap cloud VMs starved for disk IO is faster than single-point synchronous writes on one of those VMs, but is the operations overhead of orchestrating and monitoring that cluster really cheaper than provisioning the hardware it'd take to do the same throughput with a simpler traditional system? Not as often as I'd be had to believe.