6 ms·

I really wonder about the justification for using MySQL/Postgres sometimes. How is it not simply superior in almost every way to start with ScyllaDB which is fa

I really wonder about the justification for using MySQL/Postgres sometimes. How is it not simply superior in almost every way to start with ScyllaDB which is faster in every way and the only downside I can see is that you have a very limited way to query things. But so what, at least it's fast. I mean isn't that the nr.1 thing you want from a database, it's not about comfort.. it's about storing data. It is genuinely annoying though for example that you can't even alter a column type after the column was created in ScyllaDB.

christophilus 3y ago
> the only downside I can see is that you have a very limited way to query things That’s a pretty big one. If you know you need the scale, the trade off might be worth it. But I’m 99% of projects, Postgres is more than adequate and has great flexibility.
- Nathanba 3y ago
  yeah but I don't think that we can just trust that when people say this. They are invested into these single node databases for 10+ years probably. They are not going to tell you: Hey yeah, if you architect your app differently you could have x10 the performance.. but don't because it's a bit annoying when you query data. I mean is that being objective? Lots of people are not okay with the performance of mysql/postgres, that's the whole point of solutions like crdb/tidb/singlestore/vitess/yugabyte/citus and so many others. To tell someone that they should start with mysql or postgres is the equivalent of telling them that they'll have to migrate in the future and relearn a whole bunch of stuff about the operations, like backup etc. I don't really like the feeling of choosing a technology with the mindset that I'll have a lot of problems when there is a performance problem and at some point the pain will be so great that I'll have to switch to vitess.
  - richieartoul 3y ago
    It’s really not just about annoyance it’s about correctness and speed of iteration. Many apps that can be built trivially in a couple SQL queries cannot be implemented correctly in Scylla without reinventing half of tidb/vitess/etc. On top of that, it’s very hard to hire and pay the engineers who are capable of doing that in the first place. Finally, even if you manage to do it, your competitors will have shipped 100x more functionality than you in that time. That’s why people use these systems and why so many very smart people spend so much time trying to make them better
    - Nathanba 3y ago
      Well I do everything myself, I think what you said is mostly true only from the perspective of people who use orms and then everything works initially sure. But then are you really in control of your data and do you really know what's going on? I can only speak from a .net perspective now but the popular ORMs are all extremely slow, so with some research you end up using dapper. But then you end up basically writing SQL manually because that's just how it is. It wouldn't be any different for many languages, like Rust. Essentially I think unless you use java+hibernate or some other extremely common combination, you'll end up writing SQL yourself and then it's pretty painful anyway because you have a second textual DSL embedded into your other code with its own datatypes that you constantly have to map to the language's datatypes and vice versa. So iteration wise it doesnt even seem to be about mysql vs scylladb, it is about orm vs. no orm. And regarding correctness and iteration speed.. yes sure but it's easy to say that you are correct when you only run on a single node. The reason why those distributed solutions are less correct is because it's harder to be correct. For example in vitess you end up having to do a bunch of manual work with their sequences for every table. In scylladb afaik you just trust that the timeuuids are unique or you do the same and pregenerate bigints into a table. My point is: To say that mysql/postgres are safer because they are correct and you can iterate faster is only true if you go down a very narrow path with your orm and you stay on a single node. You are essentially postponing all the pain. At some point you'll stop using the orm and write your own, at some point you'll stop using the single node.
      - richieartoul 3y ago
        I think it’s the last paragraph I’m really disagreeing with. Many many application will do just fine with a single Postgres. If that is not enough even on a big box, they can scale out to a large master system like aurora. If that is not enough, it’s much easier to try and scale with something like Vitess or Cockroach that offers a lot more consistency and correctness out of the box. Scylla is great for specific workloads, but it’s a niche tool that makes sense for specific high volume workloads but I wouldn’t recommend it as a general purpose application development database. It will slow you down a lot and a lot of really simple features will be hard to implement. Also I’m not talking about ORMs. Writing SQL by hand in your application is 10-100x more productive for feature development than using something like Scyllas APIs directly IMO.
  - Aeolun 3y ago
    People saying the performance of Postgres is not adequate is my red flag for them not knowing what they’re doing. Of course, sometimes they do have actual performance issues, but 99 out of 100 times they just don’t know what an index is.
geenat 3y ago
It's a great point- I've been looking into this path myself: ScyllaDB + logic inside app. The main issue I've run into is deployment. It's just very rough compared to CockroachDB or Postgres, which are single binary or very close to it. If ScyllaDB became as easy to set up as CockroachDB (or close-ish), they'd have a lot more users, I think. You can make Postgres setup this easy: https://gist.github.com/gnat/cfe3754c3dc817c7fb8b2225ef4db628 https://gist.github.com/gnat/cfe3754c3dc817c7fb8b2225ef4db62... CockroachDB, also dead easy: https://www.cockroachlabs.com/docs/v22.2/start-a-local-cluster https://www.cockroachlabs.com/docs/v22.2/start-a-local-clust... ScyllaDB is an enormous mess of daemons and ports: https://university.scylladb.com/courses/scylla-operations/lessons/installation/topic/installation/ https://university.scylladb.com/courses/scylla-operations/le...
- Nathanba 3y ago
  yes the easy setup with crdb is really attractive but then you realize that they hide a few very important features behind the paid option. No incremental backups and no partitions (I think of any kind...). Those are some big features to lose, obviously I'm not complaining, just comparing. That was the main reason why I started developing my project in MySQL because at least MySQL has a lot of different free providers. YugabyteDB exists for Postgres too but initially I was more impressed with Vitess and TiDB because TiDB is used by some really huge chinese companies and if they don't have a problem then it must be okay. Unfortunately TiDB essentially doesn't want you to run a single node, ever, and their smallest recommended deployment is 5 nodes. That's a lot. Citus also exists for Postgres but their docs basically tell you that it's only recommended for analytics so I don't know what to think of it. And it sounds like all it's doing is a basic master-slave postgres setup with quite a few manual things you have to do to even benefit (manually altering tables to make them sharded/partitioned)
  - geenat 3y ago
    I totally hear you, been down a similar path. As for the "huge chinese" databases, not to sound like a conspiracy, but CockroachDB has outperformed them in my server load testing (stress test with very basic "SELECT FROM users" and "INSERT INTO users" on KC3000 + Ryzen 7950x)- so I'm beginning to seriously question all of the self-published benchmarks from mainland china. That, and when one considers they have a similarly rough deployment story compared to ScyllaDB, is it maybe worth just going directly to ScyllaDB in the first place? One other damming thing: I've also observed that all Spanner (Big Table) inspired databases have lower throughput compared to Postgres when under 5-ish servers: Aka you only begin to see throughput benefits after you have a sizeable cluster. ScyllaDB / Cassandra is a different architecture, though. Heck even Vitess, pretty rough deploy story as well (they really push k8s on you), but that's why PlanetScale exists, IMO.
doh 3y ago
The main difference is eventual vs strong consistency. Scylla scales great, but then the developer has to guard the consistencies. And that is quite challenging job, that usually trips even the most experienced developers. There is a middle ground, storage systems/databases that both scale and provide strong consistency. FoundationDB, Citus, etc. SQL databases are not only easier to query, but also enforce data consistency. While querying is their core job, it's hardly the only one.
tomnipotent 3y ago
> the only downside I can see is that you have a very limited way to query things Which is a huge downside. > But so what, at least it's fast So is an RDBMS for most read/write workloads. Few will see an overall benefit from Cassandra and the labor needed to replicate the conveniences offered by an RDBMS. > it's about storing data It's not about storing data, it's about using data to accomplish business objectives with the promises that entails. You opt for something like Cassandra/Scylla because it solves a technical challenge otherwise untenable or more expensive, but it's a poor choice if your problem is reasonably solved with what Postgres can offer.
stickfigure 3y ago
Looking back over my ~30 yr career I could pretty neatly divide my projects into two categories: * Low volume, complicated queries * High volume, simple queries In the rare case that I've seen high volume, complicated queries it generally involves a lot of pre-indexing, denormalization, and exotic data structures. Postgres really shines in the first case, which is probably the dominant case for "data important enough to pay people to work on".

christophilus 3y ago

> the only downside I can see is that you have a very limited way to query things That’s a pretty big one. If you know you need the scale, the trade off might be worth it. But I’m 99% of projects, Postgres is more than adequate and has great flexibility.

Nathanba 3y ago
yeah but I don't think that we can just trust that when people say this. They are invested into these single node databases for 10+ years probably. They are not going to tell you: Hey yeah, if you architect your app differently you could have x10 the performance.. but don't because it's a bit annoying when you query data. I mean is that being objective? Lots of people are not okay with the performance of mysql/postgres, that's the whole point of solutions like crdb/tidb/singlestore/vitess/yugabyte/citus and so many others. To tell someone that they should start with mysql or postgres is the equivalent of telling them that they'll have to migrate in the future and relearn a whole bunch of stuff about the operations, like backup etc. I don't really like the feeling of choosing a technology with the mindset that I'll have a lot of problems when there is a performance problem and at some point the pain will be so great that I'll have to switch to vitess.
- richieartoul 3y ago
  It’s really not just about annoyance it’s about correctness and speed of iteration. Many apps that can be built trivially in a couple SQL queries cannot be implemented correctly in Scylla without reinventing half of tidb/vitess/etc. On top of that, it’s very hard to hire and pay the engineers who are capable of doing that in the first place. Finally, even if you manage to do it, your competitors will have shipped 100x more functionality than you in that time. That’s why people use these systems and why so many very smart people spend so much time trying to make them better
  - Nathanba 3y ago
    Well I do everything myself, I think what you said is mostly true only from the perspective of people who use orms and then everything works initially sure. But then are you really in control of your data and do you really know what's going on? I can only speak from a .net perspective now but the popular ORMs are all extremely slow, so with some research you end up using dapper. But then you end up basically writing SQL manually because that's just how it is. It wouldn't be any different for many languages, like Rust. Essentially I think unless you use java+hibernate or some other extremely common combination, you'll end up writing SQL yourself and then it's pretty painful anyway because you have a second textual DSL embedded into your other code with its own datatypes that you constantly have to map to the language's datatypes and vice versa. So iteration wise it doesnt even seem to be about mysql vs scylladb, it is about orm vs. no orm. And regarding correctness and iteration speed.. yes sure but it's easy to say that you are correct when you only run on a single node. The reason why those distributed solutions are less correct is because it's harder to be correct. For example in vitess you end up having to do a bunch of manual work with their sequences for every table. In scylladb afaik you just trust that the timeuuids are unique or you do the same and pregenerate bigints into a table. My point is: To say that mysql/postgres are safer because they are correct and you can iterate faster is only true if you go down a very narrow path with your orm and you stay on a single node. You are essentially postponing all the pain. At some point you'll stop using the orm and write your own, at some point you'll stop using the single node.
    - richieartoul 3y ago
      I think it’s the last paragraph I’m really disagreeing with. Many many application will do just fine with a single Postgres. If that is not enough even on a big box, they can scale out to a large master system like aurora. If that is not enough, it’s much easier to try and scale with something like Vitess or Cockroach that offers a lot more consistency and correctness out of the box. Scylla is great for specific workloads, but it’s a niche tool that makes sense for specific high volume workloads but I wouldn’t recommend it as a general purpose application development database. It will slow you down a lot and a lot of really simple features will be hard to implement. Also I’m not talking about ORMs. Writing SQL by hand in your application is 10-100x more productive for feature development than using something like Scyllas APIs directly IMO.
- Aeolun 3y ago
  People saying the performance of Postgres is not adequate is my red flag for them not knowing what they’re doing. Of course, sometimes they do have actual performance issues, but 99 out of 100 times they just don’t know what an index is.

geenat 3y ago

It's a great point- I've been looking into this path myself: ScyllaDB + logic inside app. The main issue I've run into is deployment. It's just very rough compared to CockroachDB or Postgres, which are single binary or very close to it. If ScyllaDB became as easy to set up as CockroachDB (or close-ish), they'd have a lot more users, I think. You can make Postgres setup this easy: https://gist.github.com/gnat/cfe3754c3dc817c7fb8b2225ef4db628 https://gist.github.com/gnat/cfe3754c3dc817c7fb8b2225ef4db62... CockroachDB, also dead easy: https://www.cockroachlabs.com/docs/v22.2/start-a-local-cluster https://www.cockroachlabs.com/docs/v22.2/start-a-local-clust... ScyllaDB is an enormous mess of daemons and ports: https://university.scylladb.com/courses/scylla-operations/lessons/installation/topic/installation/ https://university.scylladb.com/courses/scylla-operations/le...

Nathanba 3y ago
yes the easy setup with crdb is really attractive but then you realize that they hide a few very important features behind the paid option. No incremental backups and no partitions (I think of any kind...). Those are some big features to lose, obviously I'm not complaining, just comparing. That was the main reason why I started developing my project in MySQL because at least MySQL has a lot of different free providers. YugabyteDB exists for Postgres too but initially I was more impressed with Vitess and TiDB because TiDB is used by some really huge chinese companies and if they don't have a problem then it must be okay. Unfortunately TiDB essentially doesn't want you to run a single node, ever, and their smallest recommended deployment is 5 nodes. That's a lot. Citus also exists for Postgres but their docs basically tell you that it's only recommended for analytics so I don't know what to think of it. And it sounds like all it's doing is a basic master-slave postgres setup with quite a few manual things you have to do to even benefit (manually altering tables to make them sharded/partitioned)
- geenat 3y ago
  I totally hear you, been down a similar path. As for the "huge chinese" databases, not to sound like a conspiracy, but CockroachDB has outperformed them in my server load testing (stress test with very basic "SELECT FROM users" and "INSERT INTO users" on KC3000 + Ryzen 7950x)- so I'm beginning to seriously question all of the self-published benchmarks from mainland china. That, and when one considers they have a similarly rough deployment story compared to ScyllaDB, is it maybe worth just going directly to ScyllaDB in the first place? One other damming thing: I've also observed that all Spanner (Big Table) inspired databases have lower throughput compared to Postgres when under 5-ish servers: Aka you only begin to see throughput benefits after you have a sizeable cluster. ScyllaDB / Cassandra is a different architecture, though. Heck even Vitess, pretty rough deploy story as well (they really push k8s on you), but that's why PlanetScale exists, IMO.

doh 3y ago

The main difference is eventual vs strong consistency. Scylla scales great, but then the developer has to guard the consistencies. And that is quite challenging job, that usually trips even the most experienced developers. There is a middle ground, storage systems/databases that both scale and provide strong consistency. FoundationDB, Citus, etc. SQL databases are not only easier to query, but also enforce data consistency. While querying is their core job, it's hardly the only one.

tomnipotent 3y ago

> the only downside I can see is that you have a very limited way to query things Which is a huge downside. > But so what, at least it's fast So is an RDBMS for most read/write workloads. Few will see an overall benefit from Cassandra and the labor needed to replicate the conveniences offered by an RDBMS. > it's about storing data It's not about storing data, it's about using data to accomplish business objectives with the promises that entails. You opt for something like Cassandra/Scylla because it solves a technical challenge otherwise untenable or more expensive, but it's a poor choice if your problem is reasonably solved with what Postgres can offer.

stickfigure 3y ago

Looking back over my ~30 yr career I could pretty neatly divide my projects into two categories: * Low volume, complicated queries * High volume, simple queries In the rare case that I've seen high volume, complicated queries it generally involves a lot of pre-indexing, denormalization, and exotic data structures. Postgres really shines in the first case, which is probably the dominant case for "data important enough to pay people to work on".