4 ms·

Fair questions, let me (Dor) answer: 1. Scylla works within different AZs too We are topology aware and can have as good and even better HA than Dynamo

Fair questions, let me (Dor) answer: 1. Scylla works within different AZs too We are topology aware and can have as good and even better HA than DynamoDB. Our design is based on C* 2. We were surprised with getting small utilization too. As I wrote in the article, I think Dynamo had a hard time reaching to 1TB that quick. The population started fine and deeper in the the population it failed. Only a decrease in the rate solved it. 3. 1.1kb The is the default table setup by YCSB. It isn't optimal for Dynamo but that's life.. think about if you store a blob of 1kb - together with the key it will be more than 1kb. We were upfront about it and you can make your own calculation. Scylla will still be better. The only use case where Dynamo is better in price is when you store lots of data but with a tiny IOPS reservation. We'll address this case over time too. Cheers!

planckscnst 8y ago
> Scylla works within different AZs too Did your test use this feature of Scylla? As I understand the documentation, it doesn't appear so (maybe that needs to be fixed). If that's the case, do you think this is an example of being diligent in your fairness? > Only a decrease in the rate solved it. I don't think that is the only way to solve it. Something is probably wrong here, and I would expect that diligence to fairness would dictate trying to understand and correct that. This benchmark would be more compelling if problems like this weren't just ignored, but actually solved. Any team that cares about the cost of their database would do that. > It isn't optimal for Dynamo but that's life. Yes, life is unfair and this can happen. This test specifically claimed to be fair, though. Again, for this benchmark to be meaningful and compelling, this kind of thing should be addressed. > The only use case where Dynamo is better in price is when you store lots of data but with a tiny IOPS reservation This is a surprising statement. If this is the case, I highly recommend addressing the issues so that you are using DynamoDB like a typical engineering team would. A blanket statement like this with questionable benchmarks as evidence is just not compelling. It would be much more helpful to see where DynamoDB really can't stand a chance competing against ScyllaDB even when it's fully optimiized. Are you ignoring maintenance costs? Do nodes get added and removed without paying an engineer? How do you keep the nodes up-to-date as far as security updates or adding new features? How much overhead cost is engineering on-call to respond when the cluster has issues (e.g. nearing capacity limits, nodes failing, loss of AZ availability)? I'm also curious if you are only considering use cases that you typically already see on ScyllaDB and Cassandra. How about the use case where I have a load that increases without warning by 10x or where I need regular backups or that requires 5000 nodes?
- thekozmo 8y ago
  About multiple AZ, I think we haven't used it in this case but it shouldn't change a bit, most of our AWS customers have AZ-aware deployments and it works great. Since I see you're with AWS, you're welcome to help us debug this error 500 on population. I guarantee we'll update the blog if we'll see it works fine I didn't get your surprise, I actually am 100% open about digging deep and come up with a uncommon use case that Dynamo can finally be a good alternative. Believe me, I can come with examples that will put Scylla in a much better light. We do offer a fully managed service on AWS with 4x-6x cost saving vs Dynamo where all the backup and auto scale is done by the Scylla team.
  - otterley 8y ago
    > About multiple AZ, I think we haven't used it in this case but it shouldn't change a bit Are you suggesting that the metrics would not change? I assure you that is not the case: inter-AZ latency is 1-2 orders of magnitude higher than intra-AZ latency. You'll need to re-run your benchmarks in a multi-AZ cluster in order to provide an apples-to-apples comparison.
    - thekozmo 8y ago
      I don't think this is the case - check this link: https://knowledgebase.progress.com/articles/Article/expected-latency-between-aws-availability-zones https://knowledgebase.progress.com/articles/Article/expected... The intra AZ latency according to AWS is sub millisecond! It's inline with what our customers see. Since all of the numbers in the blog are above a millisecond, the result does not change.
      - otterley 8y ago
        Intra-AZ latency is indeed sub-millisecond. Inter-AZ latency is in the milliseconds range. (The intra prefix means within, inter means between.) And those latencies accumulate quickly when benchmarking. Why don't you try it yourself and see the difference? Relying on some third party website is no substitute for testing.
deleted 8y ago
[deleted]

planckscnst 8y ago

> Scylla works within different AZs too Did your test use this feature of Scylla? As I understand the documentation, it doesn't appear so (maybe that needs to be fixed). If that's the case, do you think this is an example of being diligent in your fairness? > Only a decrease in the rate solved it. I don't think that is the only way to solve it. Something is probably wrong here, and I would expect that diligence to fairness would dictate trying to understand and correct that. This benchmark would be more compelling if problems like this weren't just ignored, but actually solved. Any team that cares about the cost of their database would do that. > It isn't optimal for Dynamo but that's life. Yes, life is unfair and this can happen. This test specifically claimed to be fair, though. Again, for this benchmark to be meaningful and compelling, this kind of thing should be addressed. > The only use case where Dynamo is better in price is when you store lots of data but with a tiny IOPS reservation This is a surprising statement. If this is the case, I highly recommend addressing the issues so that you are using DynamoDB like a typical engineering team would. A blanket statement like this with questionable benchmarks as evidence is just not compelling. It would be much more helpful to see where DynamoDB really can't stand a chance competing against ScyllaDB even when it's fully optimiized. Are you ignoring maintenance costs? Do nodes get added and removed without paying an engineer? How do you keep the nodes up-to-date as far as security updates or adding new features? How much overhead cost is engineering on-call to respond when the cluster has issues (e.g. nearing capacity limits, nodes failing, loss of AZ availability)? I'm also curious if you are only considering use cases that you typically already see on ScyllaDB and Cassandra. How about the use case where I have a load that increases without warning by 10x or where I need regular backups or that requires 5000 nodes?

thekozmo 8y ago
About multiple AZ, I think we haven't used it in this case but it shouldn't change a bit, most of our AWS customers have AZ-aware deployments and it works great. Since I see you're with AWS, you're welcome to help us debug this error 500 on population. I guarantee we'll update the blog if we'll see it works fine I didn't get your surprise, I actually am 100% open about digging deep and come up with a uncommon use case that Dynamo can finally be a good alternative. Believe me, I can come with examples that will put Scylla in a much better light. We do offer a fully managed service on AWS with 4x-6x cost saving vs Dynamo where all the backup and auto scale is done by the Scylla team.
- otterley 8y ago
  > About multiple AZ, I think we haven't used it in this case but it shouldn't change a bit Are you suggesting that the metrics would not change? I assure you that is not the case: inter-AZ latency is 1-2 orders of magnitude higher than intra-AZ latency. You'll need to re-run your benchmarks in a multi-AZ cluster in order to provide an apples-to-apples comparison.
  - thekozmo 8y ago
    I don't think this is the case - check this link: https://knowledgebase.progress.com/articles/Article/expected-latency-between-aws-availability-zones https://knowledgebase.progress.com/articles/Article/expected... The intra AZ latency according to AWS is sub millisecond! It's inline with what our customers see. Since all of the numbers in the blog are above a millisecond, the result does not change.
    - otterley 8y ago
      Intra-AZ latency is indeed sub-millisecond. Inter-AZ latency is in the milliseconds range. (The intra prefix means within, inter means between.) And those latencies accumulate quickly when benchmarking. Why don't you try it yourself and see the difference? Relying on some third party website is no substitute for testing.

deleted 8y ago

[deleted]