3 ms·

With the caveat that I'd love to see Scylla succeed, I'm a bit bummed to see the conditions around this bake-off so slanted in Scylla's favor from the start. S

With the caveat that I'd love to see Scylla succeed, I'm a bit bummed to see the conditions around this bake-off so slanted in Scylla's favor from the start. Seeing a multi-AZ configuration compared to a single-AZ config is an immediate disqualifier, without getting into some of the other comments on item size disparity and YCSB quirks. This was a really poor response from Scylla on these fronts: https://news.ycombinator.com/item?id=18677511 https://news.ycombinator.com/item?id=18677511

thekozmo 8y ago
I went and run a ping between 2 VMs, one on us-east1c and the other on a different AZ - us-east-1b As you can see, the ping latency is 0.5ms to 1ms on rare occasions. Of course that ping has a cost even within a single AZ so the above numbers reflect much more than the AZ round trip time. You're welcome to add 1ms to our latency numbers, it's 3x-4x better in the 99th case so there's enough room... 64 bytes from 35.173.249.24: icmp_seq=13 ttl=63 time=0.792 ms 64 bytes from 35.173.249.24: icmp_seq=14 ttl=63 time=0.584 ms 64 bytes from 35.173.249.24: icmp_seq=15 ttl=63 time=0.462 ms 64 bytes from 35.173.249.24: icmp_seq=16 ttl=63 time=0.848 ms 64 bytes from 35.173.249.24: icmp_seq=17 ttl=63 time=0.465 ms 64 bytes from 35.173.249.24: icmp_seq=18 ttl=63 time=0.451 ms 64 bytes from 35.173.249.24: icmp_seq=19 ttl=63 time=1.00 ms 64 bytes from 35.173.249.24: icmp_seq=20 ttl=63 time=0.854 ms 64 bytes from 35.173.249.24: icmp_seq=21 ttl=63 time=0.446 ms 64 bytes from 35.173.249.24: icmp_seq=22 ttl=63 time=0.460 ms 64 bytes from 35.173.249.24: icmp_seq=23 ttl=63 time=0.509 ms
- awinder 8y ago
  There’s still 2 unresolved problems (at least) here: 1. running a ping test is a fine start but wouldn’t a more realistic test be to rerun the benchmark? 2. in a multi-az setup you’re going to have to pay for inter-az transfer costs between ec2 nodes. data transfer between ec2 and dynamodb is free. Also on the pricing front reserved capacity really drops your costs with dynamodb if you know you’re going to be with it for some term of time. Not that that needed to be included in the benchmark but... aws pricing is complicated, to say the least.
  - thekozmo 8y ago
    True. In our brand new service offering we took it all into account. Networking is costly but doesn't change the game that much. Scylla is 4x-6x cheaper, depending whether it's on-demand (vs Dynamo on-demand) or reserved for a year (vs dynamo reservation) https://www.scylladb.com/product/scylla-cloud/#pricing https://www.scylladb.com/product/scylla-cloud/#pricing

thekozmo 8y ago

I went and run a ping between 2 VMs, one on us-east1c and the other on a different AZ - us-east-1b As you can see, the ping latency is 0.5ms to 1ms on rare occasions. Of course that ping has a cost even within a single AZ so the above numbers reflect much more than the AZ round trip time. You're welcome to add 1ms to our latency numbers, it's 3x-4x better in the 99th case so there's enough room... 64 bytes from 35.173.249.24: icmp_seq=13 ttl=63 time=0.792 ms 64 bytes from 35.173.249.24: icmp_seq=14 ttl=63 time=0.584 ms 64 bytes from 35.173.249.24: icmp_seq=15 ttl=63 time=0.462 ms 64 bytes from 35.173.249.24: icmp_seq=16 ttl=63 time=0.848 ms 64 bytes from 35.173.249.24: icmp_seq=17 ttl=63 time=0.465 ms 64 bytes from 35.173.249.24: icmp_seq=18 ttl=63 time=0.451 ms 64 bytes from 35.173.249.24: icmp_seq=19 ttl=63 time=1.00 ms 64 bytes from 35.173.249.24: icmp_seq=20 ttl=63 time=0.854 ms 64 bytes from 35.173.249.24: icmp_seq=21 ttl=63 time=0.446 ms 64 bytes from 35.173.249.24: icmp_seq=22 ttl=63 time=0.460 ms 64 bytes from 35.173.249.24: icmp_seq=23 ttl=63 time=0.509 ms

awinder 8y ago
There’s still 2 unresolved problems (at least) here: 1. running a ping test is a fine start but wouldn’t a more realistic test be to rerun the benchmark? 2. in a multi-az setup you’re going to have to pay for inter-az transfer costs between ec2 nodes. data transfer between ec2 and dynamodb is free. Also on the pricing front reserved capacity really drops your costs with dynamodb if you know you’re going to be with it for some term of time. Not that that needed to be included in the benchmark but... aws pricing is complicated, to say the least.
- thekozmo 8y ago
  True. In our brand new service offering we took it all into account. Networking is costly but doesn't change the game that much. Scylla is 4x-6x cheaper, depending whether it's on-demand (vs Dynamo on-demand) or reserved for a year (vs dynamo reservation) https://www.scylladb.com/product/scylla-cloud/#pricing https://www.scylladb.com/product/scylla-cloud/#pricing