3 ms·

I do not know about aerospike, have never run their software but Scylla are the champions in what they are doing. Having said that, Scylla provide full durabili

I do not know about aerospike, have never run their software but Scylla are the champions in what they are doing. Having said that, Scylla provide full durability guarantees, that means they need to touch a disk upon each write or update. Disk I/O is relativle expensive in the cloud and requires either large instances or special storage optimized family types like i3 etc. Long story short, I do think Dragonfly is the fastest in-memory database in terms of throughput and latency today. We will see if we manage to stay this way when we extend our capabilities with SSD tiering.

ddorian43 4y ago
> that means they need to touch a disk upon each write or update They don't fsync the wal on every write, it's probably done as group commit every x seconds or every x MB.
- romange 4y ago
  they need at least perform fdatasync before replying to a client, imho. so I do not think it's every x seconds. I am not deeply familiar with scylla so maybe I am mistaken.
  - ddorian43 4y ago
    I think you're wrong. It does that every x seconds. The disk writing is async and it's meant that you protect data-loss by keeping replicas in different availability zones.
    - romange 4y ago
      interesting. i still need to learn a lot about how other systems provide durability guarantees. Based on what you are saying, Scylla guarantees H/A instead of full durability. I was thinking about WAL (AOF) for dragonfly and I kinda thought to start with "fdatasync once in every X seconds", but dragonfly is not a database. I would expect stronger guarantees from databases. Maybe it's inpractical to expect this in cloud environments...
      - ddorian43 4y ago
        It wouldn't have the performance that it has. Cassandra does the same. YugabyteDB does the same https://docs.yugabyte.com/preview/reference/configuration/yb-tserver/#durable-wal-write https://docs.yugabyte.com/preview/reference/configuration/yb.... > I would expect stronger guarantees from databases. You can configure it and get lower performance. > Maybe it's inpractical to expect this in cloud environments... For benchmarks they use raid0 on local disks(not EBS) in aws vps.

ddorian43 4y ago

> that means they need to touch a disk upon each write or update They don't fsync the wal on every write, it's probably done as group commit every x seconds or every x MB.

romange 4y ago
they need at least perform fdatasync before replying to a client, imho. so I do not think it's every x seconds. I am not deeply familiar with scylla so maybe I am mistaken.
- ddorian43 4y ago
  I think you're wrong. It does that every x seconds. The disk writing is async and it's meant that you protect data-loss by keeping replicas in different availability zones.
  - romange 4y ago
    interesting. i still need to learn a lot about how other systems provide durability guarantees. Based on what you are saying, Scylla guarantees H/A instead of full durability. I was thinking about WAL (AOF) for dragonfly and I kinda thought to start with "fdatasync once in every X seconds", but dragonfly is not a database. I would expect stronger guarantees from databases. Maybe it's inpractical to expect this in cloud environments...
    - ddorian43 4y ago
      It wouldn't have the performance that it has. Cassandra does the same. YugabyteDB does the same https://docs.yugabyte.com/preview/reference/configuration/yb-tserver/#durable-wal-write https://docs.yugabyte.com/preview/reference/configuration/yb.... > I would expect stronger guarantees from databases. You can configure it and get lower performance. > Maybe it's inpractical to expect this in cloud environments... For benchmarks they use raid0 on local disks(not EBS) in aws vps.