3 ms·

~2000 write IOPS ~180000 read IOPS What are those IOPS in this case? Queries? Transactions? Disk block accesses?

ddorian43 9y ago
They probably are GET and INSERT/UPSERT/UPDATE single rows. Seems kinda low, need to also say how many GB 2B rows are.
drodgers 9y ago
I'm pretty sure they mean operations on the block storage layer (EBS) as reported by AWS CloudWatch monitoring. It's a standard measure: https://en.wikipedia.org/wiki/IOPS https://en.wikipedia.org/wiki/IOPS
- danbruc 9y ago
  That is why I asked, I am only aware of the use of IOPS in connection with disk I/O and it seems a rather unusual measure for a database where caches hopefully avoid hitting the disk too often, at least for reads. And writing 8 MiB/s assuming 4k clusters seems not really noteworthy especially given three fold redundancy. On the other hand reading 700 MiB/s per second which would also not be affected by the redundancy seems a comparatively big number especially because caches should limit the disk traffic to a small fraction.
  - z3t4 9y ago
    Databases usually don't use caches. It's cheaper to just ask the file-system for the data.
    - barkanido 9y ago
      In Aerospike case, they do not use a file system. they own the mount point and access the disk blocks directly, optimizing for local SSD disks.
    - danbruc 9y ago
      This is not true at all, bordering to utter nonsense. Databases try hard to keep the correct set of blocks in memory because that is essential for their performance. Heck, many of the fastest database systems advertise themselves as in-memory databases avoiding disks altogether.
- barkanido 9y ago
  Queries here are measured from the application side. And in our case, those are simple key-value operations such as get(key), update(key, value), write(key, value) and delete(key). Aerospike is known to optimize those into fewer block operation as it can.
  - dkoston 9y ago
    No N1QL was involved that needed to be ported/re-architected? N1QL was our primary use case for using Couchbase at my last position and it worked wonderfully (though at less than half your scale). For straight key operations, the overhead of all the JVM stuff on couchbase seems like a lot, especially when considering the alternative amount of K/V stores these days. Glad to hear Aerospike turned out well. Been keeping an eye on them over the past few years.
jtchang 9y ago
Isn't the point of AWS / RDS to be able to scale infinitely? I don't remember if RDS has one of their data backends that they are using.
- toomuchtodo 9y ago
  This is mostly marketing. You can scale to infinity only if you're app is architected in a very particular way and you're properly using the underlying primitives (be prepared to shard, cache, and queue everything at some point; try to put thought into where your demarcation points will be, it will make future decoupling less painful). Even Aurora Postgresql and Mysql only scale so far. One click it is not.

They probably are GET and INSERT/UPSERT/UPDATE single rows. Seems kinda low, need to also say how many GB 2B rows are.

I'm pretty sure they mean operations on the block storage layer (EBS) as reported by AWS CloudWatch monitoring. It's a standard measure: https://en.wikipedia.org/wiki/IOPS https://en.wikipedia.org/wiki/IOPS

danbruc 9y ago
That is why I asked, I am only aware of the use of IOPS in connection with disk I/O and it seems a rather unusual measure for a database where caches hopefully avoid hitting the disk too often, at least for reads. And writing 8 MiB/s assuming 4k clusters seems not really noteworthy especially given three fold redundancy. On the other hand reading 700 MiB/s per second which would also not be affected by the redundancy seems a comparatively big number especially because caches should limit the disk traffic to a small fraction.
- z3t4 9y ago
  Databases usually don't use caches. It's cheaper to just ask the file-system for the data.
  - barkanido 9y ago
    In Aerospike case, they do not use a file system. they own the mount point and access the disk blocks directly, optimizing for local SSD disks.
  - danbruc 9y ago
    This is not true at all, bordering to utter nonsense. Databases try hard to keep the correct set of blocks in memory because that is essential for their performance. Heck, many of the fastest database systems advertise themselves as in-memory databases avoiding disks altogether.
barkanido 9y ago
Queries here are measured from the application side. And in our case, those are simple key-value operations such as get(key), update(key, value), write(key, value) and delete(key). Aerospike is known to optimize those into fewer block operation as it can.
- dkoston 9y ago
  No N1QL was involved that needed to be ported/re-architected? N1QL was our primary use case for using Couchbase at my last position and it worked wonderfully (though at less than half your scale). For straight key operations, the overhead of all the JVM stuff on couchbase seems like a lot, especially when considering the alternative amount of K/V stores these days. Glad to hear Aerospike turned out well. Been keeping an eye on them over the past few years.

jtchang 9y ago

Isn't the point of AWS / RDS to be able to scale infinitely? I don't remember if RDS has one of their data backends that they are using.

toomuchtodo 9y ago
This is mostly marketing. You can scale to infinity only if you're app is architected in a very particular way and you're properly using the underlying primitives (be prepared to shard, cache, and queue everything at some point; try to put thought into where your demarcation points will be, it will make future decoupling less painful). Even Aurora Postgresql and Mysql only scale so far. One click it is not.