3 ms·

Stop using RAM for caching, SSDs are just as fast now with LevelDB/RocksDB and can store a magnitude more data. Look into SSDB [1] instead of Memcached. I know

by welder 5y ago

Stop using RAM for caching, SSDs are just as fast now with LevelDB/RocksDB and can store a magnitude more data. Look into SSDB [1] instead of Memcached. I know it's Redis, but it's basically the same as Memcached if you only use the key-value set/get commands. See this blog post [2] and this Gophercon 2021 talk [3] for examples of it's use in production. [1] https://github.com/ideawu/ssdb https://github.com/ideawu/ssdb [2] https://wakatime.com/blog/45-using-a-diskbased-redis-clone-to-reduce-aws-s3-bill https://wakatime.com/blog/45-using-a-diskbased-redis-clone-t... [3] https://www.youtube.com/watch?v=35eIxI_n5ZM&t=10533s https://www.youtube.com/watch?v=35eIxI_n5ZM&t=10533s

ec109685 5y ago
SSDB is faster even if data size is >> ram size? Otherwise, you’re going to be adding equivalent memory to get to equivalent performance between memcached and ssdb.
jeffbee 5y ago
I mean, hard no on both claims. A memory cache (even memcached, which might not be state of the art) stomps all over the performance claims of SSDB. Memcached can do ~250kqps on one core, roughly depending of exactly what kind of cores are in question, and can typically turn around a request in < 50µs. SSDB is only "just as fast" if the rest of your stack is so insensitive to performance that you can't measure an order of magnitude difference in service latency.
- scrubs 5y ago
  agree - most sstables use memory looking on disk if needed. ssds do not have the bandwidth or low latency RAM has apart from the additional I/O path to get it onto or out of disk. Now, not to say memory+ssd can't rock ... but ssd!=DDRAM.
- tayo42 5y ago
  > which might not be state of the art Sorry this stuck out to me. what would be state of the art in the in memory kv store world then?
  - jeffbee 5y ago
    I don’t know I’m just keeping an open mind. I imagine that Amazon or Google or one of those probably has something with an RDMA interface that smokes memcached.
aseipp 5y ago
There are no latency numbers to be seen anywhere unless I missed them. If the difference in latency between RAM/disk isn't noticeable in these workloads because you have enough slack in the system, the distinction doesn't really matter, you can use whatever (case in point, the linked article uses cost-of-egress as the sole motivation, it doesn't even mention latency or QPS at all.) Memcached also has a mechanism for storing values (but not keys) on disk for larger-than-memory datasets which works well on big parallel flash drives. EDIT: The website seems down but I was able to find a graph on archive.org via https://web.archive.org/web/20210721201604/https://ssdb.io/ https://web.archive.org/web/20210721201604/https://ssdb.io/ but without latency, only QPS in some simple benchmarks. It's not slow, but if you're pushing high QPS-per-core or low latency, there's no indication it will serve as well as alternatives, and that can have a bigger impact on ROI than just the cost of disk-vs-memory for your servers.
halayli 5y ago
Just because it's on the internet it doesn't mean it's true. You might be confused how a CPU, memory, and I/O are interconnected. First, for an application to access data in memory, all it need to do is issue a MOV with indirect addressing. It might trigger a minor page fault which gets handled quickly by the OS. For an application to read data from disk, it must submit a request to the disk controller through the OS. I am ignoring a lot of details but that's all you need to reconsider the information in the links you shared. Keep in mind that reading from memory is in the nanoseconds, while reading from NVME is in microseconds.
- jgod 5y ago
  Doesn't matter if either way, with network overhead from talking to your application from another machine, it's measured in milliseconds.
  - halayli 5y ago
    Of course it matters. You probably haven't written low latency server daemons. A busy memcached might be serving 20k to 100k requests a second, it's not serving 1 user only. 100k requests all being translated to memory vs disk reads will have a huge performance impact.
  - ec109685 5y ago
    Many caches including Memcached support batch gets as well as pipelining, so network latency won’t dominate.

by welder 5y ago

ec109685 5y ago

SSDB is faster even if data size is >> ram size? Otherwise, you’re going to be adding equivalent memory to get to equivalent performance between memcached and ssdb.

jeffbee 5y ago

I mean, hard no on both claims. A memory cache (even memcached, which might not be state of the art) stomps all over the performance claims of SSDB. Memcached can do ~250kqps on one core, roughly depending of exactly what kind of cores are in question, and can typically turn around a request in < 50µs. SSDB is only "just as fast" if the rest of your stack is so insensitive to performance that you can't measure an order of magnitude difference in service latency.

scrubs 5y ago
agree - most sstables use memory looking on disk if needed. ssds do not have the bandwidth or low latency RAM has apart from the additional I/O path to get it onto or out of disk. Now, not to say memory+ssd can't rock ... but ssd!=DDRAM.
tayo42 5y ago
> which might not be state of the art Sorry this stuck out to me. what would be state of the art in the in memory kv store world then?
- jeffbee 5y ago
  I don’t know I’m just keeping an open mind. I imagine that Amazon or Google or one of those probably has something with an RDMA interface that smokes memcached.

aseipp 5y ago

There are no latency numbers to be seen anywhere unless I missed them. If the difference in latency between RAM/disk isn't noticeable in these workloads because you have enough slack in the system, the distinction doesn't really matter, you can use whatever (case in point, the linked article uses cost-of-egress as the sole motivation, it doesn't even mention latency or QPS at all.) Memcached also has a mechanism for storing values (but not keys) on disk for larger-than-memory datasets which works well on big parallel flash drives. EDIT: The website seems down but I was able to find a graph on archive.org via https://web.archive.org/web/20210721201604/https://ssdb.io/ https://web.archive.org/web/20210721201604/https://ssdb.io/ but without latency, only QPS in some simple benchmarks. It's not slow, but if you're pushing high QPS-per-core or low latency, there's no indication it will serve as well as alternatives, and that can have a bigger impact on ROI than just the cost of disk-vs-memory for your servers.

halayli 5y ago

Just because it's on the internet it doesn't mean it's true. You might be confused how a CPU, memory, and I/O are interconnected. First, for an application to access data in memory, all it need to do is issue a MOV with indirect addressing. It might trigger a minor page fault which gets handled quickly by the OS. For an application to read data from disk, it must submit a request to the disk controller through the OS. I am ignoring a lot of details but that's all you need to reconsider the information in the links you shared. Keep in mind that reading from memory is in the nanoseconds, while reading from NVME is in microseconds.

jgod 5y ago
Doesn't matter if either way, with network overhead from talking to your application from another machine, it's measured in milliseconds.
- halayli 5y ago
  Of course it matters. You probably haven't written low latency server daemons. A busy memcached might be serving 20k to 100k requests a second, it's not serving 1 user only. 100k requests all being translated to memory vs disk reads will have a huge performance impact.
- ec109685 5y ago
  Many caches including Memcached support batch gets as well as pipelining, so network latency won’t dominate.