2 ms·

Having worked on an embedded system that dealt primarily with network I/O (10 Gbps) and hash tables (in the 10 GiB range), I can tell you that network I/O (done

Having worked on an embedded system that dealt primarily with network I/O (10 Gbps) and hash tables (in the 10 GiB range), I can tell you that network I/O (done right!) and parsing account for MAYBE 1/3 of the total latency of an operation involving a hash table lookup. Memory, like all mass storage, is SLOW once you're not working in cache, and hash tables have no locality. (By "done right!" I mean either batching requests to/from the kernel, or using a zero-copy userspace solution like DPDK. Clearly in this article network I/O was not done right. Round trips through the kernel ALWAYS will kill performance.)

MichaelGG 12y ago
I've no doubt memory is slow, taking up to 100-300 cycles in NUMA systems. But a single threaded server accessing local memory won't may those costs as much. Are you saying that a few random memory accesses are slower than sending and receiving a packet on two machines? Redis has a great position as a persistent, shareable, data structure server, but replacing in memory hashtables where they work doesn't seem like one of those cases.
- colanderman 12y ago
  Sure... on a 10 Gbps link, you can transfer up to 15 million packets per second (assuming a tiny payload... which is about all you need for a hash lookup). That's about the same order of magnitude as the latency of a single memory lookup that misses the cache. In the right environment (not TCP on a stock Linux kernel!) each packet carries very little CPU overhead. Of course, you can pipeline memory accesses to some degree, but not as easily as you can aggregate network requests into fewer packets. I'm certainly not saying the network overhead is free -- it's not! I'm just saying it needn't "eclipse" the hash table lookup itself (as the GP suggested). They're on the same order of magnitude.
  - MichaelGG 12y ago
    Alright but in the case of Redis, this low latency networking like RDMA or whatnot isn't available right? So, today, off-the-shelf, installing Redis on a remote server, you're probably gonna be off by at least an order of magnitude, no? Also, on the target server, you still have at least one cache miss to retrieve the item from the hashtable (and potentially more for large hashtables). It would be nice if there was a commonly supported API for doing this kind of stuff that doesn't require exotic hardware and came with a usable abstraction something like TCP. I know off-the-shelf Intel NICs have "Direct NIC Access" which can easily do wireline work from userspace, but afaik that's only for Intel NICs, and the API isn't as smooth as most socket users are used to. We need something like SuperSockets, but written to target NIC hardware directly, I guess.

MichaelGG 12y ago

I've no doubt memory is slow, taking up to 100-300 cycles in NUMA systems. But a single threaded server accessing local memory won't may those costs as much. Are you saying that a few random memory accesses are slower than sending and receiving a packet on two machines? Redis has a great position as a persistent, shareable, data structure server, but replacing in memory hashtables where they work doesn't seem like one of those cases.

colanderman 12y ago
Sure... on a 10 Gbps link, you can transfer up to 15 million packets per second (assuming a tiny payload... which is about all you need for a hash lookup). That's about the same order of magnitude as the latency of a single memory lookup that misses the cache. In the right environment (not TCP on a stock Linux kernel!) each packet carries very little CPU overhead. Of course, you can pipeline memory accesses to some degree, but not as easily as you can aggregate network requests into fewer packets. I'm certainly not saying the network overhead is free -- it's not! I'm just saying it needn't "eclipse" the hash table lookup itself (as the GP suggested). They're on the same order of magnitude.
- MichaelGG 12y ago
  Alright but in the case of Redis, this low latency networking like RDMA or whatnot isn't available right? So, today, off-the-shelf, installing Redis on a remote server, you're probably gonna be off by at least an order of magnitude, no? Also, on the target server, you still have at least one cache miss to retrieve the item from the hashtable (and potentially more for large hashtables). It would be nice if there was a commonly supported API for doing this kind of stuff that doesn't require exotic hardware and came with a usable abstraction something like TCP. I know off-the-shelf Intel NICs have "Direct NIC Access" which can easily do wireline work from userspace, but afaik that's only for Intel NICs, and the API isn't as smooth as most socket users are used to. We need something like SuperSockets, but written to target NIC hardware directly, I guess.