8 ms·

An FPGA-based In-line Accelerator for Memcached (2013) [pdf]

89 points by cleong 8y ago

konschubert 8y ago
These kind of optimisations will become more common as we near the end of Moore's law.
dormando 8y ago
This seems old (2013ish?). There're newer "key/value on FPGA" papers that're more modern. If memcached papers have one thing in common, it's an uncanny ability to get the comparison software to run as slowly as possible. 100k ops/sec/core is what you get when using a single client connection with blocking I/O. Using more clients (as per a normal prod setup) or pipelining queries is more like 1m ops+/core, with writes scaling worse than reads. In production it's easy to get some level of pipelining (multigets, clustered keys, etc), since you're rarely just fetching a key and then blocking. A much better FPGA paper would show scales of at what level the syscall overhead becomes most of the CPU usage, as well as any measured latency improvements. I think some of the other papers address latency at least. In reality it hardly matters. If you're hitting memcached hard enough with tiny values for it to matter, ensuring keys are clustered and pipelined is a lot less maintenance overhead than deploying FPGA's.
- ksec 8y ago
  Especially when we have Xeon -D that goes up to 16 Core. AMD EPYC that gives more Core per dollar. And more IPC soon in Zen 2. 7nm and 10nm from AMD and Intel next year on Server. Not to mention it now Support ARMv8 So Excuse my ignorance, apart from AWS or Azure Scales, what would anyone uses Memcached on FPGA? * I wouldn't mind if the system was simple plug and play and has all the benefits, cost saving without the headache. But very rarely are any technology deployment without any headache or hassle free.
- glangdale 8y ago
  Can confirm this eerie ability of FPGA and h/w folks from another domain (regular expressions - I'm the designer of Hyperscan, a s/w solution). Every corner case that could be found in s/w was always the topic of a excited benchmark. Also, the old trick of 'hey, let's drop all the matches on the floor in our h/w or FPGA, while getting a huge number of matches in s/w and making the s/w guys look ridiculous'. Every time I read a paper touting a great new speedup on FPGA (over some crap s/w implementation) I'm reminded of that old joke about the Texan visiting Israel and telling the owner of some small farm that "he can get on a tractor and ride for days without getting the the boundary of his property." The Israeli nods sympathetically and says "Yes, I too used to have a tractor like that".
  - rjeli 8y ago
    I don’t understand the joke?
    - namibj 8y ago
      I found https://www.reddit.com/r/Jokes/comments/2uj2ot/a_texas_rancher_was_visiting_a_farmer_in_israel/ https://www.reddit.com/r/Jokes/comments/2uj2ot/a_texas_ranch... And it seems to be not that good of a joke.
    - nfriedly 8y ago
      I think the Texan is thinking of a regular tractor on a huge farm, while the Israeli is thinking of a slow/broken tractor on a moderately sized farm.
  - frozenport 8y ago
    Its like this with most of academia
andrewcanis 8y ago
Our startup is working on accelerators using FPGAs on AWS including memcached. Using a single AWS F1 (FPGA) instance, our Memcached accelerator achieves over 11 million ops/sec at less than 300 microsecond latency. Compared to ElastiCache, the AWS-managed CPU Memcached server, our Memcached accelerator offers 9X better throughput, 9X lower latency, and 10X better throughput/$. We need to batch multiple requests per Ethernet packet to get around packet per sec rate limiting on AWS. See more details here: https://www.legupcomputing.com/blog/index.php/2018/05/01/deep-dive-into-the-worlds-fastest-cloud-hosted-memcached/ https://www.legupcomputing.com/blog/index.php/2018/05/01/dee... If anyone is interested we would love to hear from you, we will be showing off an online demo later this week. FPGAs are great for processing data at 10Gbps line rate with low latency. They are also good for compute tasks like compression and encryption.
- dormando 8y ago
  How is elasticache so slow? what instances does it run on? edit: r4.4xlarge as per the link. 16vcpu? You should be able to beat on latency but beating on throughput means elasticache is misconfigured, likely. Or you're putting on way too much set traffic (think I saw you set the bench to 1:1 ratio of gets to sets?)
  - andrewcanis 8y ago
    I wouldn't characterize Elasticache as running slow, a single instance in this case is handling 1.3M request/sec. But we can be 9X faster by batching multiple requests per packet and then offloading the TCP network stack and memcached operations to the FPGA. The FPGA allows us to handle the requests at network line-rate, even with small 100-byte requests. On Elasticache, past a certain point these small requests start to overload the CPU. The interesting part is the FPGA could still do much more computation (for example, compression or encryption) while maintaining the same throughput due to hardware pipelining. We described this concept further in the blog post I linked to.
    - dormando 8y ago
      I characterize it as slow because I know it can saturate the packet rate AWS gives it with software memcached. If the packet rate were much higher then you might win out. The only reason why you can claim 9x latency is because you've saturated the worker threads. You should still win on latency even if it were properly bottlenecking on the network, but 9x throughput and 9x latency is completely false as a capacity limit in this test. The other issue is 100 bytes isn't typical. It's common but almost every user has a varied workload. Deploying FPGA's for the larger cache values ends up being a waste. I designed a new storage system based off of offloading larger cold keys to flash, even.

89 points by cleong 8y ago

These kind of optimisations will become more common as we near the end of Moore's law.

This seems old (2013ish?). There're newer "key/value on FPGA" papers that're more modern. If memcached papers have one thing in common, it's an uncanny ability to get the comparison software to run as slowly as possible. 100k ops/sec/core is what you get when using a single client connection with blocking I/O. Using more clients (as per a normal prod setup) or pipelining queries is more like 1m ops+/core, with writes scaling worse than reads. In production it's easy to get some level of pipelining (multigets, clustered keys, etc), since you're rarely just fetching a key and then blocking. A much better FPGA paper would show scales of at what level the syscall overhead becomes most of the CPU usage, as well as any measured latency improvements. I think some of the other papers address latency at least. In reality it hardly matters. If you're hitting memcached hard enough with tiny values for it to matter, ensuring keys are clustered and pipelined is a lot less maintenance overhead than deploying FPGA's.

ksec 8y ago
Especially when we have Xeon -D that goes up to 16 Core. AMD EPYC that gives more Core per dollar. And more IPC soon in Zen 2. 7nm and 10nm from AMD and Intel next year on Server. Not to mention it now Support ARMv8 So Excuse my ignorance, apart from AWS or Azure Scales, what would anyone uses Memcached on FPGA? * I wouldn't mind if the system was simple plug and play and has all the benefits, cost saving without the headache. But very rarely are any technology deployment without any headache or hassle free.
glangdale 8y ago
Can confirm this eerie ability of FPGA and h/w folks from another domain (regular expressions - I'm the designer of Hyperscan, a s/w solution). Every corner case that could be found in s/w was always the topic of a excited benchmark. Also, the old trick of 'hey, let's drop all the matches on the floor in our h/w or FPGA, while getting a huge number of matches in s/w and making the s/w guys look ridiculous'. Every time I read a paper touting a great new speedup on FPGA (over some crap s/w implementation) I'm reminded of that old joke about the Texan visiting Israel and telling the owner of some small farm that "he can get on a tractor and ride for days without getting the the boundary of his property." The Israeli nods sympathetically and says "Yes, I too used to have a tractor like that".
- rjeli 8y ago
  I don’t understand the joke?
  - namibj 8y ago
    I found https://www.reddit.com/r/Jokes/comments/2uj2ot/a_texas_rancher_was_visiting_a_farmer_in_israel/ https://www.reddit.com/r/Jokes/comments/2uj2ot/a_texas_ranch... And it seems to be not that good of a joke.
  - nfriedly 8y ago
    I think the Texan is thinking of a regular tractor on a huge farm, while the Israeli is thinking of a slow/broken tractor on a moderately sized farm.
- frozenport 8y ago
  Its like this with most of academia

andrewcanis 8y ago

Our startup is working on accelerators using FPGAs on AWS including memcached. Using a single AWS F1 (FPGA) instance, our Memcached accelerator achieves over 11 million ops/sec at less than 300 microsecond latency. Compared to ElastiCache, the AWS-managed CPU Memcached server, our Memcached accelerator offers 9X better throughput, 9X lower latency, and 10X better throughput/$. We need to batch multiple requests per Ethernet packet to get around packet per sec rate limiting on AWS. See more details here: https://www.legupcomputing.com/blog/index.php/2018/05/01/deep-dive-into-the-worlds-fastest-cloud-hosted-memcached/ https://www.legupcomputing.com/blog/index.php/2018/05/01/dee... If anyone is interested we would love to hear from you, we will be showing off an online demo later this week. FPGAs are great for processing data at 10Gbps line rate with low latency. They are also good for compute tasks like compression and encryption.

dormando 8y ago
How is elasticache so slow? what instances does it run on? edit: r4.4xlarge as per the link. 16vcpu? You should be able to beat on latency but beating on throughput means elasticache is misconfigured, likely. Or you're putting on way too much set traffic (think I saw you set the bench to 1:1 ratio of gets to sets?)
- andrewcanis 8y ago
  I wouldn't characterize Elasticache as running slow, a single instance in this case is handling 1.3M request/sec. But we can be 9X faster by batching multiple requests per packet and then offloading the TCP network stack and memcached operations to the FPGA. The FPGA allows us to handle the requests at network line-rate, even with small 100-byte requests. On Elasticache, past a certain point these small requests start to overload the CPU. The interesting part is the FPGA could still do much more computation (for example, compression or encryption) while maintaining the same throughput due to hardware pipelining. We described this concept further in the blog post I linked to.
  - dormando 8y ago
    I characterize it as slow because I know it can saturate the packet rate AWS gives it with software memcached. If the packet rate were much higher then you might win out. The only reason why you can claim 9x latency is because you've saturated the worker threads. You should still win on latency even if it were properly bottlenecking on the network, but 9x throughput and 9x latency is completely false as a capacity limit in this test. The other issue is 100 bytes isn't typical. It's common but almost every user has a varied workload. Deploying FPGA's for the larger cache values ends up being a waste. I designed a new storage system based off of offloading larger cold keys to flash, even.