4 ms·
So how do you get a web server to take advantage of a hardware gzip accelerator? Is there like a custom nginx plugin that points to a custom driver? Seems very
by triangleman 6y ago
So how do you get a web server to take advantage of a hardware gzip accelerator? Is there like a custom nginx plugin that points to a custom driver? Seems very cool and I've never heard of something like this.
- castratikron 6y agoMaybe hooks into libz somehow?
- yjftsjthsd-h 6y agoOooh, that would work; just provide your own libz that's a wrapper around the hardware. You could even package it up and do a drop in replacement:)
- tverbeure 6y agoAccording to their press release, it was an Apache plugin. https://www.businesswire.com/news/home/20081008005207/en/Comtech-AHA-Announces-Apache-GZIP-Compression-Module https://www.businesswire.com/news/home/20081008005207/en/Com...
- mng2 6y agoThey mention APIs for Windows, Linux, OpenSolaris, a ZLIB, and the Apache module. http://www.aha.com/DrawProducts.aspx?Action=GetProductDetails&ProductID=14 http://www.aha.com/DrawProducts.aspx?Action=GetProductDetail...
- tyingq 6y agoIt was also pretty common in the early days of SSL to have a similar outboard accelerator and webserver plugins to call it. That still exists for Allwinner CPUs. You can offload the TLS handshake.
- triangleman 6y agoThat's so cool. Has anyone made a JSON-parsing chip, because I could see that being useful in today's servers.
- BooneJS 6y agoLots of NICs for large supercomputers have special message passing offload engines. But JSON is too ASCII and too random to find a reason to accelerate it.
- wereHamster 6y agoIs JSON parsing still a bottleneck? https://github.com/simdjson/simdjson https://github.com/simdjson/simdjson
- lpgauth 6y agomixing simd and non-simd load is not recommended. https://blog.cloudflare.com/on-the-dangers-of-intels-frequency-scaling/ https://blog.cloudflare.com/on-the-dangers-of-intels-frequen...
- bri3d 6y agothis is well-addressed here: https://github.com/simdjson/simdjson/blob/master/doc/performance.md#downclocking https://github.com/simdjson/simdjson/blob/master/doc/perform... . first off, AVX2 is a subset of SIMD instructions - SIMDjson includes a large number of implementations including several which do not use AVX2 at all. next, only a subset of AVX2 instructions cause downclocking (namely, the 512 bit wide ones), which SIMDjson does not use. throwing out the use of "SIMD" entirely because you read an article about a specific AVX512 instruction which caused down-clocking might be a bit premature, no?
- the8472 6y agoOne way to do it is to expose it as an nvme block device. Not sure how that's used from userspace, I suspect it's either the right combination of mmap/vmsplice/sendfile from userspace to do p2pdma between the devices in question or some custom ioctls. https://www.snia.org/sites/default/files/SDC/2019/presentations/Computational/Bates_Stephen_Accelerating_RocksDB_with_Eideticom%E2%80%99s_NoLoad_NVMe-based_Computational_Storage_Processor.pdf https://www.snia.org/sites/default/files/SDC/2019/presentati...