5 ms·
> - Has instructions for accelerating gzip. Has there been any research for finding out how much these custom instructions have accelerated modern computationa
by burtonator 6y ago
> - Has instructions for accelerating gzip.
Has there been any research for finding out how much these custom instructions have accelerated modern computational power in practice?
There are a lot of these instructions in modern CPUs for AES, gzip, SSL, etc but you have to have a library coded to support them.
I'm curious what the performance improvements look like in practice.
- jiggawatts 6y agoGlad you asked! I recently did a bunch of tests to see what the "ultimate bottlenecks" are for basic web applications. Think latency to the database and AES256 throughput. Some rough numbers: - Local latency to SQL Server from ASP.NET is about 150 μs, or about 6000 synchronous queries per second, max. - Even with SR-IOV and Mellanox adapters, that rises to 250 μs if a physical network hop is involved. - Typical networks have a latency floor around 500-600 μs, and it's not uncommon to see 1.3 ms VM-to-VM. Now we're down to 800 queries per second! - Similarly, older CPUs struggle to exceed 250 MB/s/core (2 Gbps) for AES256, which is the fundamental limit to HTTPS throughput for a single client. - Newer CPUs, e.g.: AMD EPYC or any recent Intel Xeon can do about 1 GB/s/core, but I haven't seen any CPUs that significantly exceed that. That's not even 10 Gbps. If you have a high-spec cloud VM with 40 or 50 Gbps NICs, there is no way a single HTTPS stream can saturate that link. You have to parallelise somehow to get the full throughout (or drop encryption.) - HTTPS accelerators such as F5 BIG IP or Citrix ADC (NetScaler) are actually HTTPS decelerators for individual users, because even hardware models with SSL offload cards can't keep up with the 1 GB/s from a modern CPU. Their SSL cards are designed for improving the aggregate bandwidth of hundreds of simultaneous streams, and don't do well at all for a single stream, or even a couple of concurrent streams. This matters when "end-to-end encryption" is mandated, because back end connections are often pooled. So you end up with N users being muxed onto just one back-end connection which then becomes the bottleneck.
- mjb 6y ago> Newer CPUs, e.g.: AMD EPYC or any recent Intel Xeon can do about 1 GB/s/core, but I haven't seen any CPUs that significantly exceed that. Modern processors can do ~2.1GB/s per core for AES-256-GCM (about 2x what they can do for AES-CBC).
- dragontamer 6y agoIt should be noted that AES-CBC doesn't have any instruction-level parallelism available. Both EPYC and Xeons have 2-AES units per core now. But CBC can only effectively use one of them at a time. (Block(n+1) cannot be computed until block(n) is done computing. Because Block(n) is used as input into Block(n+1) in CBC mode). AES-256-GCM can compute block(n) and block(n+1) simultaneously. So you need to use such a parallel algorithm if you actually want to use the 2x AES pipelines on EPYC or Xeon.
- formerly_proven 6y agoMore like ~3.9 GB/s for a desktop CPU and somewhere north of 7 GB/s for AES-256-CTR. The performance difference between CBC and CTR is pretty much exactly what you'd expect, you can only use one out of two units, and you have to wait out the full latency (1:4 disadvantage on recent cores) => ~8x slower.
- Xorlev 6y agoRTT between VMs in the same GCP zone is often well below 100us. Are you also measuring the latency of the query itself? I realize not all networks are built the same, but it seems like your benchmark case is much worse than what's possible, even without buying specialized hardware.
- jiggawatts 6y agoThe test was to run "SELECT 1" using the low-level ADO.NET database query API in a tight loop. This is the relevant metric, as it represents the performance ceiling. It doesn't matter how fast the packets can get on the wire if the application can't utilise this because of some other bottleneck. Of course, the underlying TCP latency is significantly lower. Using Microsoft's "Latte.exe" testing tool, I saw ~50 μs in Azure with "Accelerated Networking" enabled. As far as I know, they use Mellanox adapters. Something I found curious is that no matter what I did, the local latency wouldn't go below about 125 μs. Neither shared memory nor named pipes had any benefit. This is on a 4 GHz computer, so in practice this is the "ultimate latency limit" for SQL Server, unless Intel and AMD start up the megahertz war again... It would be an interesting exercise comparing the various database engines to see what their latency overheads are, and what their response time is to trivial queries such as selecting a single row given a key. Unfortunately, due to the DeWitt clauses in EULAs, this would be risky to publish...
- wmf 6y agoGiven that server processors have up to 64 cores now, it sounds like the NIC (even 200 Gbps) is squarely the bottleneck.
- jiggawatts 6y agoNot for any single user or connection. High capacity doesn't equal low latency, and scalability to many users doesn't necessarily help one user get better performance! Very often, you'll see n-tier applications where for some reason (typically load-balancers), the requests are muxed into a single TCP stream. In the past, this improved efficiency by eliminating the per-connection overhead. Now, with high core counts and high bandwidths, some parallelism is absolutely required to even begin to approach the performance ceiling. If the application is naturally single-threaded, such as some ETL jobs, these single-stream bottlenecks are very difficult to overcome. In the field, I very often see 8-core VMs with a "suspicious" utilisation graph flatlining at 12.5% because of issues like this. It boils my blood when people say that this is perfectly fine, because clearly the server has "adequate capacity". In reality, there's a problem, and the server is at 100% of its capacity an the other 7 cores are just heating the data centre air.
- sixothree 6y agoFunny. 800 queries per second is pretty much exactly where my SQL Server applications are running.
- jiggawatts 6y agoYou and pretty much every modern data centre application, irrespective of the technology stack. The typical 1.3ms latency I see kills performance across the board, but very few people are aware of the magnitude of the issue.
- sixothree 6y agoMy work around is to add more to each query. Instead of doing inserts, I do bulk copy statements with thousands of rows. Not particularly fun.
- devy 6y agoYeah, I had the same thought. Also gzip has been around for many decades and newer lossless compression algorithms are way better both in terms of speed (comparing with the same hardware) and compression ratio, e.g. LZ4 algorithm and Zstandard. It just doesn't make sense for the HW engineers and chip designers to optimize backwards for older software algorithm instead of optimizing forward for newer algorithms.
- tephra 6y agoThis depends on the prevalence on the different algorithms. If let's say gzip is 90% of the decompression load then it would make good sense.
- chungus_khan 6y agoOne such extremely common workload that does frequent gzip compression is HTTP servers because it is the standard method for compression.
- posix_me_less 6y agoIBM tech is for large old heavy-footed enterprises. I expect much GZIP, less Zstandard there.
- myself248 6y agoHow specific are those "gzip instructions", though? Are there newer algorithms that could benefit from some of the same acceleration primitives, even if they do higher-level things differently?
- wmf 6y agoThey're not instructions; it's a dedicated hardware unit. Most of the area appears to be devoted to finding matches so it should be possible to add other LZ-style algorithms without much additional area.
- giantrobot 6y agoGZip is a widely supported compression type for HTTP connections. It makes complete sense to have it be hardware accelerated.