3 ms·

The latency comparison seems a bit crappy to me. Comparing the lowest points in two series of data is very error prone.

by glastra 11y ago

bbjnicklin 11y ago
Glastra. Maybe this data is a bit better for you, direct from fio. TLSv1.2: ecdhe-rsa-aes128-gcm-sha256 QD1: (groupid=0, jobs=1): err= 0: pid=2140: Sat Mar 5 14:59:49 2016 read : io=2308.5MB, bw=78791KB/s, iops=19697, runt= 30001msec slat (usec): min=2, max=21, avg= 2.42, stdev= 0.50 clat (usec): min=39, max=56179, avg=47.66, stdev=126.45 lat (usec): min=46, max=56182, avg=50.14, stdev=126.45 clat percentiles (usec): | 1.00th=[ 46], 5.00th=[ 46], 10.00th=[ 46], 20.00th=[ 47], | 30.00th=[ 47], 40.00th=[ 47], 50.00th=[ 47], 60.00th=[ 47], | 70.00th=[ 48], 80.00th=[ 48], 90.00th=[ 49], 95.00th=[ 49], | 99.00th=[ 51], 99.50th=[ 51], 99.90th=[ 53], 99.95th=[ 56], IPsec: aes128-gcm96 QD1: (groupid=0, jobs=1): err= 0: pid=2442: Sat Mar 5 15:27:04 2016 read : io=1902.6MB, bw=64938KB/s, iops=16234, runt= 30001msec slat (usec): min=2, max=29, avg= 2.39, stdev= 0.52 clat (usec): min=52, max=57367, avg=58.51, stdev=140.69 lat (usec): min=57, max=57369, avg=60.97, stdev=140.69 clat percentiles (usec): | 1.00th=[ 56], 5.00th=[ 56], 10.00th=[ 57], 20.00th=[ 57], | 30.00th=[ 57], 40.00th=[ 57], 50.00th=[ 58], 60.00th=[ 58], | 70.00th=[ 58], 80.00th=[ 59], 90.00th=[ 60], 95.00th=[ 61], | 99.00th=[ 63], 99.50th=[ 65], 99.90th=[ 78], 99.95th=[ 82],
- feld 11y ago
  latency is especially important for small I/Os, but there are a ton of things that can be done to improve latency in general at an OS and network level
  - bbjnicklin 11y ago
    Feld, you are totally right. In fact, there has been discussion in the Linux community about improving IPsec performance. However, as the post states, the difficulty in doing so is the API contract in the kernel. That said, even with a perfectly optimized IPsec stack, having to process each packet individually will always be slower than processing a logical record that is a multiple of packet size.

The latency comparison seems a bit crappy to me. Comparing the lowest points in two series of data is very error prone.

by glastra 11y ago

The latency comparison seems a bit crappy to me. Comparing the lowest points in two series of data is very error prone.

Glastra. Maybe this data is a bit better for you, direct from fio. TLSv1.2: ecdhe-rsa-aes128-gcm-sha256 QD1: (groupid=0, jobs=1): err= 0: pid=2140: Sat Mar 5 14:59:49 2016 read : io=2308.5MB, bw=78791KB/s, iops=19697, runt= 30001msec slat (usec): min=2, max=21, avg= 2.42, stdev= 0.50 clat (usec): min=39, max=56179, avg=47.66, stdev=126.45 lat (usec): min=46, max=56182, avg=50.14, stdev=126.45 clat percentiles (usec): | 1.00th=[ 46], 5.00th=[ 46], 10.00th=[ 46], 20.00th=[ 47], | 30.00th=[ 47], 40.00th=[ 47], 50.00th=[ 47], 60.00th=[ 47], | 70.00th=[ 48], 80.00th=[ 48], 90.00th=[ 49], 95.00th=[ 49], | 99.00th=[ 51], 99.50th=[ 51], 99.90th=[ 53], 99.95th=[ 56], IPsec: aes128-gcm96 QD1: (groupid=0, jobs=1): err= 0: pid=2442: Sat Mar 5 15:27:04 2016 read : io=1902.6MB, bw=64938KB/s, iops=16234, runt= 30001msec slat (usec): min=2, max=29, avg= 2.39, stdev= 0.52 clat (usec): min=52, max=57367, avg=58.51, stdev=140.69 lat (usec): min=57, max=57369, avg=60.97, stdev=140.69 clat percentiles (usec): | 1.00th=[ 56], 5.00th=[ 56], 10.00th=[ 57], 20.00th=[ 57], | 30.00th=[ 57], 40.00th=[ 57], 50.00th=[ 58], 60.00th=[ 58], | 70.00th=[ 58], 80.00th=[ 59], 90.00th=[ 60], 95.00th=[ 61], | 99.00th=[ 63], 99.50th=[ 65], 99.90th=[ 78], 99.95th=[ 82],

feld 11y ago
latency is especially important for small I/Os, but there are a ton of things that can be done to improve latency in general at an OS and network level
- bbjnicklin 11y ago
  Feld, you are totally right. In fact, there has been discussion in the Linux community about improving IPsec performance. However, as the post states, the difficulty in doing so is the API contract in the kernel. That said, even with a perfectly optimized IPsec stack, having to process each packet individually will always be slower than processing a logical record that is a multiple of packet size.