4 ms·

The existing Linux system is useful for hardware that does less than 200MB/s, so you should be fine with HDDs. Cloudflare is optimising for SSDs. They don't t

The existing Linux system is useful for hardware that does less than 200MB/s, so you should be fine with HDDs. Cloudflare is optimising for SSDs. They don't talk about latency: all their crypto benchmarks measure throughput. Near the end they hint at response time for their overall cache system but there's no detailed discussion of latency issues. The takeaway for me is that I'm OK with what's currently in Linux for the HDDs I use for my backups but I'd probably lose out if I encrypted my main SSD with LUKS. At the end of the article they say that they're not going to upstream the patches as they are because they've only tested them with this one workload. I'd also be interested to see a benchmark comparing SW AES with FPU-saving + HW AES. Unfortunately their post does not include stats for how often their proxy falls into the HW or SW implementations. Whatever those numbers are, I'd expect FPU-saving + HW AES to be somewhere in the middle.

jlgaddis 7y ago
> The takeaway for me is that I'm OK with what's currently in Linux for the HDDs I use for my backups but I'd probably lose out if I encrypted my main SSD with LUKS. Yep, when building my latest workstation, I went with a pair of ("regular") SSDs (RAID1) for my data. Later, I decided to add an NVMe for the OS for the additional speed. I then went and encrypted all of the drives (via LUKS), however, which basically killed any additional performance I would've gotten from the NVMe drive. I would have been just fine as well off with only the SSDs and without the NVMe drive.
necovek 7y ago
You can easily achieve more than 200 MB/s with HDDs in RAID, but the bottleneck might be altogether different — I think it is an important distinction. While I applaud their wins, they have basically profiled the wrong thing, established the full overhead when disk speed/latency are basically removed, and only gone to actual production workload at the very end — in the worst case, their improvements could have been for naught, but they were "lucky" (not really, they were smart, but profiles did not really guide them — they just optimised the heck out of the system, but they could have been unlucky and not gain anything if the bottleneck was in a particular place unaffected by their code analysis). It's great that Cloudflare allows this kind of engineering to happen (investigative, explorative, and not necessarily RoI focused), but it's rare to find a company that does.
pmontra 7y ago
I'm using LUKS on my SSDs. I never benchmarked them but they are fast enough that I don't care. I'm working with VMs right now, creating and destroying them with VirtualBox (automated). Kind of a local EC2. The disks are two Samsung EVO 950 and 960, 1 TB each. They're in a laptop from 2014, a SATA III at 6 GB/s so I guess I'm already capped by the interface and the encryption overhead doesn't matter.
sweettea 7y ago
They talk about throughput, but in practice their testing regimen is actually testing latency. Dm-crypt's performance ceiling is pretty high if you consider throughput rather than latency, and I would expect the tradeoffs to decrease latency would decrease maximum throughput at least slightly (although I have not tested their patch).

jlgaddis 7y ago

> The takeaway for me is that I'm OK with what's currently in Linux for the HDDs I use for my backups but I'd probably lose out if I encrypted my main SSD with LUKS. Yep, when building my latest workstation, I went with a pair of ("regular") SSDs (RAID1) for my data. Later, I decided to add an NVMe for the OS for the additional speed. I then went and encrypted all of the drives (via LUKS), however, which basically killed any additional performance I would've gotten from the NVMe drive. I would have been just fine as well off with only the SSDs and without the NVMe drive.

necovek 7y ago

You can easily achieve more than 200 MB/s with HDDs in RAID, but the bottleneck might be altogether different — I think it is an important distinction. While I applaud their wins, they have basically profiled the wrong thing, established the full overhead when disk speed/latency are basically removed, and only gone to actual production workload at the very end — in the worst case, their improvements could have been for naught, but they were "lucky" (not really, they were smart, but profiles did not really guide them — they just optimised the heck out of the system, but they could have been unlucky and not gain anything if the bottleneck was in a particular place unaffected by their code analysis). It's great that Cloudflare allows this kind of engineering to happen (investigative, explorative, and not necessarily RoI focused), but it's rare to find a company that does.

pmontra 7y ago

I'm using LUKS on my SSDs. I never benchmarked them but they are fast enough that I don't care. I'm working with VMs right now, creating and destroying them with VirtualBox (automated). Kind of a local EC2. The disks are two Samsung EVO 950 and 960, 1 TB each. They're in a laptop from 2014, a SATA III at 6 GB/s so I guess I'm already capped by the interface and the encryption overhead doesn't matter.

sweettea 7y ago

They talk about throughput, but in practice their testing regimen is actually testing latency. Dm-crypt's performance ceiling is pretty high if you consider throughput rather than latency, and I would expect the tradeoffs to decrease latency would decrease maximum throughput at least slightly (although I have not tested their patch).