5 ms·

It appears the design of KTLS is intended to minimize the complexity necessary inside the kernel; the TLS handshake is handled in user space and then the TLS st

It appears the design of KTLS is intended to minimize the complexity necessary inside the kernel; the TLS handshake is handled in user space and then the TLS state is transcribed to a KTLS file descriptor which handles the traffic thereafter. This means the very complex public key part of TLS is user space. The kernel isn't involved with certificates etc. The kernel handles the symmetric cryptography once the user space handshake is complete. So yes, the net attack surface inside the kernel increases, but it isn't as though they are embedding some OpenSSL equivalent into the kernel. The paper reports a relatively small performance improvements. But I suppose if you're Facebook (they and RedHat implemented this) and performance improvements are measured in millions of dollars you care a lot about 7% less CPU utilization.

microcolonel 9y ago
The performance gain is in latency consistency, it seems [0] though this chart is silly, since I'm guessing the X axis is time, and you can't chart a centile, that's ridiculous.. Anyway, even the throughput improvement could be worth it, but I think the main improvement is in latency. https://twitter.com/tgraf__/status/904475786622640128 https://twitter.com/tgraf__/status/904475786622640128
- vbernat 9y ago
  You can chart a centile. Each point can be the centile of X measures.
- petters 9y ago
  > you can't chart a centile, Why? If you have servers with millions of qps, the graph would show the 99% latency over, say, a second or ten. I think this graph is pretty standard? But I'm happy for suggestions of better ones.
  - user5994461 9y ago
    There are constant variations in the environment that are more significant than the tiny effect you are trying to measure.
    - jimktrains2 9y ago
      There are many statical tests that can be used in these situations and have been honest over the years for use in scientific research. For example, chi-squared and if you can quantify the error distribution and it's normal, student's t-test.
  - microcolonel 9y ago
    Well, more accurately you could chart a centile, but it would hide the very latency patterns you're looking for. The outliers in a given time bucket are the important data. If you're looking at one bucket, it could be useful to do a centile (though 99th is not really high enough to be meaningful), but if you're charting centile buckets, then you're missing the real latency spikes. For all we know, KCM/KTLS generates the highest peak latency, but fewer times per bucket. That difference would completely change the interpretation of these data. If userspace data frame handling produces higher 99th percentile latency, that does not tell us anything about its maximum latency. Further, if we're looking at the 99th percentile, 1/100 data frames have latency higher than that percentile, this will happen hundreds, thousands, or millions of times per second depending on your interface bandwidth and typical frame size!
phkahler 9y ago
>> The paper reports a relatively small performance improvements. But I suppose if you're Facebook (they and RedHat implemented this) and performance improvements are measured in millions of dollars you care a lot about 7% less CPU utilization. You think 7 percent is a small performance improvement? IMHO it's rather large. I take the time for much smaller than that. To understand why, let me ask a question do you think doubling performance is a good thing? In other words a 100 percent increase in throughput or a 50 percent reduction in execution time. I hope your answer is yes because a lot of people think so. So what's 7 percent? Well imagine there are 7 different areas of your code that could be improved with each one contributing 7 percent. That's 49 percent or about what we'd call a significant improvement. Fixing up a number of single digit things adds up. But there is a more sinister side to it. Suppose you wanted to let those 7 little things go, and look for the big fish - the one that's going to get you 50 percent all in one shot. Guess what? It doesn't exist. Why? Because these little things add up to about half of your time, so to find the big one would imply taking the execution time for the rest of the code down to zero, because that's the only place there's 50 percent left. Unless you're doing something really poorly, there isn't a big win to be had. Performance comes from compounding the effects of many small optimizations. If just moving TLS from the application to the kernel gives 7 percent - presumably by eliminating overhead - that's a good thing. Even if it's just 7 percent of time in communication which is only a portion of application performance, it's still a good thing. --- note, this is a comment about performance and says nothing about tradeoffs in security by putting TLS in kernel ---
- deleted 9y ago
  [deleted]
- topspin 9y ago
  >> You think 7 percent is a small performance improvement? I think a 7% reduction in CPU utilization for TLS is small given the vast bulk of systems that spend most of their time IO bound on network interfaces and/or storage and will see little to no improvement in throughput. I allowed that Facebook may see this as an important improvement given their scale; it could be worth millions of dollars in power and/or hardware. >> that's a good thing ... it's still a good thing. Sure, it's good. I didn't characterize anything as "bad," merely "small."

microcolonel 9y ago

The performance gain is in latency consistency, it seems [0] though this chart is silly, since I'm guessing the X axis is time, and you can't chart a centile, that's ridiculous.. Anyway, even the throughput improvement could be worth it, but I think the main improvement is in latency. https://twitter.com/tgraf__/status/904475786622640128 https://twitter.com/tgraf__/status/904475786622640128

vbernat 9y ago
You can chart a centile. Each point can be the centile of X measures.
petters 9y ago
> you can't chart a centile, Why? If you have servers with millions of qps, the graph would show the 99% latency over, say, a second or ten. I think this graph is pretty standard? But I'm happy for suggestions of better ones.
- user5994461 9y ago
  There are constant variations in the environment that are more significant than the tiny effect you are trying to measure.
  - jimktrains2 9y ago
    There are many statical tests that can be used in these situations and have been honest over the years for use in scientific research. For example, chi-squared and if you can quantify the error distribution and it's normal, student's t-test.
- microcolonel 9y ago
  Well, more accurately you could chart a centile, but it would hide the very latency patterns you're looking for. The outliers in a given time bucket are the important data. If you're looking at one bucket, it could be useful to do a centile (though 99th is not really high enough to be meaningful), but if you're charting centile buckets, then you're missing the real latency spikes. For all we know, KCM/KTLS generates the highest peak latency, but fewer times per bucket. That difference would completely change the interpretation of these data. If userspace data frame handling produces higher 99th percentile latency, that does not tell us anything about its maximum latency. Further, if we're looking at the 99th percentile, 1/100 data frames have latency higher than that percentile, this will happen hundreds, thousands, or millions of times per second depending on your interface bandwidth and typical frame size!

phkahler 9y ago

>> The paper reports a relatively small performance improvements. But I suppose if you're Facebook (they and RedHat implemented this) and performance improvements are measured in millions of dollars you care a lot about 7% less CPU utilization. You think 7 percent is a small performance improvement? IMHO it's rather large. I take the time for much smaller than that. To understand why, let me ask a question do you think doubling performance is a good thing? In other words a 100 percent increase in throughput or a 50 percent reduction in execution time. I hope your answer is yes because a lot of people think so. So what's 7 percent? Well imagine there are 7 different areas of your code that could be improved with each one contributing 7 percent. That's 49 percent or about what we'd call a significant improvement. Fixing up a number of single digit things adds up. But there is a more sinister side to it. Suppose you wanted to let those 7 little things go, and look for the big fish - the one that's going to get you 50 percent all in one shot. Guess what? It doesn't exist. Why? Because these little things add up to about half of your time, so to find the big one would imply taking the execution time for the rest of the code down to zero, because that's the only place there's 50 percent left. Unless you're doing something really poorly, there isn't a big win to be had. Performance comes from compounding the effects of many small optimizations. If just moving TLS from the application to the kernel gives 7 percent - presumably by eliminating overhead - that's a good thing. Even if it's just 7 percent of time in communication which is only a portion of application performance, it's still a good thing. --- note, this is a comment about performance and says nothing about tradeoffs in security by putting TLS in kernel ---

deleted 9y ago
[deleted]
topspin 9y ago
>> You think 7 percent is a small performance improvement? I think a 7% reduction in CPU utilization for TLS is small given the vast bulk of systems that spend most of their time IO bound on network interfaces and/or storage and will see little to no improvement in throughput. I allowed that Facebook may see this as an important improvement given their scale; it could be worth millions of dollars in power and/or hardware. >> that's a good thing ... it's still a good thing. Sure, it's good. I didn't characterize anything as "bad," merely "small."