6 ms·
With TLS?
by halbritt 8y ago
With TLS?
- pmalynin 8y agoAfter the asymmetric phase, TLS is pretty cheap on modern hardware. About 1500mb/s per core (at least with AES-NI). This is why all modern Macs do transparent encryption even if you didn’t explicitly turn on FDE.
- halbritt 8y agoSure, sure, but did you get that kind of throughput with kafka on java? Most folks (Confluent, for example) are still on java 8 which has some very severe limitations with regards to throughput.
- pmalynin 8y agoI'm not exactly sure what that has to do with the throughput of TLS, especially considering that it's 99% likely that its implemented in a native library (openssl, etc).
- halbritt 8y agoThe problem is that it's not implemented in the native library. That's why the throughput is so abysmal.
- anotherhue 8y agoIt also prevents use of the sendfile optimisation -- meaning a lot of Kafka perf disappears.
- halbritt 8y agoOne wonders why this was downvoted.
- deanmoriarty 8y agoJava TLS implementation is 99.9% implemented in Java itself (look at JSSE), openssl was ditched a long, long time ago. Not that I would necessarily expect the throughput to be significantly different than the native version considering the good performance of JITted Java, but was just commenting on your assumption.
- pmalynin 8y agoOh, I didn't know that, thanks for the clarification. I mostly work with C++, Python and Node in my day to day, so my assumptions are biased.
- 75dvtwin 8y agoTake a look at conscrypt https://github.com/google/conscrypt https://github.com/google/conscrypt According to reports into Jetty folks, using conscript for TLS, gives 10x improvements https://webtide.com/conscrypting-native-ssl-for-jetty/ https://webtide.com/conscrypting-native-ssl-for-jetty/ Note, that, from what I read, conscrypt is not compiled for any of the BSDs so if you are testing there, you will have to compile conscrypt yourself
- Twirrim 8y agoI've worked on services running on java 8 getting high throughput with AES-NI, maxing out 10Gig interfaces. Java is rarely a problem there. Where it is, you can even JNI out to openssl for your AES-NI encryption needs and spare yourself a little CPU power (we discovered that JNI isn't as big a performance hit as people would have you believe)
- halbritt 8y agoThere's a JIRA issue that's been open for three years for kafka to support openssl with JNI: https://issues.apache.org/jira/browse/KAFKA-2561 https://issues.apache.org/jira/browse/KAFKA-2561 Not sure it's going to get done at this point. I've only recently started using kafka and found the TLS issues to be somewhat surprising. AES-NI has been around for how long now? I speculate that this is because it's come out of the hadoop community and they haven't really had much luck implementing security features.
- Twirrim 8y agohttps://jaitechwriteups.blogspot.com/2017/10/kafka-with-openssl.html https://jaitechwriteups.blogspot.com/2017/10/kafka-with-open... It looks like with Java 9 the SSL stuff in Kafka saw a drastic speed boost.