6 ms·

How performant is that setup, out of curiosity? I've always assumed the tradeoff here is a safe language is slower, and slow crypto is no bueno.

bhauer 12y ago
We (TechEmpower) have run preliminary tests of JSSE in consideration of a future round of our framework benchmarks project. Although we do not yet have any SSL tests in our project, our preliminary findings were that JSSE was considerably higher-performance than we had been lead to believe by popular opinion. I don't have the data in front of me, but using JSSE in lieu of OpenSSL did not affect request-per-second results sufficiently to make me worry about using JSSE. Since then, my chief concern with JSSE has been simply not knowing how solid it is from a correctness of the algorithms perspective, but that is something I'm not well versed in. In other words, my concern was just one of uncertainty. Provided a credible security analysis suggests JSSE is just as secure (if not moreso) than alternatives in unsafe languages, I will be confident deploying future apps on JSSE.
frankchn 12y ago
May I ask why not? I would personally prefer slower (if safer) crypto over fast but potentially vulnerable crypto.
mike_hearn 12y ago
The website in question doesn't get much traffic at the moment, so performance is not an issue. Modern JVMs can use AES-NI hardware instructions when available, in some cases at least, so I'd imagine it's not too awful. But I honestly don't know. Benchmarks would be interesting.
Someone1234 12y ago
At runtime modern Java is around 10x slower than modern C++ (assuming both are fully optimised and the benchmark is relatively unbiased). That isn't significant on modern hardware relative to other bottlenecks (network, IO, etc). Plus people keep adding additional security layers between C/C++ and the CPU which eat away at some of its advantages (e.g. Docker containers, virtual machines, exploit detection libraries, etc). Java speed matters in certain situations and for certain tasks. For example I wouldn't rewrite an SQL database into Java since performance could definitely be impactful there. But realistically CPU times are such a tiny minority of latency that it stopped being relevant a very long time ago.
- tshtf 12y ago
  [Citation required]
  - Someone1234 12y ago
    mike_hearn beat me to it. See his post for a citation (and some other numbers).
    - afsina 12y ago
      And you stood corrected.
- mike_hearn 12y ago
  At runtime modern Java is around 10x slower than modern C++ (assuming both are fully optimised and the benchmark is relatively unbiased). I don't think you can pick a single number like that, it depends very much on what the code is doing. You probably got this 10x figure from a Google paper published a few years ago (https://days2011.scala-lang.org/sites/days2011/files/ws3-1-Hundt.pdf https://days2011.scala-lang.org/sites/days2011/files/ws3-1-H...). Redux: the default out of the box 32 bit JVM (in 2011) was 12x slower than the same algorithm implemented in Java. But with a few simple GC flag changes, it became 3.7x slower. And then the Java version had a few further simple optimisations applied to the code and it became as fast as the original C++ version. Meanwhile the C++ version was optimised again and became around 3x to 5x faster, but that version relied heavily on Google proprietary data structure code and could not be open sourced (!). So for most programmers on most projects, it seems likely to be a wash. Meanwhile an alternative benchmark that reimplemented a non-trivial C++ program in Java found it became 1.09 to 1.5 times slower, but that was with Java 6 which is now two generations out of date: http://www.best-of-robotics.org/pages/publications/gherardi12java.pdf http://www.best-of-robotics.org/pages/publications/gherardi1... It would be interesting to get more recent benchmarks with the latest JVMs and C++ compilers.
  - TheLoneWolfling 12y ago
    As you say, it depends very much on what the code is doing. In particular, anything requiring manipulating unsigned values all over the place will be a lot slower. Also, see this[1] for talk about JGit. An actual project with direct comparisons. [1] http://marc.info/?l=git&m=124111702609723 http://marc.info/?l=git&m=124111702609723
umanwizard 12y ago
Does it matter? Web servers are normally not CPU-bound.
- mike_hearn 12y ago
  One of the reasons websites have been slow to roll out SSL for everything is CPU load, so it's a legitimate concern for a high traffic website. However, most of the ways SSL was speeded up involved algorithmic / protocol level improvements and not micro-optimisations of the code. So I'm not sure that a safe language has a permanent disadvantage here.
  - bkeroack 12y ago
    This was possibly true 10-20 years ago (SSL is that old). It is no longer reasonable and is usually just an excuse. That said, in my experience, native SSL/TLS within IIS is incredibly slow/inefficient. Running on 2012R2 (so a modern version), terminating SSL at IIS resulted in me being able to trivially peg the server's CPU using Apache Ab on my laptop. So clearly you need a good implementation.
  - lazyjones 12y ago
    Another reason was/is that handshakes add noticeable latency (because several packets are sent in both directions), so web browsing experience and consequently, retention and conversion rates will suffer. The worst barrier was probably the cost and hassle of obtaining browser-accepted certificates though.
pjmlp 12y ago
> I've always assumed the tradeoff here is a safe language is slower, and slow crypto is no bueno. Back in the day Extended Pascal, Modula-2,Mesa... and C compilers had similar code quality. If today compilers for safer languages are slower than C compilers, it is mainly a consequence of compiler vendors focusing in improving C optimizers.

bhauer 12y ago

We (TechEmpower) have run preliminary tests of JSSE in consideration of a future round of our framework benchmarks project. Although we do not yet have any SSL tests in our project, our preliminary findings were that JSSE was considerably higher-performance than we had been lead to believe by popular opinion. I don't have the data in front of me, but using JSSE in lieu of OpenSSL did not affect request-per-second results sufficiently to make me worry about using JSSE. Since then, my chief concern with JSSE has been simply not knowing how solid it is from a correctness of the algorithms perspective, but that is something I'm not well versed in. In other words, my concern was just one of uncertainty. Provided a credible security analysis suggests JSSE is just as secure (if not moreso) than alternatives in unsafe languages, I will be confident deploying future apps on JSSE.

frankchn 12y ago

May I ask why not? I would personally prefer slower (if safer) crypto over fast but potentially vulnerable crypto.

mike_hearn 12y ago

The website in question doesn't get much traffic at the moment, so performance is not an issue. Modern JVMs can use AES-NI hardware instructions when available, in some cases at least, so I'd imagine it's not too awful. But I honestly don't know. Benchmarks would be interesting.

Someone1234 12y ago

At runtime modern Java is around 10x slower than modern C++ (assuming both are fully optimised and the benchmark is relatively unbiased). That isn't significant on modern hardware relative to other bottlenecks (network, IO, etc). Plus people keep adding additional security layers between C/C++ and the CPU which eat away at some of its advantages (e.g. Docker containers, virtual machines, exploit detection libraries, etc). Java speed matters in certain situations and for certain tasks. For example I wouldn't rewrite an SQL database into Java since performance could definitely be impactful there. But realistically CPU times are such a tiny minority of latency that it stopped being relevant a very long time ago.

tshtf 12y ago
[Citation required]
- Someone1234 12y ago
  mike_hearn beat me to it. See his post for a citation (and some other numbers).
  - afsina 12y ago
    And you stood corrected.
mike_hearn 12y ago
At runtime modern Java is around 10x slower than modern C++ (assuming both are fully optimised and the benchmark is relatively unbiased). I don't think you can pick a single number like that, it depends very much on what the code is doing. You probably got this 10x figure from a Google paper published a few years ago (https://days2011.scala-lang.org/sites/days2011/files/ws3-1-Hundt.pdf https://days2011.scala-lang.org/sites/days2011/files/ws3-1-H...). Redux: the default out of the box 32 bit JVM (in 2011) was 12x slower than the same algorithm implemented in Java. But with a few simple GC flag changes, it became 3.7x slower. And then the Java version had a few further simple optimisations applied to the code and it became as fast as the original C++ version. Meanwhile the C++ version was optimised again and became around 3x to 5x faster, but that version relied heavily on Google proprietary data structure code and could not be open sourced (!). So for most programmers on most projects, it seems likely to be a wash. Meanwhile an alternative benchmark that reimplemented a non-trivial C++ program in Java found it became 1.09 to 1.5 times slower, but that was with Java 6 which is now two generations out of date: http://www.best-of-robotics.org/pages/publications/gherardi12java.pdf http://www.best-of-robotics.org/pages/publications/gherardi1... It would be interesting to get more recent benchmarks with the latest JVMs and C++ compilers.
- TheLoneWolfling 12y ago
  As you say, it depends very much on what the code is doing. In particular, anything requiring manipulating unsigned values all over the place will be a lot slower. Also, see this[1] for talk about JGit. An actual project with direct comparisons. [1] http://marc.info/?l=git&m=124111702609723 http://marc.info/?l=git&m=124111702609723

umanwizard 12y ago

Does it matter? Web servers are normally not CPU-bound.

mike_hearn 12y ago
One of the reasons websites have been slow to roll out SSL for everything is CPU load, so it's a legitimate concern for a high traffic website. However, most of the ways SSL was speeded up involved algorithmic / protocol level improvements and not micro-optimisations of the code. So I'm not sure that a safe language has a permanent disadvantage here.
- bkeroack 12y ago
  This was possibly true 10-20 years ago (SSL is that old). It is no longer reasonable and is usually just an excuse. That said, in my experience, native SSL/TLS within IIS is incredibly slow/inefficient. Running on 2012R2 (so a modern version), terminating SSL at IIS resulted in me being able to trivially peg the server's CPU using Apache Ab on my laptop. So clearly you need a good implementation.
- lazyjones 12y ago
  Another reason was/is that handshakes add noticeable latency (because several packets are sent in both directions), so web browsing experience and consequently, retention and conversion rates will suffer. The worst barrier was probably the cost and hassle of obtaining browser-accepted certificates though.

pjmlp 12y ago

> I've always assumed the tradeoff here is a safe language is slower, and slow crypto is no bueno. Back in the day Extended Pascal, Modula-2,Mesa... and C compilers had similar code quality. If today compilers for safer languages are slower than C compilers, it is mainly a consequence of compiler vendors focusing in improving C optimizers.