5 ms·

Any benchmark that includes a JVM language always makes me look for a disclaimer that they allowed for proper JIT/warming time by running the benchmark a few ti

by dpratt 9y ago

Any benchmark that includes a JVM language always makes me look for a disclaimer that they allowed for proper JIT/warming time by running the benchmark a few times in the same VM instance before collecting the numbers. I may have missed it in this article, but I didn't see it. Yes, startup time is a concern with the JVM, but that's not what it's built for - it's meant for long-runnning repeatable code paths.

alayne 9y ago
He doesn't do warmups for JITted languages.
- peterashford 9y ago
  ...which means the measurements are meaningless. They will include the time to analyse the hot paths and then compile them.
  - igouy 9y ago
    They will include a few tenths of a second. Note the timings are seconds and tens-of-seconds.
  - deleted 9y ago
    [deleted]
mpweiher 9y ago
> that's not what it's built for There was nothing in the original design that said "only long running server processes". In fact, Java was originally aimed at set-top boxes, then web applets, then cross platform GUI apps. It was only after it failed in all of these that the server was discovered, despite the fact that "write once, run anywhere" is almost completely pointless when you control the hardware it runs on. The long startup times are a limitation of the implementation technology, not a principled design choice. See also: Jitterdämmerung ( http://blog.metaobject.com/2015/10/jitterdammerung.html http://blog.metaobject.com/2015/10/jitterdammerung.html )
Safety1stClyde 9y ago
A program written in C will execute faster if you run it multiple times, and slower if it has not been run recently. Presumably all of the benchmarks consist of many thousands of repetitions of the same process.
- azinman2 9y ago
  The difference is not as dramatic as with the JVM.
  - igouy 9y ago
    The difference is not as dramatic with tiny tiny programs that run for more than a few tenths of a second.
    - peterashford 9y ago
      Oh, yes it is. I've benchmarked code I've written in C++ vs Java before and allowing hotspot to optimise the code makes a huge difference
      - deleted 9y ago
        [deleted]
      - igouy 9y ago
        How "huge" on a 20 second run of nbody 50000000 ? http://benchmarksgame.alioth.debian.org/u64q/program.php?test=nbody&lang=java&id=4 http://benchmarksgame.alioth.debian.org/u64q/program.php?tes... The measurement shown is 21.54 secs. The average JMH shows is 23.367 ± 0.062.
igouy 9y ago
Any comments that "startup time is a concern with the JVM" always make me look for evidence that the person has made some attempt to investigate startup time for the specific circumstances they discuss. http://benchmarksgame.alioth.debian.org/sometimes-people-just-make-up-stuff.html#jvm-startup-time http://benchmarksgame.alioth.debian.org/sometimes-people-jus...

by dpratt 9y ago

alayne 9y ago

He doesn't do warmups for JITted languages.

peterashford 9y ago
...which means the measurements are meaningless. They will include the time to analyse the hot paths and then compile them.
- igouy 9y ago
  They will include a few tenths of a second. Note the timings are seconds and tens-of-seconds.
- deleted 9y ago
  [deleted]

mpweiher 9y ago

> that's not what it's built for There was nothing in the original design that said "only long running server processes". In fact, Java was originally aimed at set-top boxes, then web applets, then cross platform GUI apps. It was only after it failed in all of these that the server was discovered, despite the fact that "write once, run anywhere" is almost completely pointless when you control the hardware it runs on. The long startup times are a limitation of the implementation technology, not a principled design choice. See also: Jitterdämmerung ( http://blog.metaobject.com/2015/10/jitterdammerung.html http://blog.metaobject.com/2015/10/jitterdammerung.html )

Safety1stClyde 9y ago

A program written in C will execute faster if you run it multiple times, and slower if it has not been run recently. Presumably all of the benchmarks consist of many thousands of repetitions of the same process.

azinman2 9y ago
The difference is not as dramatic as with the JVM.
- igouy 9y ago
  The difference is not as dramatic with tiny tiny programs that run for more than a few tenths of a second.
  - peterashford 9y ago
    Oh, yes it is. I've benchmarked code I've written in C++ vs Java before and allowing hotspot to optimise the code makes a huge difference
    - deleted 9y ago
      [deleted]
    - igouy 9y ago
      How "huge" on a 20 second run of nbody 50000000 ? http://benchmarksgame.alioth.debian.org/u64q/program.php?test=nbody&lang=java&id=4 http://benchmarksgame.alioth.debian.org/u64q/program.php?tes... The measurement shown is 21.54 secs. The average JMH shows is 23.367 ± 0.062.

igouy 9y ago

Any comments that "startup time is a concern with the JVM" always make me look for evidence that the person has made some attempt to investigate startup time for the specific circumstances they discuss. http://benchmarksgame.alioth.debian.org/sometimes-people-just-make-up-stuff.html#jvm-startup-time http://benchmarksgame.alioth.debian.org/sometimes-people-jus...