6 ms·

Has anybody tried the ahead of time compilation? Does it significantly reduce script startup time?

aardvark179 11y ago
You mean Direct Linkage? To be clear the difference is not whether functions are compiled ahead of time but whether function calls go through the standard indirections or are turned into static method calls more amenable to JIT optimisation. I haven't tried it but would expect it makes more of a difference to peak performance than startup time. The VM will still need to do those initial method lookups, and the normal var and Fn parts will have become very hot very quickly. From personal experience class io, verification, method linkage, and reflection play a big role in startup of jvm languages (along with call site target setting if you're using invokeDynamic bytecodes), but removing indirection helps for long running processes.
- puredanger 11y ago
  Direct linking does (indirectly) affect startup time. Specifically, using direct linking means that many vars in the function class initializer are no longer needed. There is some new logic that avoids adding unused var initialization in the compiled class. The result is a reduction in class size and static initializer size, which reduces classloading time (a significant part of startup costs).
- pjmlp 11y ago
  > From personal experience class io, verification, method linkage, and reflection play a big role in startup of jvm languages (along with call site target setting if you're using invokeDynamic bytecodes), but removing indirection helps for long running processes. Yes, but Clojure itself is the one to blame for slow startup as it as been proven multiple times by community members.
mej10 11y ago
Probably not enough to matter. Even if it halved the startup time it would still be too slow for the tasks it is too slow for. Note: Clojure itself performs fine, I am just talking about the startup time. That being said, I use Clojure for scripting tasks frequently and it isn't so bad. It just isn't as awesome as it could be. POTENTIAL SOLUTION: I think for script startup time someone is going to have to bite the bullet and write some kind of nREPL client in a language with no startup time (Go, C, etc). Then you could just send code to be evaluated to already running environments. You could use boot.pods to isolate the Clojure runtimes. Seems straightforward-ish.
- georgewsinger 11y ago
  You can also target nodejs with ClojureScript, which leads to blazing fast scripts/startup time. See as an example: https://github.com/georgewsinger/yaml-table https://github.com/georgewsinger/yaml-table This script takes about 100 ms on my computer and about 2.25 seconds (!) to run via pure clojure on the JVM.
- georgewsinger 11y ago
  You can also target nodejs with ClojureScript, which leads to blazing fast scripts/startup time. See as an example: https://github.com/georgewsinger/yaml-table https://github.com/georgewsinger/yaml-table This ClojureScript program takes about 100 ms to run on my computer, while the corresponding Clojure version takes about 2.25 seconds (!) to run via the JVM.
- ziggy1337 11y ago
  I think https://github.com/technomancy/grenchman https://github.com/technomancy/grenchman is exactly what you're looking for.
asimjalis 11y ago
Here is the reason for the slow startup time. This is also why AOT compilation does not solve it. http://martinsprogrammingblog.blogspot.de/2012/02/why-is-clojure-so-slow.html http://martinsprogrammingblog.blogspot.de/2012/02/why-is-clo... “[It] spends 95% of the startup-time loading the clojure.core namespace (the clojure.lang.RT class in particular) and filling out all the metadata/docstrings etc for the methods. This process stresses the GC quite a bit, some 130k objects are allocated and 90k free-d during multiple invokes of the GC (3-6 times), the building up of meta data is one big source of this massive object churn.” See also: http://blog.ndk.io/clojure-bootstrapping.html http://blog.ndk.io/clojure-bootstrapping.html
- puredanger 11y ago
  Keep in mind the timings in the first blog are 4 years old and the second blog are 2 years old. Both Java and Clojure have made improvements and released new versions.A simple hello world uberjar with the default settings is about 0.775 s for me. Also, Clojure can be AOT compiled with direct-linking and elided metadata which alleviate some of the broader concerns (they're unlikely to affect a hello world type app though). See: http://clojure.org/reference/compilation http://clojure.org/reference/compilation Var class loading is still a concern, so there is of course still room for improvement. Experiments in this area are ongoing. Delayed var loading has the significant downside of making every var call thereafter slower (as the code needs a guard to check whether the class has been loaded or not). invokedynamic shows some promise is getting the best of both worlds there.

You mean Direct Linkage? To be clear the difference is not whether functions are compiled ahead of time but whether function calls go through the standard indirections or are turned into static method calls more amenable to JIT optimisation. I haven't tried it but would expect it makes more of a difference to peak performance than startup time. The VM will still need to do those initial method lookups, and the normal var and Fn parts will have become very hot very quickly. From personal experience class io, verification, method linkage, and reflection play a big role in startup of jvm languages (along with call site target setting if you're using invokeDynamic bytecodes), but removing indirection helps for long running processes.

puredanger 11y ago
Direct linking does (indirectly) affect startup time. Specifically, using direct linking means that many vars in the function class initializer are no longer needed. There is some new logic that avoids adding unused var initialization in the compiled class. The result is a reduction in class size and static initializer size, which reduces classloading time (a significant part of startup costs).
pjmlp 11y ago
> From personal experience class io, verification, method linkage, and reflection play a big role in startup of jvm languages (along with call site target setting if you're using invokeDynamic bytecodes), but removing indirection helps for long running processes. Yes, but Clojure itself is the one to blame for slow startup as it as been proven multiple times by community members.

mej10 11y ago

Probably not enough to matter. Even if it halved the startup time it would still be too slow for the tasks it is too slow for. Note: Clojure itself performs fine, I am just talking about the startup time. That being said, I use Clojure for scripting tasks frequently and it isn't so bad. It just isn't as awesome as it could be. POTENTIAL SOLUTION: I think for script startup time someone is going to have to bite the bullet and write some kind of nREPL client in a language with no startup time (Go, C, etc). Then you could just send code to be evaluated to already running environments. You could use boot.pods to isolate the Clojure runtimes. Seems straightforward-ish.

georgewsinger 11y ago
You can also target nodejs with ClojureScript, which leads to blazing fast scripts/startup time. See as an example: https://github.com/georgewsinger/yaml-table https://github.com/georgewsinger/yaml-table This script takes about 100 ms on my computer and about 2.25 seconds (!) to run via pure clojure on the JVM.
georgewsinger 11y ago
You can also target nodejs with ClojureScript, which leads to blazing fast scripts/startup time. See as an example: https://github.com/georgewsinger/yaml-table https://github.com/georgewsinger/yaml-table This ClojureScript program takes about 100 ms to run on my computer, while the corresponding Clojure version takes about 2.25 seconds (!) to run via the JVM.
ziggy1337 11y ago
I think https://github.com/technomancy/grenchman https://github.com/technomancy/grenchman is exactly what you're looking for.

asimjalis 11y ago

Here is the reason for the slow startup time. This is also why AOT compilation does not solve it. http://martinsprogrammingblog.blogspot.de/2012/02/why-is-clojure-so-slow.html http://martinsprogrammingblog.blogspot.de/2012/02/why-is-clo... “[It] spends 95% of the startup-time loading the clojure.core namespace (the clojure.lang.RT class in particular) and filling out all the metadata/docstrings etc for the methods. This process stresses the GC quite a bit, some 130k objects are allocated and 90k free-d during multiple invokes of the GC (3-6 times), the building up of meta data is one big source of this massive object churn.” See also: http://blog.ndk.io/clojure-bootstrapping.html http://blog.ndk.io/clojure-bootstrapping.html

puredanger 11y ago
Keep in mind the timings in the first blog are 4 years old and the second blog are 2 years old. Both Java and Clojure have made improvements and released new versions.A simple hello world uberjar with the default settings is about 0.775 s for me. Also, Clojure can be AOT compiled with direct-linking and elided metadata which alleviate some of the broader concerns (they're unlikely to affect a hello world type app though). See: http://clojure.org/reference/compilation http://clojure.org/reference/compilation Var class loading is still a concern, so there is of course still room for improvement. Experiments in this area are ongoing. Delayed var loading has the significant downside of making every var call thereafter slower (as the code needs a guard to check whether the class has been loaded or not). invokedynamic shows some promise is getting the best of both worlds there.