7 ms·
Fast JVM launching without the hassle of persistent JVMs
- sandGorgon 14y agotime ./drip -jar /home/user/research/jruby-complete-1.6.0.RC3.jar --1.9 -e 'a=1;puts a' 1 ./drip -jar /home/user/research/jruby-complete-1.6.0.RC3.jar --1.9 -e 0.03s user 0.03s system 4% cpu 1.282 total time java -jar /home/user/research/jruby-complete-1.6.0.RC3.jar --1.9 -e 'a=1;puts a' 1 java -jar /home/user/research/jruby-complete-1.6.0.RC3.jar --1.9 -e 3.65s user 0.12s system 183% cpu 2.056 total Interesting!! if it runs rails effectively, this could be awesome for jruby.
- mickeyben 14y agoexactly what I was thinking! I recently moved one of my project to jruby and the startup time is the only thing I'm complaining about.
- riffraff 14y agoI wonder, have you tried `-noverify`and `-client` ? (I am not sure if `-noverify` is compatible with jruby but I seem to remember so)
- tlogan 14y agoWas anybody able to run jruby on rails with drip?
- z3phyr 14y agoHappier Clojure hacking days ahead :)
- dcolgan 14y agoThis seems like a really clever solution. I remember trying to set up nailgun a while ago back when I was experimenting with Clojure because the startup time for running a program was so long, but I could never quite get it to work. I missed being able to run python myprog.py and getting instant feedback.
- briggers 14y agoPersevere! Dynamic programming is wonderful. I prefer swank over nailgun, even with Vim.
- pjmlp 14y agoWhat about just using AOT compilation?
- gtrak 14y agoJVM startup overhead, not clojure runtime overhead. That's still there.
- pjmlp 14y agoWhat startup overhead? I meant AOT to native code.
- MichaelGG 14y agoAOT to Native code just removes the JIT time. Any overhead of actually running the Java runtime and running initialization code is still there.
- pjmlp 14y agoWhat?!
- gtrak 14y agoAOT already means something to clojurists. It means removing the extra runtime compilation phase of .clj into classfiles. The JIT is what the JVM does, which still occurs just like before.
- pjmlp 14y agoUnless the bytecode gets translated to native code ahead of time (AOT). Although OpenJDK/Oracle JVM don't offer this option, other vendors do.
- bickfordb 14y agoMost of the Clojure app startup time (~1.2s on my machine) is the Clojure runtime, not the JVM (usually less than 100ms).
- wiradikusuma 14y agoSweet! I immediately updated runner script for Scala and Groovy to use it! EDIT: Hmm, I encountered this: "Could not connect to compilation daemon after 300 attempts." but re-running it ("scala") works.
- 6ren 14y agoSo obvious, trade-off space for time, yet I wouldn't have thought of it... I mean, I've thought about this problem, written a persistent JVM solution, and didn't think of it. Memory is cheaper than my intuition realises. I wonder how many other "obvious" solutions I'm missing like this? EDIT for the code I tried, user time is almost 3 times faster, but real time is only around 10% better... I don't understand linux well enough to know why - anyone care to explain please? EDIT Yes, drip had already run. (I picked typical times from about 10 runs each). $ time java... real 0m1.466s user 0m1.216s sys 0m0.180s $ time drip... real 0m1.378s user 0m0.412s sys 0m0.260s BTW: For server-like workloads, an advantage of a persistent JVM is that it gets dramatically faster over repeated runs of the same code, as it improves its hotspot-style adaptive optimisations. I really like his quickstart "standalone" installation. WARNING "drip kill" crashed my system. The kill functions are kill_jvms and kill_jvm (https://github.com/flatland/drip/blob/develop/bin/drip https://github.com/flatland/drip/blob/develop/bin/drip). I'm using an older ubuntu 10.04 LTS.
- bretthoerner 14y agoDid you do two exact same drip runs in a row? It has to hash and preload a JVM with the same classpath setup to work. $ time java -jar clojure-1.3.0.jar add_1_and_1.clj real 0m0.690s user 0m0.827s sys 0m0.030s $ time drip -jar clojure-1.3.0.jar add_1_and_1.clj real 0m0.879s user 0m1.020s sys 0m0.040s $ time drip -jar clojure-1.3.0.jar add_1_and_1.clj real 0m0.177s user 0m0.033s sys 0m0.020s $ time drip -jar clojure-1.3.0.jar add_1_and_1.clj real 0m0.176s user 0m0.050s sys 0m0.017s
- yason 14y agoHasn't anyone ported zygote over to desktop Linux/Windows? You just keep the preconfigured jvm process running and fork it indefinitely for each new process. You'd still suffer some overhead depending on each application but that's just expected anyway. The jvm startup overhead isn't.
- ansible 14y agoThe Dalvik VM does run on x86 now... You'd just need to convert all your .jars and that's it. I haven't heard of any other incompatibilities with standard Java. Edit: Is there a port of Dalvik to amd64? A cursory search did not find one.
- eropple 14y agoI don't know if such a port exists, but I do know that I wouldn't want one. Dalvik's performance characteristics are scary-bad and there are plenty of incompatibilities with Java libraries that do any sort of weaving or code generation (so you still can't use Groovy, Rhino, etc.).
- ansible 14y ago... and there are plenty of incompatibilities with Java libraries that do any sort of weaving or code generation ... Ah, good to know. I've mostly just had to bring in things like a database library, and in one case LuaJ. None of those did need to do bytecode generation.
- eropple 14y agoYeah, for that you'll be fine, but take Rhino for example - Rhino offers, through LiveConnect, the ability to create JavaScript objects, at runtime, that satisfy Java interfaces. Which is cool, and useful for a lot of stuff. But the version of Rhino you can use on Dalvik has all of the bytecode generation disabled (obviously) and as a consequence, it can't emit the necessary bytecode to create the interface-fulfilling Java wrapper to make the JS object do what you want. It's generally bogus. Also it's really hard to express how much slower Dalvik is than HotSpot in the general case, too. It seems faster with the Android 4.2 version (by about 30%, in my highly unofficial testing) but it's still pretty agonizingly slow.
- honr 14y agoInteresting idea (especially, hashing based on command line options, and probably also the libraries in the classpath); I should add this to Clove (http://hovel.ca/clove http://hovel.ca/clove : a small persistent-jvm that I wrote for *nix, with VERY fast connection time, e.g. suitable for scripts; sorry for shameless plug).
- wiradikusuma 14y agohey, nice project! how does this relate to drip? e.g. alternative or addition?
- honr 14y agoThey do slightly different things. I think each can benefit by incorporating the main idea of the other: - Working on multiple projects who need different JVMs running at the same time can be a hassle in clove (you have to create a service for each), so it can be fixed by doing that the same way drip does. - Running scripts and having a persistent JVM is still very useful, and clove (albeit I haven't cleaned up its code properly) gives a terminal into the JVM, really fast. So, drip can use the same technique (passing capabilities; a *nix feature) to let you hook into a JVM very quickly. [see the "screenshot" part at the bottom of http://hovel.ca/clove http://hovel.ca/clove]
- olaf 14y agoMy experiences with drip were very mixed (on Ubuntu 10.04), it did not make a stable, reliable impression on me. I wouldn't recommend it for professional use.
- pbiggar 14y agoDoes anyone know how to use with with lein?
- welp 14y agoUsing lein2, you can run the following: $ JAVA_CMD=drip lein ...
- pbiggar 14y agoStill using lein1 :(
- airlocksoftware 14y agoWhat does this do exactly? Will it permanently change lein to use drip? Or only temporarily for whichever command replaced the "...". Also, do you know if the advice here: https://github.com/flatland/drip/wiki/Clojure https://github.com/flatland/drip/wiki/Clojure works with lein2?
- mcav 14y agoThat command runs drip only for that particular invocation of the command. You can put the following: export JAVA_CMD=drip into your ~/.profile (etc) file to have it affect all future Java launches from leiningen.
- aaroniba 14y agoIt looks like drip does not automatically work with lein: https://github.com/flatland/drip/issues/36 https://github.com/flatland/drip/issues/36 Here's my workaround to use drip to start up lein repls instantly: https://gist.github.com/4103370 https://gist.github.com/4103370 Should work with lein1 & lein2.
- deleted 14y ago[deleted]
- mitchi 14y agoI read his explanation but I don't have a great understanding of why a JVM would need to clean up like this. Doesn't it have a Garbage collector just for that? Why does it get slower over time? The only thing I know that gets slower over time without you doing anything special is my dad's mac os x word-only macbook pro.
- georgeorwell 14y agoAs a further optimization, why not memcpy an uncorrupted JVM to some backup place in memory and then when you want to 'reboot' just memcpy the image back again?
- isbadawi 14y agoI don't know how I feel about this: https://github.com/flatland/drip/blob/develop/src/org/flatland/drip/Main.java#L139 https://github.com/flatland/drip/blob/develop/src/org/flatla...
- zmmmmm 14y ago> It keeps a fresh JVM spun up in reserve with the correct classpath and other JVM options so you can quickly connect and use it when needed So I assume it doesn't help if you are launching JVMs very rapidly (like, scripting stuff in a tight loop). Slow JVM launching has pretty much killed languages such as Groovy for scripting for me, because once I start using them in loops things get horribly slow.
- jrockway 14y agoWhy not move the loop to your script?
- thomaslee 14y agoBecause unix? :)
- pretoriusB 14y agoThe Unix philosophy has nothing to do with this. It's not like the stuff inside the script can't be standalone classes of the "do one thing and do it well" variety --they just don't get to be processes.
- mercurial 14y agoOn the other hand, you may not want to reimplement sed, grep, etc in your scripting language of choice. There are times where a good "perl -e" can be very useful in a shell script.
- azth 14y ago> The main problem is that the state of the persistent JVM gets dirty over time, producing strange errors... Does anyone know what kind of errors the author is referring to? Does any long running JVM instance run into these issues?