13 ms·
A JVM Does That? (2011) [pdf]
- markokrajnc 10y agoThese slides are great! "Standing on the shoulders of giants..." If time-to-market is important, JVM is the way to go... Manually managed memory is for low-level/OS software, GC is for the rest (apps)... with some exceptions...
- cmrx64 10y ago(2011) Good slides anyway. Anyone know of a recording of the corresponding talk?
- chollida1 10y agoIf you are part of the HFT crowd who uses the JVM then you know who the author, Cliff Click, is. His Blogs (former and current) are goldmines for high performance JVM information. http://www.azulsystems.com/blog/author/cliff http://www.azulsystems.com/blog/author/cliff http://www.cliffc.org/blog/ http://www.cliffc.org/blog/ I had one CTO of a public HFT firm tell me that short of FPGA's Azul's pause-less GC (https://www.azul.com/products/zing/pgc/ https://www.azul.com/products/zing/pgc/) was the biggest performance win they'd had in the past few years, note: this was a few years ago. EDIT: someone emailed asking for how HFT firms write their java code. I haven't written java in 3 years so I'm probably not the best person to author a list but this is what I'd include: In order of importance: - Measure, measure, measure. Every HFT firm can tell you to the nanosecond, as much as that's possible, what their time is from receiving a packet to when it's replied to. Focus as much, if not more on the 95%, 99% times as the average time. This time increasing is considered a bug in the same way that an app crashing due to user input is considered a bug, which is to say that you just don't ship with this kind of bug. - no GC's, everything else starts to become less important if the GC is called every 30 seconds. Some guys will have only one GC call a day, which means you have a really large Eden GC space - short call stacks, and no recursion - one physical core per thread - non locking data structures and use the one writer principle where possible - don't use the kernel if possible, ie solar flare Ethernet cards that have userland drivers - cache friendly data structures Watch this video: https://www.youtube.com/watch?v=iINk7x44MmM https://www.youtube.com/watch?v=iINk7x44MmM
- duaneb 10y agoAzul is still state of the art, certainly for the JVM.
- alexhutcheson 10y agoGreat interview with him from way back in 2007: https://web.archive.org/web/20071020182028/http://theserverside.com/tt/knowledgecenter/knowledgecenter.tss?l=MetalMeetsJVM https://web.archive.org/web/20071020182028/http://theservers...
- bitmapbrother 10y agoThere's also this video by Gil Tene on Low Latency Java in the Real World: LMAX Exchange: https://www.youtube.com/watch?v=6-oS2XHSGvk https://www.youtube.com/watch?v=6-oS2XHSGvk
- azeirah 10y agoIf performance is THAT critical in HFT, why do they pick java as the starting language? I understand that java is very fast, faster than most people believe, but does it beat hand-optimized assembly? Fortran? C? Why Java?
- PaulHoule 10y agoIt's close to impossible to get certain kinds of code correct in C, for instance, string parsing. Also until pretty recently, C and C++ did not have a sane memory model so what happened when threads was involved was dependent on what CPU you are on, the sign of the moon, etc. Java could very well be the first programming language designed by adults and it shows. (That is, they solve the hard problems, not just pretend they don't matter)
- revelation 10y agoEh it's HFT, I'm not sure there is much string parsing going on. Or variable length anything.
- 10y ago
- geodel 10y agoThis is interesting – (Azul GPGC: 100's of Gig's w/10msec) – (Stock full GC pause: 10's of Gig's w/10sec) – (IBM Metronome: 100's Megs w/10microsec) Some how these numbers are hard to come by in Java land where any request for hard numbers receives response like Java GCs are 'generational', 'state of the art', 'best in industry' etc. All the technical arguments while true of course but they do not tell numbers.
- nulltype 10y agoI think when comparing numbers like that you might need throughput figures as well. Supposedly there is a tradeoff between GC pauselessness and throughput.
- emmelaich 10y ago"Supposedly?" I would have thought inevitably.
- nulltype 10y agoWell I've never implemented a GC or built a processor specifically for implementing GC on.
- snaky 10y agoYes, it's being discussed recently https://news.ycombinator.com/item?id=12044044 https://news.ycombinator.com/item?id=12044044
- cliffc 10y agoAzul's GC required a read-barrier (some instructions to be executed on every pointer read) - which cost something like 5% performance on an X86. In exchange max GC pause time is something in the low microsecond range (I helped implement the Azul JVM and portions of the GC relating to starting and stopping threads)
- Gibbon1 10y ago
- ww520 10y agoThe JVM's memory model was something amazing when I first encountered it.
- derefr 10y agoIf you were incentivized enough (by being an HFT firm, say), is there anything stopping you from re-implementing the Erlang memory/GC-model on the JVM? I know the JVM has green-threads (somewhere amongst a million other concurrency primitives); but could it be modified to give each green-thread its own heap, and then do background-GC passes of each green-thread heap when that green-thread isn't currently scheduled? You'd certainly have to write your Java somewhat differently—though you wouldn't necessarily have to move to a full-fledged no-shared-memory message-passing model to see a benefit. (For example, every JVM object could be treated similarly to Erlang's large binaries, where they exist as refcounted objects in a shared heap, and then the ref-handle within the green-thread heap is itself refcounted by the green-thread, so the shared heap only needs to be synchronously updated when a green-thread heap is discovered, on GC, to have released its last ref-ref.)
- newobj 10y agoRelevant: http://mechanical-sympathy.blogspot.com/ http://mechanical-sympathy.blogspot.com/
- mamcx 10y agoExist a resource like this for .NET?
- alblue 10y agoEach thread does have its own local TLAB in which bump allocations are performed; but you still have the GC effect even if you're using that. Provided you stay entirely within Eden then your GC will just drop the space - but it is preferable here to not generate any garbage in these situations by eg pre-allocating the necessary objects and just doing field writes.
- mtrn 10y ago> is there anything stopping you from re-implementing the Erlang memory/GC-model on the JVM Apparently not, never used it, but impressive: https://github.com/trifork/erjang https://github.com/trifork/erjang
- vonnik 10y agoRelevant: JavaCPP, the glue code that acts as a bridge to C++/C https://github.com/bytedeco/javacpp https://github.com/bytedeco/javacpp JavaCPP's creator works with Skymind, and we use it for our open-source deep learning framework, Deeplearning4j https://github.com/deeplearning4j/deeplearning4j/ https://github.com/deeplearning4j/deeplearning4j/ We're push the limits of the JVM (and we love Cliff's work!).
- badlogic 10y agoJavaCPP does not compile Java bytecode to C++. It is a Java to C/C++ bridge ala JNA, just better. Major difference.
- Thaxll 10y agoI'm curious about going above 32GB of heap size, heard that compressed pointers kill performance?
- sluukkonen 10y agoGoing above 32G doesn't kill performance per se, it just means that there is not point in using heap sizes between 32G and 48G due to the larger pointers. If you go above that, it's all good again.
- etatoby 10y agoI can't believe you people. FPGAs? Seriously? HFT is an abomination. The fact that it's not banned is proof that the entire financial sector is the cancer of society.
- JBReefer 10y agoHFT reduces the spread, which is actually great for consumers. It also pushes the frontiers on high speed scalable computing out and produces a ton of great research. Finally, it is great for the tech scene in NYC.
- duaneb 10y ago> Finally, it is great for the tech scene in NYC. I dunno, the fact that I have to sell my soul to finance to find a decent tech job doesn't speak particularly well to its health.
- jgalt212 10y ago> I dunno, the fact that I have to sell my soul to finance to find a decent tech job doesn't speak particularly well to its health. As opposed to selling your soul to the advertising industry. At the end of the end of the day, unless you are independently wealthy or live in a very low cost area, you're gonna have to make peace with Adam Smith's invisible hand.
- gozur88 10y agoAs opposed to not having a tech job at all?
- tajen 10y ago> HFT reduces the spread ... by chopping off the difference and putting it in the pocket of the HFT, right? I mean, without them, the earnings would go to the pocket of more loosely programmed organizations (=probably, our banks).
- greenleafjacob 10y ago
- dmytroi 10y agoMight be unpopular opinion, but I kinda like to look how people introduce will-be-a-problem-in-the-future things (let's say GC) and than heroically are solving following problems (pause-less GC), it's like Don Quixotes of modern age, except you build windmills yourself.
- usrusr 10y agoPriorities. Getting rid of manual memory management can be a worthwhile goal even with slow gc, just like not having expensive runtime memory management automatisms can be a worthwhile goal. Making gc faster (or just more predictable) on top of that is not heroically solving a follow-up problem, it's an optional bonus achievement.
- dmytroi 10y agoI might be too old fashioned, but the only case where some sort of memory management is pleasant and comfortable - constructing/modifying strings, though even here many languages separate mutable and immutable strings into different beasts. In all other cases just allocate virtual page from OS (way too old school I understand, people usually don't use this one and believe that C Runtime Library with malloc is the only "approved" way to allocate memory), you get from 4kb to many megabytes of continuous memory - do whatever you want there, and deallocate whole virtual page later on, no leaks, no problems. Sure I do understand that manually managing millions of objects on heap is hard, but managing 1 to 10 virtual pages? it's simple and easy, just forged about generic allocators and use specialized ones (like block allocator for example). Also it's good to know if you even need to deallocate at all ... for example ninja build system (https://ninja-build.org https://ninja-build.org) simply doesn't deallocate memory after build is done, just because it's too slow to do it (though they do use heap allocations), just kill the app and OS will clean up after you. PS. On other hand what I'm saying is just a personal rant. Reality is that the industry is mostly moving to tools that don't allow writing unsafe code (for example languages without pointer arithmetic), and this might be very limiting in language expressiveness. There is an another approach - statically proving that unsafe code is safe. This is what "Checked C" is all about, and this is how Windows drivers are working without crashing the kernel (at least BSODs are pretty rare on Windows nowadays). But I do agree that tools for statically proving unsafe code are so non casual that only greybeards are interested in them nowadays :(
- hyperpape 10y agoThere's another talk that you might want to watch if you liked these slides: https://www.youtube.com/watch?v=vzzABBxo44g https://www.youtube.com/watch?v=vzzABBxo44g Bits of advice for VM writers.
- frankpf 10y agoI'm probably missing something obvious here, but on page 25 it says: Azul Systems has been busy rewriting lots of it - Many major subsystems are simpler, faster, lighter - >100k diffs from OpenJDK If Azul Systems' JVM is based on OpenJDK, shouldn't it be open source? OpenJDK is licensed under GPLv2.
- desdiv 10y agoOpenJDK is licensed under GPLv2 but it used to be available under commercial licenses as well. Kinda like how MySQL is GPL/commercial dual licensed.
- bogomipz 10y agoI am curious if someone could shed some light on page 15 - "Illusion: Consistent Memory Models", specifically: X86 is very conservative, so is Sparc Power, MIPS less so IA64 & Azul very aggressive I dont't thing I've heard the term "conservative memory model" before, in what sense are x86 and SPARC conservative? Also why is Azul mentioned as an ISA?
- PeCaN 10y agoAzul has their own hardware for facilitating pauseless garbage collection with their JVM. I recall they have a pure software pauseless JVM now as well, but it's not nearly as high performing.
- bogomipz 10y agoThanks, I had no idea they started with hardware. For anyone else not that familiar it looks like Vega is the hardware appliance and Zing is the VM version.
- cliffc 10y agoI have board with 2 Vega chips in my living room - 24 cores each; each a full 64-bit RISC with 32 registers and ie754 FP math, read & write barriers in hardware; inline-cache virtual calls in hardware, and transactional memory. Later chip versions had 48 and then 54 cores, and you could jam 16 chips in a cache-coherent shared memory box - with top-10 supercomputer bandwidth. But you needed 54*16 parallel working threads to use it all. :-) Cliff
- bogomipz 10y agoSo in the bit of reading I've done it seems like the idea of read and write barriers in hardware were what allowed them to have puase-less GC and that was the primary reason for Azul producing hardware appliances. Is that accurate? At least initially? What did that board cost you if you don't mind my asking? What does it cost you to run it is maybe a better question.
- AlexCoventry 10y agoWhat does this mean? Large chunks of code are fragile (or very 'fluffy' per line of code)
- cliffc 10y agoIt means what it says it means? "fragile" code is code that can't be touched because touching it causes it to "break" - exhibit bugs that can't easily be fixed. "fluffy" code is bulky code: code that says in 1000 lines what could better be said in 10 lines. Cliff
- sievebrain 10y agoWhat's your view on Graal? Is that making headway on de-fluffing and robustifying the code or does this not really tackle the issues you had in mind?
- AlexCoventry 10y agoThanks. "Fragile" made sense, "fluffy" was new to me.
- ktRolster 10y agoIs this assertion really true? Class files are a lousy way to describe programs Class files seem alright to me
- lordnacho 10y agoSitting at the offices of an HFT firm here. I think there's more than one kind of fast. There's development speed, and there execution speed. Both are important. I don't know what qualifies as HFT these days, but a loose definition is people who trade more often than once a day. That's still a whole lot of different people. At one end, there's textbook, pure footrace arbitrage. You something offered at 100 on one exchange and bid 101 on another. You rush to do them both. This is possibly the most obvious strategy ever, and the only thing that matters is how fast you can get those messages to the relevant exchanges. On the other end, there's more intricate stuff like statistical arbitrages (a wide category), where there's more than one way to skin a cat. Some principles are known from finance 101, but your implementation will be slightly different to other people's. You still want to execute fast, because the opportunity may not be there forever, but it's not like every time you seen an opportunity you know it's first come, first served. My sense is that those closer to the former will tend to use c++ over JVM. Most people are not going to have the time to carefully test what's faster, and c++ has the reputation as the thing that will be faster. Similarly, JVM languages have a reputation for being faster to code in. As strategies get more complex, you need code agility. There's a lot of changing things up when you're writing strategy. Of course, results vary, but if you're building a strategy platform, you probably go for JVM. But that's just what my gut feeling tells me. I've only met the people I've met, and it's not that easy to find public info on just what people are getting up to. I'd love to hear what kinds of strats are running on Azul.
- nvarsj 10y agoIt's been a while since I was working in HFT, but the HFT firms making the most money were all using Java predominantly. Yeah a lot of firms use C++ but I think that's more because of myth than actual benefits. Pure simple arbitrage is a race to the bottom very quickly, and it's very difficult to remain profitable with those sort of simple strategies regardless of what language you use. Maybe FPGAs and radio networks could let you make money off of simple arbitrage (someone must be making money from it). C++ isn't going to give much advantage over Java with GC disabled.
- lordnacho 10y ago
- bshanks 10y agoThank you pas for this link (via https://news.ycombinator.com/item?id=12025929 https://news.ycombinator.com/item?id=12025929 )