7 ms·
Java is very fast, if you don’t create many objects
- greenbit 4y agoAh, but java seems to go out of its way to make you make so very many objects. It's almost as though it tries to rub your face in the fact that you don't have to remember to delete objects, by needlessly making you make them. Iterators for example. Why (why!) should it be necessary to incur memory thrash, just to traverse a collection? Why not allow a class that has a collection also have a reusable iterator? Just a simple .reset() method would work wonders for many of these "disposable" objects. And if .reset() offended anyone, they could always ignore it and continue to blithely throw candy wrappers out the car window, as it were.
- peter_lawrey 4y agoIterators tend to be placed on the stack in hot code. In the benchmark I had to make sure these object weren't optimised away.
- grumpyprole 4y agoA bigger issue I think is Java's need to box values, primitive or otherwise. For example, even an ArrayList of integers creates an object for each one. IMHO, the Java runtime has many of the design hallmarks of a dynamic language, with strong (generic) types somewhat bolted on.
- peter_lawrey 4y agoA friend of mine is working at Oracle on collections that support primitives e.g. List<int>, Set<long> https://openjdk.org/jeps/8261529 https://openjdk.org/jeps/8261529
- WastingMyTime89 4y agoIt’s not always obvious what’s actually boxed and what isn’t however. The JVM tries to do clever optimisation. For example, I’m fairly certain that when you manipulate an array of primitive numerical values, they are not actually unboxed before each operation before being boxed again.
- kjeetgill 4y agoHmmm. As much as I hate having to fix boxing problems, I usually find it pretty obvious when boxing can occur. Pretty much any time a variable/parameter type or a function return type uses capitalized Integer, Long, Boolean, etc. instead of lowercase int, long, boolean, etc. boxing is probably occurring. Those ... and assigning those to Object or Number. I think that's exhaustive. You can pretty much grep through and remove them from 90 % of coffee with a little refactoring and replacing HashMaps with specialized ones. Only if there's null returns or ConcurrentHashMaps involved does it get tricky. An array of primitives like byte[] or long[] isn't Byte[] or Long[]. The former don't cause boxing. Maybe you're thinking of ArrayList<Byte>?
- WastingMyTime89 4y ago> Pretty much any time a variable/parameter type or a function return type uses capitalized Integer, Long, Boolean, etc. instead of lowercase int, long, boolean, etc. boxing is probably occurring. That probably occurring hides a lot of complexity. That’s my original point. Optimisations are happening especially when Integer and Long are involved. There are sometimes actually less boxing than you would expect reading the code.
- kjeetgill 4y agoTrue, but I'd argue you should treat those probable boxing situations and definitely boxing and remove them.
- zbentley 4y ago> the Java runtime has many of the design hallmarks of a dynamic language, with strong (generic) types somewhat bolted on. I'd say it's the opposite. Dynamic languages largely use boxing as the indirection mechanism for their highly dynamic type systems, since it works well there. But boxing isnt synonymous with type flexibility, and Java uses it for other reasons (mostly).
- grumpyprole 4y agoThe Java runtime provides runtime type information for all types, and extensive reflection is available, including for example invoking arbitrary code. Very dynamic and a significant problem for static analysis, optimisation and security.
- ivan_gammel 4y agoNo, thanks, that's a pretty bad idea. 1. There's no point in optimizing memory usage by hunting iterators. If the collection overhead is becoming visible, then you need an array, not a reset method. 2. You cannot easily add "reset()" to all iterators now, since it will be a breaking change. 3. Having reset() on all iterators 25 years ago would reduce the number of use cases for it, if this method had to be reliably supported. 3. Throwing UnsupportedOperationException in default method to maintain compatibility of libraries with existing user code would require the new client code using that method to catch the exception and handle it, which ruins the whole idea of optimization (you will get more overhead on that). I you are concerned about performance, you have to design your code with respect to it. Building your application the same way as if performance wasn't a big problem, using the same libraries and APIs is just a bad idea that won't be salvaged by such "optimizations".
- peter_lawrey 4y agoMost Iterators should be placed on the stack anyway. If they are not and you have a random access collection like ArrayList you can use an index, but this is rarely needed. If you are going to change code, perhaps Stream(s) are a better choice, these are also placed on the stack most of the time.
- thargor90 4y agoSometimes arrays are not an option, because objects may not be in memory at all times. A fun trick we use is to have the iterator be attached to the object lifecycle, so that repeated (parallel) iteration does not require multiple iterators. I can't remember the exact percentage, but this change alone increased our throughput by 20-25%.
- ivan_gammel 4y agoThat sounds both familiar and weird, like some strange lazy collection that performs an I/O operation to hydrate every time it creates iterator. Whatever it is, it's certainly not something that would justify a modification of standard API to cover this use case.
- 4y ago
- jeroenhd 4y agoIt's quite a paradox; Java is really fast if you keep allocations down, but Java "best practices" often involve tons of object creation. It can also be quite hard to find out if your code will be fast or not because there are a whole bunch of rules to determine if and when your Java code will be compiled to machine code or executed through JIT. Not having a way to find out if the compiler is smart enough to cache certain values or optimize some boilerplate code away like you can with native compilers is quite annoying sometimes.
- peter_lawrey 4y agoThis is where using a profiler like YourKit or Flight Recorder is your friend. It will show you the objects that can't be optimised away and where code bottlenecks are. It's hard to determine this in advance for Java.
- golergka 4y ago> It's quite a paradox; Java is really fast if you keep allocations down, but Java "best practices" often involve tons of object creation. That's because best practices are aimed at a task that is usually more important than performance: correctness and maintainability.
- Diggsey 4y agoYes, but the language would be better if correctness and maintainability were not so at odds with performance.
- UncleMeat 4y agoIt would be different. A large number of real world applications aren't especially performance sensitive and it makes sense to have tools that prioritize other things than performance. We also see how languages that do prioritize performance can suffer in other aspects (C++ is a clear example). I think that this is fundamental in some way. Consider arithmetic. Fixed with integers suck from an ergonomics perspective. But variable-size integers are necessarily slower. Even designing something as basic as how arithmetic will work in your language forces you to consider performance against usability.
- jt2190 4y ago> Java can be very fast, however, it can be well worth avoiding object creation. I would be extremely hesitant about taking this as general advice for your program. Java object creation should be treated as “fast enough until it isn’t”, at which point you can try applying an optimization technique like this.
- tralarpa 4y agoIndeed. Looking at the benchmark results, the conclusion should be that the overhead of object creation can be neglected for most types of applications (until it can't, as you wrote). Edit: Based on my experience, object creation is ridicously fast in the OpenJDK JVM and trying to avoid it will, for most applications, just result in code that is harder to maintain.
- m_mueller 4y agoI'd take "premature optimization is root of all evil" type of advice with a big grain of salt. IMO it's sensible for procedures, since they can often be optimized rather quickly. It's NOT sensible if you designing the application's basic data structures AND have an idea about the scale you want to achieve, which for data heavy applications is often the case (e.g. you know the rough file sizes that go into your importer beforehand). From personal experience, once #records times #fields goes into the millions, it starts to make sense to think about a data centric design. Functional interfaces are perfect to encapsulate e.g. a bag of arrays and not expose it to all the rest of your application.
- jerf 4y agoIf you are designing your application's basic data structures and have an idea about the scale you want to achieve, the optimization is not premature. "Premature" isn't a decorative word, it's doing real work in that phrase. There are plenty of people reading this article right now and going yeah, yeah, this is exactly what I need! while their task is writing an API that will receive 3 requests per minute that run in less than ten milliseconds each. We don't warn against "failing to optimize when it is needed" not because it isn't a problem, but because historically the bigger problem has been premature optimization. (I phrase that carefully, because I'm not sure the balance isn't shifting. I feel like I'm seeing more failure to do basic optimizations lately than people going crazy prematurely optimizing. I've started lightly banging on this drum and in 10 years we may need a luminary to say something like "FFS, people, adding up a few thousand integers shouldn't take three seconds! How did you even make your code that slow??", only, you know, pithily and quotable. But I can vouch for the fact that historically, going all the way back to when Knuth said that, premature and excessive optimization has been the bigger problem.)
- vips7L 4y agoReally wish that when people benchmark the GC they would include the GC settings they used or at the very least experiment across the several GC’s the VM has to offer.
- peter_lawrey 4y agoIn each case the default settings were used (i.e. no options were provided). In both Java 8 (Parallel GC) and Java 11+ (G1), As the GC was a small portion of the time, changing the GC wasn't expected to make a difference anyway.
- kjeetgill 4y agoBefore any one panics, please actually read the article. For some context, Peter Lawry is pretty famous for being the OpenHFT guy so he (rightfully) cares at scales that have very different concerns than general application code. He's working at the scale of 67 million events per second. Yea, for him a single allocation looks like : 67.8 M event/s, 472 ns vs. 50.1 M event/s, 638 ns As always with perf: scale and context matters! An allocation in these tests at these volumes was still only 160ns and at lower volumes was less than 50ns. These are closer to the scale of cache misses then a C malloc call.
- ape4 4y agoAiming for that speed, in addition to avoiding creating objects per-transaction, you should avoid anything else that could be time consuming.
- Karellen 4y agoEven so, I wonder what advantages Java has, that makes using it in that environment so overwhelmingly compelling over other languages. Enough that it overcomes the fact that you have to write very non-idiomatic Java if you don't want to incur unacceptable performance costs. It would be interesting to see what other languages were considered for the use case, and which cost/benefit considerations made Java come out on top.
- another2another 4y ago>have to write very non-idiomatic Java if you don't want to incur unacceptable performance costs. You would also, I guess have to choose any library dependencies very carefully as well. Or implement the functionality yourself, negating any advantage that might bring in speeding up development.
- chmod775 4y ago> Even so, I wonder what advantages Java has, that makes using it in that environment so overwhelmingly compelling over other languages. Presumably not the whole thing needs to be that fast, only the part that actually handles the individual events. Java has the advantage of being an industry standard, or at least being very common in the financial sector. So you can work in a single language while only having to write some un-idiomatic Java in places where performance is critical.
- gwbas1c 4y ago> The cost of object creation can be far higher than cleaning them up if they are very short-lived. I thought this problem was solved decades ago? Doesn't Java have a well-tuned generational garbage collector that can handle this? .Net was hyper-optimized for this over a decade ago with its 3-generation system, where the 0th generation allowed for extremely rapid allocation and collections. I thought Java had similar capabilities? (Summary, which BTW is probably outdated) 0 generation: Pre-allocate a continuous chunk of RAM. Every allocation goes to the next bytes of the chunk. When there's no more free RAM left, find all reachable objects in the 0 generation and copy them to the 1 generation. 1 generation: Works the same as the 0 generation, except it's larger and objects are copied to the 2 generation. 2 generation: Garbage collections here are traditional "mark and sweep" collections.
- peter_lawrey 4y agoThe GC isn't involved with allocation, the expensive bit here. The GC handles the clean up. The allocations create memory pressure that can easily saturate the bus to DDR if you are not careful.
- pveentjer 4y agoThe GC can pace the allocation when it can't keep up.
- ivan_gammel 4y agoI suggest that you pay attention to the numbers in the article. Even super-fast allocation can become an avoidable cost at certain scale.
- gwbas1c 4y agoI get that; but the issue isn't language / runtime specific. You'll hit similar issues in pretty much any programming language / runtime. Reusing memory used to be pretty standard for C/C++ programmers; and is also standard practice for "real time Java."
- habibur 4y agoSpeed is one part of the equation. Other parts are about how much resource it consumes for delivering that speed. Like memory and cores.
- oaiey 4y agoAnd man hours
- arnold_palmur 4y agoBig Peter Lawry fan - I use Chronicle Wire (the library mentioned in the article) often, it's a really nice serialization/deserialization library when performance/allocation is a focus. Combining Chronicle tools + Real Logic tools you can build some extremely performant Java applications.
- xyzzy4747 4y agoJust stop writing new software that isn’t I/O bound in garbage-collected languages. Use Rust.
- oaiey 4y agoThere are system languages and application languages. And application languages need a non operating system underpinning. So Java and .NET needs their high performance foundation.
- jqpabc123 4y ago
- rkalla 4y agoI used an approach similar to Peter's here to both an old streaming parsing library as well as a GUI (event handling) library -- effectively, re-use the same instance of the event shell with new data every time an event fired and copy the data out if you need to retain it for a long-lived operation or persistence, otherwise calculate and move on... the reduction in object creation overhead was significant and performance increase was around +30% for doing this in the two individual use-cases... BUT, the API was like a loaded gun pointed at your face. I knew what I was doing (with it) so it wasn't a problem, but if I over open sourced the API and provided it as a library I would envision a large portion of the population trying to handle the events in a multi-threaded context or throwing them into a List only to find the values changing on them during use (while the parser was still running on another thread). Performance was so tempting, but usability-face-shot-gun was the greater evil.
- peter_lawrey 4y agoReusing your objects can be confusing for other developers. You need to be careful with anything exposed via an API, however internally you can me more optimised.
- ben7799 4y agoThere's good advice there but you definitely need to not go crazy and get into premature optimization land. If you've got a tight section of critical code like he does this is a good technique, however reuse of the same objects through the critical section implies: - Mutable objects - Harder to verify correctness - Need to be very careful to avoid some very very serious bugs and security issues. E.x. in his HFT example. Improper reuse of an object without correctly resetting/setting all the fields could create bugs as serious as executing a trade against the wrong user or leaking data between users. None of this is really rocket science or java-specific. It's all basic old school optimization & computer science.
- peter_lawrey 4y agoMutable objects have their own issues, which can catch people out by surprise. Having good test coverage is important.
- karmakaze 4y agoIt's a short article. I was looking for any distinction between heap allocated and stack allocated objects. The compiler does escape analysis and will stack allocate anything that it can to a large degree. It then goes on to say > While allocation is as efficient as possible, it doesn’t avoid the memory pressure on the L1/L2 caches of your CPUs and when many cores are busy, they are contending for memory in the shared L3 cache. So I suspect that it's not all about gc, but also the overhead of object memory and especially object packing for caching and use of objects between threads. That's why we have things like LMAX Disruptor[0]. Until we get Project Valhalla value objects, you're likely better off using Go than Java for the object packing efficiency. [0] https://lmax-exchange.github.io/disruptor/ https://lmax-exchange.github.io/disruptor/
- kaba0 4y ago> but also the overhead of object memory I don’t think objects have the usual overhead when escape analysis says they can be stack-allocated. Regarding Go: Except that Go will be slower in most, complex cases.
- karmakaze 4y agoWhat would make 'complex' cases slower in Go? I know channels can be slow but you don't have to use them (though it's *ideomatic*) The benchmarks game shows them about the same[0] by How many times more CPU seconds. [0] https://benchmarksgame-team.pages.debian.net/benchmarksgame/box-plot-summary-charts.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
- kaba0 4y agoComplex programs will likely need objects (not ‘value types’)+, and Go’s GC is nowhere near as performant as Java’s. Benchmarkgame’s binary tree test is the one that is specifically made to stress test the GC and Java beats out every single managed language by a huge margin there. + Even in Rust/c++ one often has to go with dynamic life times like (A)RC, shared pointers, etc.
- nvarsj 4y agoWhen I worked in HFT with Java code, creating caches/pools of commonly used objects (generally serialized/deserialized real time parameter updates) was pretty common. It significantly reduced memory allocation in hotspots. But the better benefit, generally, was substantially reduced GC pause times. Java was used back in the 2000s because it was seen as the cool kid. More productive than writing C++ and easier to hire and train for.
- smabie 4y agoNot that many Java systems remain for HFT.
- kaba0 4y agoDepends on the type of HFT. On one end, not even C++ cuts it and FPGAs are used, while on the other hand I heard of plenty of Java deployments. Sure though, HFT itself is a niche.
- aasasd 4y agoYeah, afaik HFT folks also disable GC and just restart the app, or even the machine, at the end of the day. Java game devs are also fond of object pools, to my knowledge: they do as much allocation as possible at the start of a game level, and then avoid it until the level ends.
- nvarsj 4y agoYep that was what the critical loop trading components did (the ones w/ the models for buy/sell hooked up to the exchange feed). Completely disable GC and restart it every day. Every line of code in those critical loops was pored over to make it as fast as possible. These days it is a lot of FPGA based stuff but I like to believe the JVM stuff is still running somewhere.
- zumu 4y agoReminds me of https://lmax-exchange.github.io/disruptor/ https://lmax-exchange.github.io/disruptor/ The pre-allocation is a key feature. If you can lower GC you get all the benefits of hotspot and much less draw back. Whether or not you actually need this is another question. Anything outside of real-time trading is highly suspect.
- backenddevelopm 4y ago[flagged]