6 ms·
Why low-latency Java still requires discipline?
- dominicrose 3mo agopage doesn't load "En attente de la réponse de chronicle.software."
- PaulHoule 3mo ago+ low latency anything requires discipline. if you lose 5ms you can't get it back.
- motoboi 3mo agoI found claude and GPT very helpful on this, because java have a very sofisticated monitoring harness. Just ask the agent to connect to the running application (on kubernetes or whatever) on prod and do a java flight recording then analyze allocations. I managed to improve some applications of ours from several garbage collections per second to several minutes between collections. That _really_ improves p99.
- wmichelin 3mo ago[dead]
- declan_roberts 3mo agoThat sounds like a great idea. What kind of prompt do you use?
- motoboi 3mo agoNothing much smarter than run the agent in the project folder, make sure it has means to interact with production (like configured kubectl) and ask it to jcmd the hell out of it.
- haglin 3mo agoI work on OpenJDK, so I may be biased, but I’ve also found that JFR works really well with LLMs. $ java -XX:StartFlightRecording:maxsize=10M,filename=dump.jfr -jar app.jar $ jfr view all-views dump.jfr > report.txt $ jfr print dump.jfr > all.txt Then ask Codex, or whatever AI tool you use, to analyze report.txt for issues and use all.txt to dig deeper, if needed.
- re-thc 3mo ago> but I’ve also found that JFR works really well with LLMs There's also async profiler. LLMs just use whatever. Lots of choices.
- exabrial 3mo agoTake a peek at jolokia if you've never used it :) Attach it as a JVM agent to _any_ process to expose it's JMX attributes as http calls, and it even gives you a neat little console to log into. If you want to collect these, telegraf has a jolokia plugin. It's an incredible combination!
- chuckadams 3mo agoI get that blog posts often advertise a company's products, but this one had absolutely zero content other than advertising.
- peter_lawrey 3mo agoThis is a high-level article; for lower-level details, I also posted. https://blog.vanillajava.blog/2026/06/testing-java-memory-management-with.html https://blog.vanillajava.blog/2026/06/testing-java-memory-ma... https://blog.vanillajava.blog/2026/06/why-you-should-tun-code-before-your.html https://blog.vanillajava.blog/2026/06/why-you-should-tun-cod... Suggestions welcome, it can be hard to know what to include or not.
- jbellis 3mo agoYes, disappointing given that Chronicle actually does have legit expertise here. (Chronicle Map is not very well-known even in the Java space but it's by far the best larger-than-memory Map available.)
- peter_lawrey 3mo agoThank you for the feedback
- simpaticoder 3mo agoI think it said something very meaningful and actionable (admittedly at a high level) about keeping the hot path deterministic: low allocations (including guidelines for GC frequency targets), don't use vthreads (instead use one thread on a pcore that never sleeps, to avoid amdahl), caution about techiques that work at p99 and fail at 99.999... This is not typical for "enterprise" Java, and is an interesting article for those who wish their systems were more deterministic (but may not be aware of the techniques or their cost). This is not a "how to", but that's fine. It's still good content.
- peter_lawrey 3mo agoGood feedback
- opentokix 3mo agoHow about not using Java? Then you can have low latency. Average go, rust, c++ and c will outperform amazing java programs, and the former will also be way way more easy to run, troubleshoot, interpret logs from. Java is usch garbage in every stack.
- motoboi 3mo agoRust? OK. C++ or go? Then you'll have to take a very closer look, because the java JIT is wonderful. A masterpiece of several hands, actually.
- bluGill 3mo agoIf low latency is your goal than you don't want JIT. JIT has two issues in low latency, first the first time through your code isn't compiled yet and so you get high latency. Second, it optimizes for the common case, which means when you hit an exception that exception will be higher latency because everything hits a branch miss. Of course we are talking generals here. Sometimes the above is acceptable and Java/JIT is just fine. Sometimes it is unacceptable and you cannot use Java/JIT. Know your domain. Of course in all cases (C, Rust, C++), you have to understand the system and what it is doing. Every language has a standard library that will do things that are not low latency on you. You have to know which library functions will do what, memory allocation and copies are both things that code often does without thought that are incompatible with low latency. No matter what you need to know what your language does that is against you.
- re-thc 3mo ago> If low latency is your goal than you don't want JIT. JIT has two issues in low latency... There's startup "AOT cache" via Leyden that speeds up startup. Isn't native speed up it's quite a big boost. Then there's GraalVM that does give you a native image. Real AOT.
- pretzellogician 3mo agoValid points, but this doesn't mean JIT doesn't work for relatively lower-latency coding. To avoid the compilation etc. hit, common practice is to do some "warmups" before serving users. (Another reply has other ways to avoid this hit.) Handling exceptions is higher latency, but they can/should be optimized out, so you're not hitting exceptions as part of your standard workflow (or even your 1% workflow).
- pjmlp 3mo agoEven C requires discipline to write low latency code, if you think otherwise, you never used a profiler.
- madduci 3mo agoAnd Compiler Explorer also
- __s 3mo agoYes. & in the end my experience has been that you can write low latency code with gc, but only by expending more effort than it'd take to use manual memory management
- pjmlp 3mo agoFirst of all it depends on the language, what GC models it supports, and what are the performance SLAs. Secondly if we are talking about something like real time audio, or sending the maximum amount of triangles per second, then there is zero memory management, the code should be allocation free across all the critical path, regardless of the language.
- whaleofatw2022 3mo agoAka 'follow JSF guidelines that still apply to C and not just C++'. I.e. all alloc should happen up front, functions should have predictable execution time, etc etc
- PaulDavisThe1st 3mo agoNot just allocation free. It needs to be page-in free too. And critical loops need to be cacheline friendly.
- pjmlp 3mo agoYeah, that as well.
- treyd 3mo agoOn mobile the whole page text is aligned center which makes it really hard to read.
- sandeepkd 3mo agoHad to write a router software once in Java back in 2014. The past experience in me believed that its not a good choice to use something with stop the world GCs for routers. The goal was to have everything under 1ms (max 2ms in rare cases). It has http server on head and websocket connections on the tail end. Turns out the class instantiations and JSON parsing were a bottleneck. Thread safe resources were valuable, the libraries which did not had thread safe resources were re-written to support what we needed. The http server was re-written barebones, the JSON parsing was re-written with barebones. And it worked.
- gwittel 3mo agoChronicle has a lot of great resources. I’ve been out of Java for a few years but when I had to write high perf code, the libraries and blog insights were invaluable. A lot of the advice is good in general - keeping things simple, generating fewer objects, etc. Profiling with Yourkit or JMH to find and improve slow spots. This let us build high performance software that had a vast difference in say p90 input size and p99. It involved rewriting third party libraries to let us 3x speed and substantially reduce GC. I think in the end we were around a p99 of 7ms and p90 well under that (the data spread was kbs to megabytes) across hundreds of millions of inputs per day.