4 ms·

Show HN: Blunders.io, a tool to profile JVMs in production

46 points by antonhag 6y ago

antonhag 6y ago
Hi HN! I'm Anton, and I wanted to show you this tool that I have been building over the last year or so. It's based on experience from working with performance of Java apps (particularly Elasticsearch), and would have made my life so much easier in that role. I would love feedback on essentially everything! You can see the core of the product by pressing the "demo" button on the top of the landing page. Thanks in advance :)
- modoc 6y ago
  Anton, looks cool. A couple questions if I may: 1) What is the performance impact under typical production loads? Many JVM profilers are absolutely brutal to JVM performance and can't be realistically used in Production. 2) How does this compare to the lightweight JVM profilers offered by New Relic and Dynatrace? Thanks!
  - antonhag 6y ago
    Thanks for checking it out! 1) The agent is based on the Java Flight Recorder (https://en.m.wikipedia.org/wiki/JDK_Flight_Recorder https://en.m.wikipedia.org/wiki/JDK_Flight_Recorder) which is a technology built into the (Oracle/OpenJDK) JVM. The performance impact is thus rather small, around 1-2% of CPU. 2) To the best of my knowledge, New Relic and Dynatrace can't provide similar levels of JVM metrics (e.g. which methods allocate the most memory), since they are based on JMX metrics rather than flight recorder data. They do however have better APM functionality and framework integration than Blunders.
- jacques_chester 6y ago
  A note: the "Memory allocation flame chart" image isn't rendering for me in Firefox. It renders in Chrome.
  - antonhag 6y ago
    Thanks for noticing, I have been noticing the same thing. Trying to figure out why, usually it works just fine in FF as well.
philipkglass 6y ago
How does this compare with Flight Recorder? As of Java 11, Flight Recorder and its analysis tools (JDK Mission Control, formerly Java Mission Control) are free and open source: https://github.com/openjdk/jmc https://github.com/openjdk/jmc I have found Flight Recorder light weight enough to have it always enabled on production services. I didn't start using it until last year because it was a commercial feature on older Java releases. EDIT: another comment says that this is based on Flight Recorder. The value add proposition is around the interface rather than the instrumentation, then. I think there is an opportunity there. JMC is fairly heavyweight and doesn't have good visualization/exploration tools for everything.
- antonhag 6y ago
  You are exactly right! The value added here is a having automatic uploads of JFR dumps, which are viewable based on machine and timespan through a web UI. If you have it enabled in production, I assume that it requires some effort to find the right file and download it to view it on JMC? And if you want to share that view with a colleague they have to do the same. Blunders aims to make such things easier. I've also tried to make a "better" visualization (based on flame graphs), not sure if I succeeded though :) Edit: typo
  - philipkglass 6y ago
    Dumps go automatically to an S3 bucket with an expiration policy. But everyone who wants to visualize the dumps with JMC needs to build JMC from source locally, since there is still no stable release process or packaging for it AFAICT.
    - eggsnbacon1 6y ago
      AdoptOpenJDK provides builds for JMC now https://adoptopenjdk.net/jmc.html https://adoptopenjdk.net/jmc.html New versions of VisualVM can also open flight recorder files
runT1ME 6y ago
This seems pretty cool. Would I still get value if I only want to sample performance a few hours a week, say, under high load? Do you allow for that?
- antonhag 6y ago
  In theory yes, but there is some building that I'd need to do to make that easy. You can make it happen now by restarting the processes with and without the agent, but the goal is to have it configurable in the UI. If you like to, I could try to get back to you once that's built?
sqrt17 6y ago
Is there a similar tool that you can throw at Electron/NodeJS apps to figure out why Slack and Teams are eating all the CPU and memory again?
- pjmlp 6y ago
  I guess Chrome DevTools, since they can plug into other instances.
SNosTrAnDbLe 6y ago
I would not attach a java agent which sends out JVM profiling to a remote site and that too in production. Do you intend to have an offline version of this tool?
- antonhag 6y ago
  For offline, I don't really see the point? The biggest use case here is being able to pull in data from many machines into one place. Self-hosted solutions would be possible though. I'd also like to clarify what kind of data is uploaded - it's what kind of classes and methods are used, not any of the actual data. E.g. it might contain information that you use BCrypt, but not input nor output. It is however, of course, your decision if you judge that data sensitive or not.
liquid153 6y ago
Pretty cool tool, could easily see myself using this

46 points by antonhag 6y ago

Hi HN! I'm Anton, and I wanted to show you this tool that I have been building over the last year or so. It's based on experience from working with performance of Java apps (particularly Elasticsearch), and would have made my life so much easier in that role. I would love feedback on essentially everything! You can see the core of the product by pressing the "demo" button on the top of the landing page. Thanks in advance :)

modoc 6y ago
Anton, looks cool. A couple questions if I may: 1) What is the performance impact under typical production loads? Many JVM profilers are absolutely brutal to JVM performance and can't be realistically used in Production. 2) How does this compare to the lightweight JVM profilers offered by New Relic and Dynatrace? Thanks!
- antonhag 6y ago
  Thanks for checking it out! 1) The agent is based on the Java Flight Recorder (https://en.m.wikipedia.org/wiki/JDK_Flight_Recorder https://en.m.wikipedia.org/wiki/JDK_Flight_Recorder) which is a technology built into the (Oracle/OpenJDK) JVM. The performance impact is thus rather small, around 1-2% of CPU. 2) To the best of my knowledge, New Relic and Dynatrace can't provide similar levels of JVM metrics (e.g. which methods allocate the most memory), since they are based on JMX metrics rather than flight recorder data. They do however have better APM functionality and framework integration than Blunders.
jacques_chester 6y ago
A note: the "Memory allocation flame chart" image isn't rendering for me in Firefox. It renders in Chrome.
- antonhag 6y ago
  Thanks for noticing, I have been noticing the same thing. Trying to figure out why, usually it works just fine in FF as well.

philipkglass 6y ago

How does this compare with Flight Recorder? As of Java 11, Flight Recorder and its analysis tools (JDK Mission Control, formerly Java Mission Control) are free and open source: https://github.com/openjdk/jmc https://github.com/openjdk/jmc I have found Flight Recorder light weight enough to have it always enabled on production services. I didn't start using it until last year because it was a commercial feature on older Java releases. EDIT: another comment says that this is based on Flight Recorder. The value add proposition is around the interface rather than the instrumentation, then. I think there is an opportunity there. JMC is fairly heavyweight and doesn't have good visualization/exploration tools for everything.

antonhag 6y ago
You are exactly right! The value added here is a having automatic uploads of JFR dumps, which are viewable based on machine and timespan through a web UI. If you have it enabled in production, I assume that it requires some effort to find the right file and download it to view it on JMC? And if you want to share that view with a colleague they have to do the same. Blunders aims to make such things easier. I've also tried to make a "better" visualization (based on flame graphs), not sure if I succeeded though :) Edit: typo
- philipkglass 6y ago
  Dumps go automatically to an S3 bucket with an expiration policy. But everyone who wants to visualize the dumps with JMC needs to build JMC from source locally, since there is still no stable release process or packaging for it AFAICT.
  - eggsnbacon1 6y ago
    AdoptOpenJDK provides builds for JMC now https://adoptopenjdk.net/jmc.html https://adoptopenjdk.net/jmc.html New versions of VisualVM can also open flight recorder files

runT1ME 6y ago

This seems pretty cool. Would I still get value if I only want to sample performance a few hours a week, say, under high load? Do you allow for that?

antonhag 6y ago
In theory yes, but there is some building that I'd need to do to make that easy. You can make it happen now by restarting the processes with and without the agent, but the goal is to have it configurable in the UI. If you like to, I could try to get back to you once that's built?

sqrt17 6y ago

Is there a similar tool that you can throw at Electron/NodeJS apps to figure out why Slack and Teams are eating all the CPU and memory again?

pjmlp 6y ago
I guess Chrome DevTools, since they can plug into other instances.

SNosTrAnDbLe 6y ago

I would not attach a java agent which sends out JVM profiling to a remote site and that too in production. Do you intend to have an offline version of this tool?

antonhag 6y ago
For offline, I don't really see the point? The biggest use case here is being able to pull in data from many machines into one place. Self-hosted solutions would be possible though. I'd also like to clarify what kind of data is uploaded - it's what kind of classes and methods are used, not any of the actual data. E.g. it might contain information that you use BCrypt, but not input nor output. It is however, of course, your decision if you judge that data sensitive or not.

liquid153 6y ago

Pretty cool tool, could easily see myself using this