12 ms·

It's hard to square these articles with the reality I see on the ground: our baseline memory usage for common types of Java service is 1 GB, vs 50 MB for Go. We

by robfig 7y ago

It's hard to square these articles with the reality I see on the ground: our baseline memory usage for common types of Java service is 1 GB, vs 50 MB for Go. We do have a few mammoth servers at the top end in both languages though (e.g. 75 GB heaps) The deploy JARs have 100+ MB of class files, so perhaps it's a function of all the dependencies that you "need" for an enterprise Java program and not something more fundamental. These blog posts also present AOT as if its just another option you can toggle, while my impression is that it's incompatible with common Java libraries and requires swapping / maintaining a parallel compiler in the build toolchain, configuring separate rules to build it AOT, etc. I don't have actual experience with it though so I could be missing something.

mpweiher 7y ago
> It's hard to square these articles with the reality I see on the ground This is not a new phenomenon: back in 2000, the IBM System's Journal had a special issue on Java. Most of it was filled with articles detailing how amazingly awesome performance now was and getting better all the time. The last article in that issue was from IBM's San Francisco Project, the only one with real-world usage. Which reported much more, er, "mixed" results. More accurately: atrocious performance on their real-world tasks, and a lot of extra engineering effort to get it to work at least half reasonably. (I write a little more about this in "Jittterdämmerung" https://blog.metaobject.com/2015/10/jitterdammerung.html https://blog.metaobject.com/2015/10/jitterdammerung.html)
evacchi 7y ago
it's not "free", but there are frameworks that may help with that; e.g. Quarkus [1] provides a common EE/MicroProfile stack that does a lot of AoT codegen to get faster boot times and lower memory footprint; it is also compatible with GraalVM AoT, for even lower footprint and faster bootup. Compile times are still large, but you get the benefit of a familiar stack. Notice that, however, if your workload is throughput-intensive, you may still want to go with JVM mode (which is still pretty fast because of codegen) (full disclosure: I work at Red Hat and I have contributed to the project) [1] https://quarkus.io/ https://quarkus.io/
- vips7L 7y ago
  I've written a small project in Quarkus and I have enjoyed it. My main problems are the windows support still isn't there yet and the support for graal 19.3 (the one that supports java 11) isn't done yet either.
blinkingled 7y ago
> baseline memory usage for common types of Java service is 1 GB, vs 50 MB for Go. There's nothing inherent to the JVM that'd need 1GB of memory footprint - the jars are compressed and loaded on demand so unless your server needs all of them instantiated at once that doesn't explain the memory usage. Typically the region where class metadata are stored is the PermGen and it is not more than 256M for even the most heavyweight programs. My guess is that in your case the Xms (min heap) is set to 1Gb and so that's what you see as baseline. OpenJ9 is designed for lower memory usage than the hotspot VM and also supports AOT by the way if memory usage is your concern. Not disputing that the JVM needs more memory than anything else because it comes with a lot of features and it is very easy to use more but many J2EE apps can be made to work with a Gig and you do get a lot of features for that. And memory is cheap so you can decide what to optimize for.
- shizcakes 7y ago
  You're almost certainly right about Xms being set to 1GB. However, even if you can experimentally set it lower, the first time the JVM app hits GC pressure, the first thing anyone is going to try is bumping that back up over 1GB to give it some breathing room. Memory may be "cheap", but wasting 950MB of memory per process because the GC might flip out at the wrong time isn't cheap when you multiply it out to many processes. Also, I find the claim that the JVM starts and is running in 90ms very dubious. I would like to see an all-in timer, something like run each in a container, and see which completes first from container startup to shutdown.
  - blinkingled 7y ago
    > first time the JVM app hits GC pressure, the first thing anyone is going to try is bumping that back up over 1GB to give it some breathing room. Yeah but in that case your app legitimately needs that much. Not the JVM. Secondly unless your app is extremely performance sensitive setting Xms to lower value doesn't make the GC flip out - it's not as if the GC is going to collect repeatedly to stay within your lower bound - on the contrary it will expand the heap upwards until Xmx limit. Sure there will be a cost to expand the heap above Xms up to Xmx but it is not at all significant due to how clever the GC is.
shantly 7y ago
It’s been the case ever since I can remember that “Java’s basically as efficient as native, it’s super fast” but all the actual java software I encounter is a slowish, bloated memory hog. I don’t know why, but that’s how it is.
- yumaikas 7y ago
  A big part of that is that this benchmark is not representative of what general software is like, not in Go, not in Java. For one thing, most software isn't churning on an array of integers. It's allocating and managing various types of objects. I haven't written a ton of java, but I suspect that the various reflection-heavy frameworks also impose a lot of overhead into the resulting software that isn't necessarily inherently part of Java the language.
  - slaymaker1907 7y ago
    Reflection actually isn’t too inefficient, especially compared to using dynamic languages. I think a lot of the inefficiency is a bunch of small inefficiencies which combine together to make it very heavyweight.
    - chrisseaton 7y ago
      > Reflection actually isn’t too inefficient My understanding is that it's completely uncached. It couldn't be any less efficient.
      - kasperni 7y ago
        The overhead of, for example, invoking a method via reflection is measured in single-digit nanoseconds. Still not as fast as direct invocation, but unlikely to be a bottleneck unless you perform 10-of-millions of them per second.
        LgWoodenBadger 7y ago
        Actually finding the method via reflection is the time consumer. Once you have the method reference, actually invoking it isn’t any different than a regular method.
walkingolof 7y ago
Isn't one difference also between the JVM heap and the Go heap that the former allocates upfront, while Go allocates when it needs more?
- sorokod 7y ago
  You may be interested in this: https://blog.golang.org/ismmkeynote https://blog.golang.org/ismmkeynote
  - walkingolof 7y ago
    Thanks, seems like MaxHeap Is yet to be released feature of Go.
nradov 7y ago
If you want to square it with what you see on the ground then you'll have to take a heap dump and analyze where the memory is actually being used. (The number of class files is unlikely to be a factor.)
royjacobs 7y ago
You are comparing apples to oranges, then. A modern Java framework like Spring Boot, Micronaut or Quarkus can start up with a 20mb heap and will require a Docker container or jar in the range of tens of megabytes. Now, clearly to do more complex processing you'll want to give it more than tens of megabytes of heap, but if you are running Java applications that needs hundreds of megs of dependencies and 1gb of memory then you're probably running some legacy application server.
geodel 7y ago
> These blog posts also present AOT as if its just another option you can toggle, while my impression is that it's incompatible with common Java libraries I agree. I have experimented to make one of my smaller application (~20 Java classes) to Graal native image. After 4-5 days of trying I gave up. Most of the commonly used libraries like JDBC drivers, Kafka, log4j etc work in totally opposite way that Graal Native image like to work. The way I see it is J2EE all over again. Various vendors will find overall complexity of native images for practical or business Java apps as consulting or business opportunity. We can already see Quarkus, Micronaut and others are trying to offer native images for their framework. This would mean only a set of libraries would be compatible to native image in practical sense.
Twirrim 7y ago
The company I worked for a couple of jobs ago used to host approximately 130-140 java based web applications, all using Spring, on just 4 servers (primarily hosted on 2 of those) With only a few exceptions the java webapp was configured for a 256MB heap. Devs followed good practices to avoid creating excess garbage, and kept the number of libraries down to a minimum. They also struck a good balance between doing transformations in-memory, and on the database side. side note: There was _one_ web app that we had to host that wasn't developed in-house. It was absolutely god-awful, badly scaling, websphere based crap (I have no idea if Websphere is crap, but that application sure was bloated and underperforming). Websphere was irritating to deal with. What they did with it was worse. We eventually got the source code from them so we could fix it up, after a bunch of legal wrangles. They had 4 different custom created ORMs in it, each clearly developed by a different dev. Then for reports they never used the ORM, instead they'd make a "one query spits out the final report" query. Those queries would often be 30 - 40 subselects deep. `SELECT thing FROM foo WHERE bar in (SELECT bars IN monkey WHERE ....` and so on down the line.
- an_d_rew 7y ago
  It is absolutely possible to write amazingly performant Java / JVM applications, and absolutely possible to write horrible blobby slow Go applications. But given most of the standard corporate devs that I've worked with, it is infinitely easier to write crappy Java and not-bad Go than the other way around. Sadly.
  - ivan_gammel 7y ago
    It's also way easier to find a Java developer. Go is still in early adoption stage and people don't start programming on Go because they've decided to change their job from marketing to IT. As soon as any platform becomes popular the volume of crappy code increases dramatically.
  - andrewprock 7y ago
    Just wait until enterprise Go is released.
edem 7y ago
I don't know what are you using but I've built apps using Spring (known as a resource hug) which take up 10MB of memory and start up super fast. You only need some fine tuning to get these numbers. Alternatively you can also use things like Quarkus: https://quarkus.io/ https://quarkus.io/ which are hilariously cheap to operate and can work with Lambda as well.
- ptx 7y ago
  To get super-fast Spring start-up, do you have to turn off classpath scanning? That seems like it would always inherently be slow. What else do you tune?
  - edem 7y ago
    you just fine tune the jvm settings and override autoconfiguration with specific config classes.
imtringued 7y ago
It's a problem that is inherent in the way the JVM's GCs work. They never release memory even if you have low heap usage. I have a micronaut based microservice. Visual VM reports less than 30MB memory usage. Sometimes I get request spikes and it goes to 200MB. It will allocate 200MB from the OS and once the spike is gone it will hang onto the 200MB. There is also a fixed overhead. Even if I set the heap maximum to 30MB at the cost of running out of memory during spikes the total memory used by the JVM is still around 130MB. If you want to run 10 microservices that each do very little then you will need anywhere from 1.3GB to 3.3GB (as soon as you have one traffic spike and then never again). With go you could probably get away with just 300MB and during temporary traffic spikes your maximum will be 2.3GB at most but it is unlikely that all your microservices will spike at the same time so you will need significantly less than 2.3GB.
- miskin 7y ago
  It depends on Java version that you use. Recent GC returns unused memory back to OS [1] and class data sharing can speed up startup time of your microservices and save memory as shared classes are loaded only once for all JVMs [2] [1] https://openjdk.java.net/jeps/346 https://openjdk.java.net/jeps/346 [2] https://openjdk.java.net/jeps/310 https://openjdk.java.net/jeps/310
sreque 7y ago
The problem is enterprise Java and the Java community, not the jvm. If people wrote more god-like code in Java, we'd have much leaner Java processes. Instead, we have heavyweight, performance-insensitive frameworks like spring and hibernate used everywhere when they shouldn't be.
ignoramous 7y ago
> These blog posts also present AOT as if its just another option you can toggle, while my impression is that it's incompatible with common Java libraries and requires swapping / maintaining a parallel compiler in the build toolchain, configuring separate rules to build it AOT, etc. No, the blog post doesn't, and I quote: "Java AOT comes at a price. Huge compilation times (may slow-down your CI/CD pipeline). Very limited reflection API, limiting the usage of several frameworks or requiring of extra, complex, configuration: JPA, Spring... (This aspect will be treated in a future blog post)."
moomin 7y ago
I mean, that’s all well and good, but the article is about short-lived processes and you seem to be talking “enterprise” long-lived processes.
nine_k 7y ago
At one of my recent employers, we wondered why some of our services consume whopping 200 MB when a normal service would run on 50 MB. A service requiring 1-2 GB would be considered a behemoth. Unless your code is extremely complicated, or you are running a Jack-of-all-trades monolith, the only way to reasonably consume gobs of RAM is to keep an in-memory database / cache. This trades RAM consumption for speed. This is what e.g. IntelliJ does. Running a heap larger than you need slows you down in the long term, because GC over a large heap consumes more time and CPU.
c048 7y ago
Look into Micronauts and GraalVM. Simple apps have shown to have startup times of sub 20 ms and memory footprints als low as 18 mb. It's simply not fair of a comparison to GO when you drag in heavy frameworks like Spring, Hibernate, etc... . I'd also like to add that even without GraalVM and Micronauts, an app that requires 1 Gig of memory seems to be indicative of its architecture and not the choice of language.
- nobleach 7y ago
  Micronaut and Quarkus are fantastic! Those startup times are exactly what you're looking for if you want to run a cloud function or lambda. I will say, that for a standard microservice, where startup time is not as important, I'd not compile to GraalVM. The cpu/memory overhead to create a GraalVM image is pretty intense. Furthermore, GraalVM doesn't optimize as it warms up like HotSpot does. That need may depend on your use case. I'm currently running Micronaut services in production that take around 5 seconds to start on JDK 1.8. The Spring Boot equivalent..... well.... 10 times that amount of time maybe?

by robfig 7y ago

mpweiher 7y ago

> It's hard to square these articles with the reality I see on the ground This is not a new phenomenon: back in 2000, the IBM System's Journal had a special issue on Java. Most of it was filled with articles detailing how amazingly awesome performance now was and getting better all the time. The last article in that issue was from IBM's San Francisco Project, the only one with real-world usage. Which reported much more, er, "mixed" results. More accurately: atrocious performance on their real-world tasks, and a lot of extra engineering effort to get it to work at least half reasonably. (I write a little more about this in "Jittterdämmerung" https://blog.metaobject.com/2015/10/jitterdammerung.html https://blog.metaobject.com/2015/10/jitterdammerung.html)

evacchi 7y ago

it's not "free", but there are frameworks that may help with that; e.g. Quarkus [1] provides a common EE/MicroProfile stack that does a lot of AoT codegen to get faster boot times and lower memory footprint; it is also compatible with GraalVM AoT, for even lower footprint and faster bootup. Compile times are still large, but you get the benefit of a familiar stack. Notice that, however, if your workload is throughput-intensive, you may still want to go with JVM mode (which is still pretty fast because of codegen) (full disclosure: I work at Red Hat and I have contributed to the project) [1] https://quarkus.io/ https://quarkus.io/

vips7L 7y ago
I've written a small project in Quarkus and I have enjoyed it. My main problems are the windows support still isn't there yet and the support for graal 19.3 (the one that supports java 11) isn't done yet either.

blinkingled 7y ago

> baseline memory usage for common types of Java service is 1 GB, vs 50 MB for Go. There's nothing inherent to the JVM that'd need 1GB of memory footprint - the jars are compressed and loaded on demand so unless your server needs all of them instantiated at once that doesn't explain the memory usage. Typically the region where class metadata are stored is the PermGen and it is not more than 256M for even the most heavyweight programs. My guess is that in your case the Xms (min heap) is set to 1Gb and so that's what you see as baseline. OpenJ9 is designed for lower memory usage than the hotspot VM and also supports AOT by the way if memory usage is your concern. Not disputing that the JVM needs more memory than anything else because it comes with a lot of features and it is very easy to use more but many J2EE apps can be made to work with a Gig and you do get a lot of features for that. And memory is cheap so you can decide what to optimize for.

shizcakes 7y ago
You're almost certainly right about Xms being set to 1GB. However, even if you can experimentally set it lower, the first time the JVM app hits GC pressure, the first thing anyone is going to try is bumping that back up over 1GB to give it some breathing room. Memory may be "cheap", but wasting 950MB of memory per process because the GC might flip out at the wrong time isn't cheap when you multiply it out to many processes. Also, I find the claim that the JVM starts and is running in 90ms very dubious. I would like to see an all-in timer, something like run each in a container, and see which completes first from container startup to shutdown.
- blinkingled 7y ago
  > first time the JVM app hits GC pressure, the first thing anyone is going to try is bumping that back up over 1GB to give it some breathing room. Yeah but in that case your app legitimately needs that much. Not the JVM. Secondly unless your app is extremely performance sensitive setting Xms to lower value doesn't make the GC flip out - it's not as if the GC is going to collect repeatedly to stay within your lower bound - on the contrary it will expand the heap upwards until Xmx limit. Sure there will be a cost to expand the heap above Xms up to Xmx but it is not at all significant due to how clever the GC is.

shantly 7y ago

It’s been the case ever since I can remember that “Java’s basically as efficient as native, it’s super fast” but all the actual java software I encounter is a slowish, bloated memory hog. I don’t know why, but that’s how it is.

yumaikas 7y ago
A big part of that is that this benchmark is not representative of what general software is like, not in Go, not in Java. For one thing, most software isn't churning on an array of integers. It's allocating and managing various types of objects. I haven't written a ton of java, but I suspect that the various reflection-heavy frameworks also impose a lot of overhead into the resulting software that isn't necessarily inherently part of Java the language.
- slaymaker1907 7y ago
  Reflection actually isn’t too inefficient, especially compared to using dynamic languages. I think a lot of the inefficiency is a bunch of small inefficiencies which combine together to make it very heavyweight.
  - chrisseaton 7y ago
    > Reflection actually isn’t too inefficient My understanding is that it's completely uncached. It couldn't be any less efficient.
    - kasperni 7y ago
      The overhead of, for example, invoking a method via reflection is measured in single-digit nanoseconds. Still not as fast as direct invocation, but unlikely to be a bottleneck unless you perform 10-of-millions of them per second.
      - LgWoodenBadger 7y ago
        Actually finding the method via reflection is the time consumer. Once you have the method reference, actually invoking it isn’t any different than a regular method.

walkingolof 7y ago

Isn't one difference also between the JVM heap and the Go heap that the former allocates upfront, while Go allocates when it needs more?

sorokod 7y ago
You may be interested in this: https://blog.golang.org/ismmkeynote https://blog.golang.org/ismmkeynote
- walkingolof 7y ago
  Thanks, seems like MaxHeap Is yet to be released feature of Go.

nradov 7y ago

If you want to square it with what you see on the ground then you'll have to take a heap dump and analyze where the memory is actually being used. (The number of class files is unlikely to be a factor.)

royjacobs 7y ago

You are comparing apples to oranges, then. A modern Java framework like Spring Boot, Micronaut or Quarkus can start up with a 20mb heap and will require a Docker container or jar in the range of tens of megabytes. Now, clearly to do more complex processing you'll want to give it more than tens of megabytes of heap, but if you are running Java applications that needs hundreds of megs of dependencies and 1gb of memory then you're probably running some legacy application server.

geodel 7y ago

> These blog posts also present AOT as if its just another option you can toggle, while my impression is that it's incompatible with common Java libraries I agree. I have experimented to make one of my smaller application (~20 Java classes) to Graal native image. After 4-5 days of trying I gave up. Most of the commonly used libraries like JDBC drivers, Kafka, log4j etc work in totally opposite way that Graal Native image like to work. The way I see it is J2EE all over again. Various vendors will find overall complexity of native images for practical or business Java apps as consulting or business opportunity. We can already see Quarkus, Micronaut and others are trying to offer native images for their framework. This would mean only a set of libraries would be compatible to native image in practical sense.

Twirrim 7y ago

The company I worked for a couple of jobs ago used to host approximately 130-140 java based web applications, all using Spring, on just 4 servers (primarily hosted on 2 of those) With only a few exceptions the java webapp was configured for a 256MB heap. Devs followed good practices to avoid creating excess garbage, and kept the number of libraries down to a minimum. They also struck a good balance between doing transformations in-memory, and on the database side. side note: There was _one_ web app that we had to host that wasn't developed in-house. It was absolutely god-awful, badly scaling, websphere based crap (I have no idea if Websphere is crap, but that application sure was bloated and underperforming). Websphere was irritating to deal with. What they did with it was worse. We eventually got the source code from them so we could fix it up, after a bunch of legal wrangles. They had 4 different custom created ORMs in it, each clearly developed by a different dev. Then for reports they never used the ORM, instead they'd make a "one query spits out the final report" query. Those queries would often be 30 - 40 subselects deep. `SELECT thing FROM foo WHERE bar in (SELECT bars IN monkey WHERE ....` and so on down the line.

an_d_rew 7y ago
It is absolutely possible to write amazingly performant Java / JVM applications, and absolutely possible to write horrible blobby slow Go applications. But given most of the standard corporate devs that I've worked with, it is infinitely easier to write crappy Java and not-bad Go than the other way around. Sadly.
- ivan_gammel 7y ago
  It's also way easier to find a Java developer. Go is still in early adoption stage and people don't start programming on Go because they've decided to change their job from marketing to IT. As soon as any platform becomes popular the volume of crappy code increases dramatically.
- andrewprock 7y ago
  Just wait until enterprise Go is released.

edem 7y ago

I don't know what are you using but I've built apps using Spring (known as a resource hug) which take up 10MB of memory and start up super fast. You only need some fine tuning to get these numbers. Alternatively you can also use things like Quarkus: https://quarkus.io/ https://quarkus.io/ which are hilariously cheap to operate and can work with Lambda as well.

ptx 7y ago
To get super-fast Spring start-up, do you have to turn off classpath scanning? That seems like it would always inherently be slow. What else do you tune?
- edem 7y ago
  you just fine tune the jvm settings and override autoconfiguration with specific config classes.

imtringued 7y ago

It's a problem that is inherent in the way the JVM's GCs work. They never release memory even if you have low heap usage. I have a micronaut based microservice. Visual VM reports less than 30MB memory usage. Sometimes I get request spikes and it goes to 200MB. It will allocate 200MB from the OS and once the spike is gone it will hang onto the 200MB. There is also a fixed overhead. Even if I set the heap maximum to 30MB at the cost of running out of memory during spikes the total memory used by the JVM is still around 130MB. If you want to run 10 microservices that each do very little then you will need anywhere from 1.3GB to 3.3GB (as soon as you have one traffic spike and then never again). With go you could probably get away with just 300MB and during temporary traffic spikes your maximum will be 2.3GB at most but it is unlikely that all your microservices will spike at the same time so you will need significantly less than 2.3GB.

miskin 7y ago
It depends on Java version that you use. Recent GC returns unused memory back to OS [1] and class data sharing can speed up startup time of your microservices and save memory as shared classes are loaded only once for all JVMs [2] [1] https://openjdk.java.net/jeps/346 https://openjdk.java.net/jeps/346 [2] https://openjdk.java.net/jeps/310 https://openjdk.java.net/jeps/310

sreque 7y ago

The problem is enterprise Java and the Java community, not the jvm. If people wrote more god-like code in Java, we'd have much leaner Java processes. Instead, we have heavyweight, performance-insensitive frameworks like spring and hibernate used everywhere when they shouldn't be.

ignoramous 7y ago

> These blog posts also present AOT as if its just another option you can toggle, while my impression is that it's incompatible with common Java libraries and requires swapping / maintaining a parallel compiler in the build toolchain, configuring separate rules to build it AOT, etc. No, the blog post doesn't, and I quote: "Java AOT comes at a price. Huge compilation times (may slow-down your CI/CD pipeline). Very limited reflection API, limiting the usage of several frameworks or requiring of extra, complex, configuration: JPA, Spring... (This aspect will be treated in a future blog post)."

moomin 7y ago

I mean, that’s all well and good, but the article is about short-lived processes and you seem to be talking “enterprise” long-lived processes.

nine_k 7y ago

At one of my recent employers, we wondered why some of our services consume whopping 200 MB when a normal service would run on 50 MB. A service requiring 1-2 GB would be considered a behemoth. Unless your code is extremely complicated, or you are running a Jack-of-all-trades monolith, the only way to reasonably consume gobs of RAM is to keep an in-memory database / cache. This trades RAM consumption for speed. This is what e.g. IntelliJ does. Running a heap larger than you need slows you down in the long term, because GC over a large heap consumes more time and CPU.

c048 7y ago

Look into Micronauts and GraalVM. Simple apps have shown to have startup times of sub 20 ms and memory footprints als low as 18 mb. It's simply not fair of a comparison to GO when you drag in heavy frameworks like Spring, Hibernate, etc... . I'd also like to add that even without GraalVM and Micronauts, an app that requires 1 Gig of memory seems to be indicative of its architecture and not the choice of language.

nobleach 7y ago
Micronaut and Quarkus are fantastic! Those startup times are exactly what you're looking for if you want to run a cloud function or lambda. I will say, that for a standard microservice, where startup time is not as important, I'd not compile to GraalVM. The cpu/memory overhead to create a GraalVM image is pretty intense. Furthermore, GraalVM doesn't optimize as it warms up like HotSpot does. That need may depend on your use case. I'm currently running Micronaut services in production that take around 5 seconds to start on JDK 1.8. The Spring Boot equivalent..... well.... 10 times that amount of time maybe?