6 ms·

That's not always true. Most of the time when Java uses a ton of memory it's because people use the default memory settings. If you tell Java to use up to 90% o

That's not always true. Most of the time when Java uses a ton of memory it's because people use the default memory settings. If you tell Java to use up to 90% of system RAM, it will. Garbage collection is expensive so it will delay until memory is depleted. This is a different GC design than V8 and Go, which use older collector designs with high overhead. They need to collect very frequently because their stop the world pauses get longer with more garbage. Javas new collectors are near constant time, even with terabytes of garbage, so it's much more efficient to wait until the heap builds up and collect much less frequently. Ironically, Java appears to use tons of ram because it has better garbage collectors. When you configure Java GC to collect frequently, it turns out Java uses 2-3X less ram than JS, and far less than Python, Ruby, etc. It does use more than Go, about 2X. But the point is it uses a lot less ram than most other popular languages for web dev. Unfortunately the "Java uses too much ram" is used in defence of using things like JS and interpreted languages, when in actually it uses much less if you configure it to.

philipkglass 6y ago
I'll shamefully admit that I have been running and writing JVM based services for years and I didn't know this. I thought that the fixed memory overhead for a simple JVM service was simply higher than with CPython as a fact of life. There are times when I'd happily trade more frequent GC pauses for a smaller per-process memory footprint. How do you find a reasonably small Xmx that doesn't lead to OutOfMemoryError exceptions?
- spullara 6y ago
  Depends on your application and how much memory you actually need. I would also use ZGC as it returns unused memory to the system and low ms maximum pause times for terabytes of heap.
- throwaway189262 6y ago
  > How do you find a reasonably small Xmx that doesn't lead to OutOfMemoryError exceptions? That's tough to figure out. In new versions of Java, I think 14+, if you use ZGC collector it will return unused memory to the OS. Memory options vary depending on collector, but new versions of ZGC support "soft max" heap size and uncommit. Together it might be close to what you're looking for https://malloc.se/blog/zgc-softmaxheapsize https://malloc.se/blog/zgc-softmaxheapsize I should mention the GC situation was worse until the last few years. Until ZGC and Shenandoah came around, Java still didn't collect frequently but when it did there were long pauses. This is what V8 and Go's collectors were designed to avoid. They have more overhead from collecting frequently, but low pauses. With the new Java collectors you get the best of both worlds. You can set heap ratios and such for older collectors to decrease Java memory use with those, but IMO you're better off using ZGC and uncommit these days
  - philipkglass 6y ago
    Thanks for the advice. I'm stuck on 11 until the next LTS release but I'll definitely be trying ZGC next year. (ZGC is in 11 but still marked experimental, and I'm conservative about making changes.)
    - throwaway189262 6y ago
      That's probably a good decision. When ZGC was first released it didn't support class unloading, including in 11. This will lead to puzzling memory leaks in applications that generate a lot of code at runtime. Newer versions of ZGC support class unloading and have some other performance optimizations
    - sideeffffect 6y ago
      Shenandoah might help you. See my other comment for more details: https://news.ycombinator.com/item?id=24571054 https://news.ycombinator.com/item?id=24571054
  - topspin 6y ago
    > That's tough to figure out. Indeed. That fact obviates this option in most cases. You have to spend time tweaking obscure, unstable knobs (the X in Xmx means Oracle is free to alter its meaning at any time) and risk either a.) serious failures in production or b.) poor results because the conservative choices necessary to avoid 'a' achieved little improvement and you wasted your time. The real world for most enterprises is a vast heard of communicating components and toying with GC switches multiplied by N things is a nonstarter. So while you're technically correct that excessive memory use by conventional JVMs is "not always true," in practice you are wrong. That reality comes with a real cost that appears on a real bill every single month.
    - throwaway189262 6y ago
      The cost of setting a few command line args? They're not really obscure or unstable, just different depending on the GC you use. Turning on ZGC and setting the right options is like 4 command line flags. It's very easy, the reason it's not frequently done is that nobody reads documentation and it's not the default. Go's approach of minimal knobs leads to unfixable problems in production. Java gives more options to tune GC for your use case
deepakarora3 6y ago
All OK except when you need the same level of performance for those 0.001% of requests when the GC kicks in and takes the response time outside acceptable limits. Due to this reason alone, my company is planning to move off a popular Java based API gateway and to a C++ envoy side car implemented service mesh. And I am wondering if this is really worth it.
- zmmmmm 6y ago
  They should definitely investigate the new low-latency gc options in the JVM.
- throwaway189262 6y ago
  Consider trying ZGC first. The main selling point is low worst case pause time. I've seen some tests where P99.9 pause time was less than 5ms, vs several hundred for older collectors. In my limited tests I never saw a GC pause over 5ms. I was basically hammering a Spring Boot application with HTTP load tester.
  - pkolaczk 6y ago
    GC doesn't need to pause to have a negative effect on application latency. While not-pausing is generally a huge improvement (after several decades of GC development), now what about thrashing of CPU caches?
- sideeffffect 6y ago
  Shenandoah might help you. See my other comment for more details: https://news.ycombinator.com/item?id=24571054 https://news.ycombinator.com/item?id=24571054
zmmmmm 6y ago
Not sure that is all of it. You can try turning down the max total memory to eg: 64mb and see what you can still run. An awful lot of stuff just can't, while similar applications in conceptually higher level languages (eg: Python, PHP, etc) happily execute. The various factors I see are: - JVM overhead 10-15mb seems to be required just to get off the ground, which directly relates to the JVM replicating a bunch of OS functionality (inevitable tradeoff for portability) - Missing value types - any complex data structure ends up with big overhead from storing references - Stack frames - if you need a lot of threads you need a lot of stack, and a surprising number of threads run Some of this will get addressed in upcoming releases, will be interesting to see how it goes.
titzer 6y ago
> This is a different GC design than V8 and Go, which use older collector designs with high overhead. I think this information is a bit dated. Go has a highly advanced concurrent collector with very low pause times (~1-100μs). V8 also has incremental marking, concurrent marking, and parallel compaction. Its pause times are more like 100-1000μs. V8's GC has been tuned more and more to save memory (i.e. smaller heaps) because people have so many tabs open these days.
- throwaway189262 6y ago
  I don't agree with your pause time estimates. Most collectors are fast with small amounts of garbage in a small application. It's high allocation rates where GC time becomes a problem, and I don't see anything about Go or V8 designs to prevent very bad worst-case pause times. Uh, most of the rant about Go GC below is out of date, they made some huge improvements from 2016-now. I'm leaving it up because someone replied to it Unless something has changed in the last year or two, Go's GC is similar to Java's old CMS collector which is being deprecated. Go's GC is non generational and non compacting, both hallmarks of modern GC algorithms. It's not a modern "moving" collector. It also has several stop the world phases. It's basically a design from the 70's. The GC uses old simple algorithms because the team was under a time crunch when it was developed. This may have changed, but that's how it was in 2018-2019 timeframe. The pause times are short because GC runs very frequently. It has to because performance with large amounts of garbage is quite bad. This results in significant GC CPU overhead. I don't know much about V8 collector besides that it is generational and compacting, so more modern than Go. But it's still a "stop the world" design. In that regard it's still closest to Javas old CMS collector. Javas new collectors, ZGC and Shenandoah, both have near constant time stop the world phases. You can collect terabytes of garbage with only a few milliseconds pause. In V8 or Go, this would be many seconds pause time as the mark phase is "stop the world" in both. You can find benchmarks that show one way or the other, but in badly behaving or allocation heavy applications, Java's new collectors or older G1 will perform far better. V8 and Go are dishonest about their GC performance by showing average pause times with high collection frequency. The important GC cycles are the long ones, so you really want to measure worse case pause time under load. Under heavy load Go's design falls over. It's not compacting, not generational, and the mark phase pauses the application. IMO, it's just not a good GC. V8 is better, it is generational and compacting, but mark phase is still STW. Java's ZGC isn't generational but importantly, the mark and sweep phases don't stop execution. No matter how big your heap is and how much garbage, your GC pauses will be short
pkolaczk 6y ago
GC and memory settings are not the only cause. There are several other factors: * JVM overhead - all these advanced optimizing compilers and GCs have non-zero footprint. They need RAM to run their code and they need RAM to perform their tasks. * Compiled code cache - JVM keeps both the original bytecode and generated machine code in RAM. * OOP overhead - each object has 2 or 3 words of overhead for the object header vs zero in languages like C or Rust. Even when you don't need dynamic dispatch or object locking, you pay for it. * Inability to compose bigger structures other than by allocating separate objects and using pointers to reference them - these pointers need space and are not cheap on 64-bit architectures. This is going to probably partially improve with Valhalla, but at this point it is mostly guessing and it has been in development for years. * No support for packed arrays. * The smallest unit of loadable code is a class. If you needed a single function, the JVM loads a whole class containing it, and its required dependencies. This is not only bad for memory usage but also for startup time. Unless you pay a lot of attention, it is easy to load 80% of code in order to just display a help message (this is based on a real issue I worked on - I'm not making this up). Compare that to code in languages like C - the OS loads code that gets executed. And back to GC - I agree the default settings are often to blame, but there is a reason JVM defaults to using RAM so aggressively. GC becomes very inefficient when it doesn't have enough "room". And low pause GCs achieve their low pause goals by trading throughput. Switch from parallel STW GC to G1 and your maximum sustainable allocation rate goes down by a few times.
rcoveson 6y ago
When I hear "Java uses too much ram," lazy reclamation of the heap and other GC overhead are not the first things that come to my mind, personally. Java doesn't have a "struct". If you want to represent an array of 64-bit signed integers, Java has you covered with its primitive arrays. But if you want to represent an array of anything more interesting than that, (say, a tuple of a double and a long), you have to serialize and deserialize those objects to and from parallel primitive arrays or byte buffers. Because if you do the language-natural thing and use an Object array, you're paying a huge price in memory: 4 or 8 bytes per pointer in that array, plus a 16 byte Object header on each Object. And, of course, those Objects are all individual allocations, not necessarily contiguous. That's a lot of overhead! Of course, Java programmers concerned with memory usage don't put up with this. Lots of solutions have been devised. OpenHFT's Chronicle Values[0] is one example I came across recently. But this feels like fighting with the language compared to how easy it is to be efficient with memory in C. If you told a beginner C programmer to make an array of compound objects, it's not unlikely that their array will take up exactly as much space in memory as it intuitively seems like it should. (8 byte double + 8 byte long) * 100 values = 1600 bytes in C, no fuss. If you asked the average Java programmer for that they'd give you something that would take up 3 times as much memory. And because Java makes that behavior natural, it "uses too much ram". It doesn't matter as much that it's possible to convince it not too. 0. https://github.com/OpenHFT/Chronicle-Values https://github.com/OpenHFT/Chronicle-Values
- imtringued 6y ago
  What you are saying is true but even when you use optimized frameworks like Micronaut and stay below e.g. 50MB java heap usage you still end up using more RAM than what the Java heap says. Even the most optimized Java program on the planet will be beaten by a carelessly written C++ program. At least in terms of memory usage. I know it because I once wrote a React app with a C++ backend that also included sqlite. The C++ version needed 4MB of RAM at worst. Meanwhile the Java version didn't even include a database!
- pjmlp 6y ago
  Because fighting the language is worth it given the benefits that come with it. Plus many people overlook just writing a couple of native methods and be done with it. Somehow this is how people write "Python". Also Valhalla and Panama are around the corner.
imtringued 6y ago
I have never managed to get a JVM to use less than 100MB RAM. The application in question needed significantly less than 32MB of Java heap. Meanwhile equivalent lua programs can do just fine with 1MB RAM.

philipkglass 6y ago

I'll shamefully admit that I have been running and writing JVM based services for years and I didn't know this. I thought that the fixed memory overhead for a simple JVM service was simply higher than with CPython as a fact of life. There are times when I'd happily trade more frequent GC pauses for a smaller per-process memory footprint. How do you find a reasonably small Xmx that doesn't lead to OutOfMemoryError exceptions?

spullara 6y ago
Depends on your application and how much memory you actually need. I would also use ZGC as it returns unused memory to the system and low ms maximum pause times for terabytes of heap.
throwaway189262 6y ago
> How do you find a reasonably small Xmx that doesn't lead to OutOfMemoryError exceptions? That's tough to figure out. In new versions of Java, I think 14+, if you use ZGC collector it will return unused memory to the OS. Memory options vary depending on collector, but new versions of ZGC support "soft max" heap size and uncommit. Together it might be close to what you're looking for https://malloc.se/blog/zgc-softmaxheapsize https://malloc.se/blog/zgc-softmaxheapsize I should mention the GC situation was worse until the last few years. Until ZGC and Shenandoah came around, Java still didn't collect frequently but when it did there were long pauses. This is what V8 and Go's collectors were designed to avoid. They have more overhead from collecting frequently, but low pauses. With the new Java collectors you get the best of both worlds. You can set heap ratios and such for older collectors to decrease Java memory use with those, but IMO you're better off using ZGC and uncommit these days
- philipkglass 6y ago
  Thanks for the advice. I'm stuck on 11 until the next LTS release but I'll definitely be trying ZGC next year. (ZGC is in 11 but still marked experimental, and I'm conservative about making changes.)
  - throwaway189262 6y ago
    That's probably a good decision. When ZGC was first released it didn't support class unloading, including in 11. This will lead to puzzling memory leaks in applications that generate a lot of code at runtime. Newer versions of ZGC support class unloading and have some other performance optimizations
  - sideeffffect 6y ago
    Shenandoah might help you. See my other comment for more details: https://news.ycombinator.com/item?id=24571054 https://news.ycombinator.com/item?id=24571054
- topspin 6y ago
  > That's tough to figure out. Indeed. That fact obviates this option in most cases. You have to spend time tweaking obscure, unstable knobs (the X in Xmx means Oracle is free to alter its meaning at any time) and risk either a.) serious failures in production or b.) poor results because the conservative choices necessary to avoid 'a' achieved little improvement and you wasted your time. The real world for most enterprises is a vast heard of communicating components and toying with GC switches multiplied by N things is a nonstarter. So while you're technically correct that excessive memory use by conventional JVMs is "not always true," in practice you are wrong. That reality comes with a real cost that appears on a real bill every single month.
  - throwaway189262 6y ago
    The cost of setting a few command line args? They're not really obscure or unstable, just different depending on the GC you use. Turning on ZGC and setting the right options is like 4 command line flags. It's very easy, the reason it's not frequently done is that nobody reads documentation and it's not the default. Go's approach of minimal knobs leads to unfixable problems in production. Java gives more options to tune GC for your use case

deepakarora3 6y ago

All OK except when you need the same level of performance for those 0.001% of requests when the GC kicks in and takes the response time outside acceptable limits. Due to this reason alone, my company is planning to move off a popular Java based API gateway and to a C++ envoy side car implemented service mesh. And I am wondering if this is really worth it.

zmmmmm 6y ago
They should definitely investigate the new low-latency gc options in the JVM.
throwaway189262 6y ago
Consider trying ZGC first. The main selling point is low worst case pause time. I've seen some tests where P99.9 pause time was less than 5ms, vs several hundred for older collectors. In my limited tests I never saw a GC pause over 5ms. I was basically hammering a Spring Boot application with HTTP load tester.
- pkolaczk 6y ago
  GC doesn't need to pause to have a negative effect on application latency. While not-pausing is generally a huge improvement (after several decades of GC development), now what about thrashing of CPU caches?
sideeffffect 6y ago
Shenandoah might help you. See my other comment for more details: https://news.ycombinator.com/item?id=24571054 https://news.ycombinator.com/item?id=24571054

zmmmmm 6y ago

Not sure that is all of it. You can try turning down the max total memory to eg: 64mb and see what you can still run. An awful lot of stuff just can't, while similar applications in conceptually higher level languages (eg: Python, PHP, etc) happily execute. The various factors I see are: - JVM overhead 10-15mb seems to be required just to get off the ground, which directly relates to the JVM replicating a bunch of OS functionality (inevitable tradeoff for portability) - Missing value types - any complex data structure ends up with big overhead from storing references - Stack frames - if you need a lot of threads you need a lot of stack, and a surprising number of threads run Some of this will get addressed in upcoming releases, will be interesting to see how it goes.

titzer 6y ago

> This is a different GC design than V8 and Go, which use older collector designs with high overhead. I think this information is a bit dated. Go has a highly advanced concurrent collector with very low pause times (~1-100μs). V8 also has incremental marking, concurrent marking, and parallel compaction. Its pause times are more like 100-1000μs. V8's GC has been tuned more and more to save memory (i.e. smaller heaps) because people have so many tabs open these days.

throwaway189262 6y ago
I don't agree with your pause time estimates. Most collectors are fast with small amounts of garbage in a small application. It's high allocation rates where GC time becomes a problem, and I don't see anything about Go or V8 designs to prevent very bad worst-case pause times. Uh, most of the rant about Go GC below is out of date, they made some huge improvements from 2016-now. I'm leaving it up because someone replied to it Unless something has changed in the last year or two, Go's GC is similar to Java's old CMS collector which is being deprecated. Go's GC is non generational and non compacting, both hallmarks of modern GC algorithms. It's not a modern "moving" collector. It also has several stop the world phases. It's basically a design from the 70's. The GC uses old simple algorithms because the team was under a time crunch when it was developed. This may have changed, but that's how it was in 2018-2019 timeframe. The pause times are short because GC runs very frequently. It has to because performance with large amounts of garbage is quite bad. This results in significant GC CPU overhead. I don't know much about V8 collector besides that it is generational and compacting, so more modern than Go. But it's still a "stop the world" design. In that regard it's still closest to Javas old CMS collector. Javas new collectors, ZGC and Shenandoah, both have near constant time stop the world phases. You can collect terabytes of garbage with only a few milliseconds pause. In V8 or Go, this would be many seconds pause time as the mark phase is "stop the world" in both. You can find benchmarks that show one way or the other, but in badly behaving or allocation heavy applications, Java's new collectors or older G1 will perform far better. V8 and Go are dishonest about their GC performance by showing average pause times with high collection frequency. The important GC cycles are the long ones, so you really want to measure worse case pause time under load. Under heavy load Go's design falls over. It's not compacting, not generational, and the mark phase pauses the application. IMO, it's just not a good GC. V8 is better, it is generational and compacting, but mark phase is still STW. Java's ZGC isn't generational but importantly, the mark and sweep phases don't stop execution. No matter how big your heap is and how much garbage, your GC pauses will be short

pkolaczk 6y ago

GC and memory settings are not the only cause. There are several other factors: * JVM overhead - all these advanced optimizing compilers and GCs have non-zero footprint. They need RAM to run their code and they need RAM to perform their tasks. * Compiled code cache - JVM keeps both the original bytecode and generated machine code in RAM. * OOP overhead - each object has 2 or 3 words of overhead for the object header vs zero in languages like C or Rust. Even when you don't need dynamic dispatch or object locking, you pay for it. * Inability to compose bigger structures other than by allocating separate objects and using pointers to reference them - these pointers need space and are not cheap on 64-bit architectures. This is going to probably partially improve with Valhalla, but at this point it is mostly guessing and it has been in development for years. * No support for packed arrays. * The smallest unit of loadable code is a class. If you needed a single function, the JVM loads a whole class containing it, and its required dependencies. This is not only bad for memory usage but also for startup time. Unless you pay a lot of attention, it is easy to load 80% of code in order to just display a help message (this is based on a real issue I worked on - I'm not making this up). Compare that to code in languages like C - the OS loads code that gets executed. And back to GC - I agree the default settings are often to blame, but there is a reason JVM defaults to using RAM so aggressively. GC becomes very inefficient when it doesn't have enough "room". And low pause GCs achieve their low pause goals by trading throughput. Switch from parallel STW GC to G1 and your maximum sustainable allocation rate goes down by a few times.

rcoveson 6y ago

When I hear "Java uses too much ram," lazy reclamation of the heap and other GC overhead are not the first things that come to my mind, personally. Java doesn't have a "struct". If you want to represent an array of 64-bit signed integers, Java has you covered with its primitive arrays. But if you want to represent an array of anything more interesting than that, (say, a tuple of a double and a long), you have to serialize and deserialize those objects to and from parallel primitive arrays or byte buffers. Because if you do the language-natural thing and use an Object array, you're paying a huge price in memory: 4 or 8 bytes per pointer in that array, plus a 16 byte Object header on each Object. And, of course, those Objects are all individual allocations, not necessarily contiguous. That's a lot of overhead! Of course, Java programmers concerned with memory usage don't put up with this. Lots of solutions have been devised. OpenHFT's Chronicle Values[0] is one example I came across recently. But this feels like fighting with the language compared to how easy it is to be efficient with memory in C. If you told a beginner C programmer to make an array of compound objects, it's not unlikely that their array will take up exactly as much space in memory as it intuitively seems like it should. (8 byte double + 8 byte long) * 100 values = 1600 bytes in C, no fuss. If you asked the average Java programmer for that they'd give you something that would take up 3 times as much memory. And because Java makes that behavior natural, it "uses too much ram". It doesn't matter as much that it's possible to convince it not too. 0. https://github.com/OpenHFT/Chronicle-Values https://github.com/OpenHFT/Chronicle-Values

imtringued 6y ago
What you are saying is true but even when you use optimized frameworks like Micronaut and stay below e.g. 50MB java heap usage you still end up using more RAM than what the Java heap says. Even the most optimized Java program on the planet will be beaten by a carelessly written C++ program. At least in terms of memory usage. I know it because I once wrote a React app with a C++ backend that also included sqlite. The C++ version needed 4MB of RAM at worst. Meanwhile the Java version didn't even include a database!
pjmlp 6y ago
Because fighting the language is worth it given the benefits that come with it. Plus many people overlook just writing a couple of native methods and be done with it. Somehow this is how people write "Python". Also Valhalla and Panama are around the corner.

imtringued 6y ago

I have never managed to get a JVM to use less than 100MB RAM. The application in question needed significantly less than 32MB of Java heap. Meanwhile equivalent lua programs can do just fine with 1MB RAM.