9 ms·

Running Java in a Container

212 points by edw519 9y ago

Skunkleton 9y ago
Wasn't the original promise of the JVM to provide container like services to applications? This field is way outside where I normally work, I would love to hear some expert commentary about this.
- sk5t 9y ago
  It is perhaps doable by getting very intimate with Java's classloaders and security policies, but most people don't want to deal with that stuff--it's complex, not fun, not portable. That said, there are practical examples to be found in common servlet containers, with some shared libraries, but individual servlets mostly firewalled from each other.
  - cuspycode 9y ago
    Servlet containers have two levels of isolation: individual apps are mostly firewalled from each other as you say, but when running multiple servlet containers they are as isolated from each other as multiple docker processes are when it comes to shared libraries, security policies etc. JVM processes are easy to reason about at the OS level, while servlet applications are more complex. But we are comparing Docker with JVM, not Docker with e.g. Tomcat webapps.
- techman9 9y ago
  Sorta. The JVM provides portability but has less fine controls for things like disk, network and CPU isolation and limiting.
- kev009 9y ago
  You are effectively correct, and to take it a bit further the "VM" in JVM does mean the same thing as the VM in VMWare even if it feels a lot different in practice. If all your dependencies are JVM based and bundled into a JAR/WAR, OS containers aren't doing much for you.
  - weego 9y ago
    They're doing a ton for your CV though.
- szemet 9y ago
  Yes, but some JVM implemetations have(had?) more support for this use case than others: e.g. https://www.ibm.com/developerworks/library/j-multitenant-java/index.html https://www.ibm.com/developerworks/library/j-multitenant-jav...
- hibikir 9y ago
  It kind of does: Jave EE was all about having single Java Application Servers running on each host, and deploying applications/services on top. In practice every service was built by the same company though, so focused moved toward shared libraries in containers and things like that: Search for OSGI. The issue is both that the security concerns were way softer than they are today, and that all your dependencies better be deplorable in a JVM too. Modern ideas of having databases deployed along with the services that need them don't work quite as well as just using OS level virtualization. That said many a crufry old company still deploys hundreds of services to production by loading .war files into a cluster of servers running JBoss or WebLogic.
  - gunnihinn 9y ago
    > and that all your dependencies better be deplorable in a JVM too. Remember: Never use the standard library when a forgotten, half-assed CPAN module you can run via Perlito will kind of do.
  - timv 9y ago
    > your dependencies better be deplorable in a JVM too Sadly, many Java dependencies are quite deplorable.
- tytytytytytytyt 9y ago
  No more so than any other language with a VM, which typically isn't much at all.
- fulafel 9y ago
  This got me thinking about how it is on Clojure, and turns out there's a library for doing that: https://github.com/RutledgePaulV/clj-embed https://github.com/RutledgePaulV/clj-embed
- hinkley 9y ago
  The PR for memory protected preemotive multitasking operating systems still sounds remarkably similar to the container sale pitch today (at least the first half of it). I wonder what we’ll be championing in another 10 years.
- cuspycode 9y ago
  The JVM is a container by itself, and it has been able to isolate dependencies and memory requirements for many years before docker was conceived. The JVM doesn't do everything that docker does, for example it doesn't provide overlay file systems, but this is trivial to add via standard tools (e.g. aufs), and it also doesn't limit CPU usage per container but this is also trivial to add without having to deal with the extra complications of using docker. I think docker is great for non-JVM applications that run into conflicts with other applications, dependency-wise or resource-wise, but I don't see the point of using docker for pure Java applications. It just adds an unnecessary layer that provides no benefit. Edit: If the point is to encapsulate and isolate a Java application as much as possible, I would consider using a unikernel like OSv before considering Docker. This would be more efficient and also more secure.
Edmond 9y ago
The real killer app would be the ability to fully containerize all jvm instances running on any given box. It will be a setup where one jvm instance on the host basically serves the role of "master" in terms of class data and shared object loading while each container instance uses its memory allotment only for running computations specific to the application in that container while sharing memory objects with other containers as much as possible. It is possible to do something similar at the moment but it requires going through a hodge-podge of painful hacks. A seamless solution to this would basically make the jvm an out of the box poor man's polyglot PaaS platform.
- jjtheblunt 9y ago
  Aren't you describing every OS since Solaris 2.x that uses shared pages across processes as well as Solaris 2.x did?
  - Edmond 9y ago
    My OS internals knowledge is rusty so I am not sure they are quite the same but likely similar. If you think of a typical jvm application (true for non-jvm apps as well), a significant chunk of class data will be shared since apps are typically using the same libraries (with deltas in versions), allowing easy reuse of class data across all container instances on a host would be a major scalability advance.
    - karambahh 9y ago
      There is a java feature called class data sharing that does that since Java 5 but I've never seen it in the wild.[0] Java 10 will bring AppCDS via JEP 310[1] [0]https://fosdem.org/2018/schedule/event/class_data_sharing https://fosdem.org/2018/schedule/event/class_data_sharing [1]http://openjdk.java.net/jeps/310 http://openjdk.java.net/jeps/310
- bokchoi 9y ago
  There was a JSR-121 Application Isolation proposal, that sounds similar to what you're talking about. I'm not sure why it didn't gain traction: http://www.oracle.com/technetwork/articles/javase/mvm-141094.html http://www.oracle.com/technetwork/articles/javase/mvm-141094... https://www.jcp.org/en/jsr/detail?id=121 https://www.jcp.org/en/jsr/detail?id=121
kev009 9y ago
It would be nice to have some kind of cooperative API for runtimes like Java, Go etc to where the OS or container manager can provide watermark hints for the heap size and when to run collections.
- techman9 9y ago
  How does the JVM determine the number of cores on a Linux system by default?
  - kev009 9y ago
    I don't know the precise answer off hand, but probably in an OS dependent code path in OpenJDK. I know for instance OpenJDK uses the global sysctl API to get total RAM on a FreeBSD machine. To use jails as multi-tenancy system you therefore want to fake that API out to the resource controled max for the jail.
  - joerg84 9y ago
    One of the authors here: This is the OpenJDK 8 implementation: http://hg.openjdk.java.net/jdk8/jdk8/hotspot/file/tip/src/os/linux/vm/os_linux.cpp#l4967 http://hg.openjdk.java.net/jdk8/jdk8/hotspot/file/tip/src/os...
- sitkack 9y ago
  Based on request traffic or maximum memory? For the JVM there is -Xmx512m to set the max heap size (512MB in this case). Or are you thinking of something more organic so individual containers could overcommit on memory, and the OOM killer would first send a "collect" signal to all the containers that could GC (and also give the memory back).
  - kev009 9y ago
    Yes when I say cooperative I mean nice processes are opting in to being told what and when to do something (change heap size, run a light or heavy collection, etc) apart from their own internal view of affairs.
xcq1 9y ago
Anyone here actually use Java containers in production? Sadly the article mentions very little in terms of practical advice. We've tried running some small Java 8 Spring Boot containers in Kubernetes which are configured to use max ~50M heap and ~150M total off-heap yet decide to use in excess of double that memory so we end up with either a lot of OOMKills or overly large memory limits.
- flipp3r 9y ago
  Where I work most of our dockerized services are Spring Boot in Kubernetes, they do need more memory than what you've posted and generally run with about 300M ~ 600M usage depending on what they need to do. You can also use smaller frameworks (Vertx? Javalin? possibly Spring Boot 2). I hope that with Java 9 we won't see this amount of memory usage anymore, however our organisation isn't there yet though.
- yazr 9y ago
  What are the other common JVM Memory offenders ?! (Electron i guess being the obvious JS-to-native example) (And isn't Sprint Boot the light-weight solution to the Sprint memory problem? /s)
- maaaats 9y ago
  If you ditch Spring Boot and wire Spring yourself the startup-time and footprint should be smaller. But 250M may be a bit optimistic.
- jacques_chester 9y ago
  Spring Boot is, by design, eager to load everything it sense you might need. Most of the memory usage is not Spring itself, it's the libraries it's pulling in on your behalf. In Spring Boot 2 I'm told you can can use functional beans to cut down on RAM usage. Not sure how it works. But really, it comes down to deciding if you need a dependency or not. As is normal for questions about Spring Boot performance, Dave Syer has done a lot of investigating: https://github.com/dsyer/spring-boot-memory-blog https://github.com/dsyer/spring-boot-memory-blog
- oftenwrong 9y ago
ognyankulev 9y ago
The just-released Java 10 has improvements in the container case: https://www.opsian.com/blog/java-on-docker/ https://www.opsian.com/blog/java-on-docker/ edit: at the end of the article, it is acknowledged that there are improvements in Java 10 and the following link is provided: https://bugs.openjdk.java.net/browse/JDK-8146115 https://bugs.openjdk.java.net/browse/JDK-8146115
- ChristianGeek 9y ago
  There was a discussion about this yesterday on Reddit: https://reddit.com/r/java/comments/85t7dt/java_on_docker_will_no_longer_suck_improvements/ https://reddit.com/r/java/comments/85t7dt/java_on_docker_wil...
- joerg84 9y ago
  IMO these are a great improvement but one still needs to be aware of the underlying container limits. E.g., understanding why setting MaxRAMFraction=1 might not be a good idea as the container might get killed...
mcgin 9y ago
These options were also backported to Java 8 in update 131 - https://blogs.oracle.com/java-platform-group/java-se-support-for-docker-cpu-and-memory-limits https://blogs.oracle.com/java-platform-group/java-se-support...
r4um 9y ago
Not just the JVM, lot of libraries build thread pools based on number of processors and some of them even hard code the multipliers (X the number of processors etc). Setting up each one becomes a lot of work fast, so we wrote a LD_PRELOAD[1] hook that overrides sysconf[1] _NC_PROCESSORS* call to get number of processors availabale/online to a certain specified value and is baked in the docker image by default during builds. [1] http://man7.org/linux/man-pages/man3/sysconf.3.html http://man7.org/linux/man-pages/man3/sysconf.3.html
- _pmf_ 9y ago
  That's horrible and irresponsible; I love it.
- MaxBarraclough 9y ago
  Can we expect that kind of horror to fade away as Java evolves? In the .Net world there's the ThreadPool class to manage the single process-wide thread-pool, and the Task class to enqueue and orchestrate concurrent jobs (it uses ThreadPool and almost completely abstracts away the thread management). (You could write your own thread-pool, of course, but for most code that wouldn't make sense.) As I understand it, the JVM is rather behind in this department. (Not to mention async/await.)
  - jacques_chester 9y ago
    Java has these concepts too (ExecutorService since at least Java 5, circa 2004). The problem is not with the JDK libraries, it's been in the JVM's assumption that it's running on bare metal or a VM. Linux containers leak information about the "true" environment in a way that upset JVM assumptions before 9 and 10.
  - badpun 9y ago
    > In the .Net world there's the ThreadPool class to manage the single process-wide thread-pool, and the Task class to enqueue and orchestrate concurrent jobs (it uses ThreadPool and almost completely abstracts away the thread management). > (You could write your own thread-pool, of course, but for most code that wouldn't make sense.) Java had thread pools and tasks since 2004 (https://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/ThreadPoolExecutor.html https://docs.oracle.com/javase/1.5.0/docs/api/java/util/conc...). For typical Java programs, no one is writing their own thread pool functionality as far as I know.
jrudolph 9y ago
This. I've seen container builds for Java applications broken in the described ways fail in production over and over again. Just because Docker makes building an image very easy doesn't mean that you will end up with a production ready image using three lines of code in a Dockerfile. Most of the time people don't even bother to use a non-root user to execute the jvm in their container... That's why I feel platforms like Cloud Foundry are a much better fit for teams that don't have tons of container experience but want to get the benefits of a containerized runtime. The CF java buildpack[1] for example automatically handles OOM and heap settings calculation while building your application container. disclaimer: co-founder at meshcloud, we offer a public cloud service for Cloud Foundry and K8s hosted in german datacenters. [1] https://github.com/cloudfoundry/java-buildpack
- jacques_chester 9y ago
  In particular, the Java buildpack uses a component developed entirely to calculate JVM settings. I am not sure what the plans are for Java 9 and 10 yet. Ben Hale works on JBP pretty much fulltime and the Spring team tend to experiment pretty early on JDKs. So I can't see it falling far behind. [0] https://github.com/cloudfoundry/java-buildpack-memory-calculator https://github.com/cloudfoundry/java-buildpack-memory-calcul...
- nerdwaller 9y ago
  I wrote up my experience[0] on containerizing JVM based applications a bit ago using the cloud foundry java buildpack’s memory calculator. Fortunately the JVM now has a way to respect cgroup memory[1] making it a bit simpler. [0]: https://medium.com/@matt_rasband/dockerizing-a-spring-boot-application-6ec9b9b41faf https://medium.com/@matt_rasband/dockerizing-a-spring-boot-a... [1]: https://twitter.com/codepitbull/status/934384652806221825?s=20 https://twitter.com/codepitbull/status/934384652806221825?s=...
mboehm 9y ago
>> But the in case of containers with a hard memory limit, the entire container will simply be killed without warning. At least in my experience that is not true. I quite often run into this issue and the OOM-killer will only kill one of the processes inside the container, not the entire container. >> The same is true for default memory limits. The JVM looks at the host overall memory and uses that to set its defaults. Well, I guess if you launch a JVM anywhere without setting appropriate memory settings, you are doing something fundamentally wrong.
- rad_gruchalski 9y ago
  >> At least in my experience that is not true. I quite often run into this issue and the OOM-killer will only kill one of the processes inside the container, not the entire container. Hence one does not run multiple processes in the container or / and handle crashes of potential child processes correctly.
  - mboehm 9y ago
    Well, I guess, sometimes you need to break the rules. But you are right.
matttproud 9y ago
We created a model and applied unsupervised ML to tuning the JVM for execution in containers and eventually open sourced the project: https://github.com/sladeware/groningen https://github.com/sladeware/groningen Unfortunately the experiment state persistence management capability is broken.
allsunny 9y ago
We use shaded/fat jars where I work because it's easier to ensure we never run into issues with missing dependencies. It does however come at the cost of longer build times. I've been hoping that Docker might be able to help with this by allowing us to keep all of the dependencies in the container base image and then just add the new class files in the build process. Is that a reasonable assumption?
rdsubhas 9y ago
Fabric8 has really good base java images[1] with a script that simply sets environment variables with right GC and CPU parameters before launching Java, with nice sane defaults. Heavily encourage anyone running Java in containers to use their base image, or for larger organizations to create standard base image dockerfiles that set these JVM envvar parameters. A simple contract is: ENTRYPOINT belongs to the base image, CMD belongs to downstream application images (unless something else essential). Just don't use vanilla "FROM openjdk:8-jre" and expect it to work. That's the worst way to kill application performance and reliability in a container. 1: https://github.com/fabric8io-images/java/blob/master/images/centos/openjdk8/jre/run-java.sh#L3 https://github.com/fabric8io-images/java/blob/master/images/...
- organsnyder 9y ago
  +1 for Fabric8. It makes running Java in containers much more pleasant, and provides a complete opinionated ecosystem, if that's desired. I work at an enterprise shop that is a Red Hat customer (so we get commercial support on this stuff), and it has made our lives much easier in many respects.
- deleted 9y ago
  [deleted]
- deepakhj 9y ago
  Jdk8u131+ and 9 support detecting cpu and memory limits to set heap and core usage. https://blogs.oracle.com/java-platform-group/java-se-support-for-docker-cpu-and-memory-limits https://blogs.oracle.com/java-platform-group/java-se-support...
  - rdsubhas 9y ago
    We tried those flags in the beginning since it was introduced in docker openjdk image[1]. When we dug in further, we find its just not trouble free (i.e. experimental). The default is to use 1/4th of RAM which is entirely inefficient [2]. The "MaxRAMFraction" parameter allows to specify 1/n fraction and not possible to efficiently use 65% or 75% of memory. The only place to start is to set MaxRAMFraction=2 and that already means only 50% of memory is used for heap. That produces a lot of wastage. A lot of resource efficiency is gained by starting with 65% or 80%. OpenJDK 10 is introducing a new option "MaxRAMPercentage" [3] and that goes closer to making a script unnecessary. TL;DR - The default flags are still experimental in JDK 8/9, and deemed to be better on Java 10. A script is just better for consistency. 1: https://github.com/docker-library/docs/pull/900 https://github.com/docker-library/docs/pull/900 2: https://news.ycombinator.com/item?id=16636544 https://news.ycombinator.com/item?id=16636544 3: https://bugs.openjdk.java.net/browse/JDK-8186248 https://bugs.openjdk.java.net/browse/JDK-8186248
stuff4ben 9y ago
Have to say I really liked this article, but most of it was spent on how containers actually work on the backend. Which I think is fantastic as it's one of the most concise and easy to understand articles I've seen on the subject. I forwarded it to my team because a few of them still think Docker is some black-magic voodoo.
jorgec 9y ago
We don't need it. Sheesh with the containers. I can install many versions of JRE/JDK on my machine at the same time and I can decide (with a single environment variable) which version of JRE run. So, why I need a container?. In the same way, I can have installed different Application Server on a single machine, however, I can't run many servers at the same time without changing ports but it's not the point of Docker.
fokinsean 9y ago
What a great read! I'm mostly in application land all day, and even though I have daily interaction with docker containers I don't have a good understanding what is going on under the hood. Now I partially understand why my coworker who manages our K8s cluster added some of the VM arguments to our java application images.
joerg84 9y ago
One of the authors here: First of all thanks for the great disucssion here, it really motivated to write another follow-up article. There were some comments asking for more practical advice which is totally fair as this was supposed to be more informational and creating problem awareness. What else would you like to see covered in a follow-up?
- spotlmnop 9y ago
  A base Java image and a set of recommendations. Most of us here (I assume) are running containers in K8s or Swarm. Our issue is - there is no base docker image which the java community can embrace. I have pods which restarted 35 times in the last one month.
adrianmonk 9y ago
I wonder if it might be a good general rule to never give just 1 CPU to any multi-threaded application (JVM or otherwise). Often, there's a mix of threads, some of which are doing CPU-intensive stuff and some which just need to do some quick thing to unblock something else, like start some new IO when some IO completes. With 1 CPU, any time any CPU-intensive stuff is happening, these quick things have to wait their turn until the next time slice. With 2 CPUs, you have twice the work and twice the likelihood of having to deal with CPU-intensive stuff, but CPU-intensive moments don't necessarily happen at the same time, so you have better odds of having one of those CPUs immediately available to do the small, quick stuff.

212 points by edw519 9y ago

Wasn't the original promise of the JVM to provide container like services to applications? This field is way outside where I normally work, I would love to hear some expert commentary about this.

sk5t 9y ago
It is perhaps doable by getting very intimate with Java's classloaders and security policies, but most people don't want to deal with that stuff--it's complex, not fun, not portable. That said, there are practical examples to be found in common servlet containers, with some shared libraries, but individual servlets mostly firewalled from each other.
- cuspycode 9y ago
  Servlet containers have two levels of isolation: individual apps are mostly firewalled from each other as you say, but when running multiple servlet containers they are as isolated from each other as multiple docker processes are when it comes to shared libraries, security policies etc. JVM processes are easy to reason about at the OS level, while servlet applications are more complex. But we are comparing Docker with JVM, not Docker with e.g. Tomcat webapps.
techman9 9y ago
Sorta. The JVM provides portability but has less fine controls for things like disk, network and CPU isolation and limiting.
kev009 9y ago
You are effectively correct, and to take it a bit further the "VM" in JVM does mean the same thing as the VM in VMWare even if it feels a lot different in practice. If all your dependencies are JVM based and bundled into a JAR/WAR, OS containers aren't doing much for you.
- weego 9y ago
  They're doing a ton for your CV though.
szemet 9y ago
Yes, but some JVM implemetations have(had?) more support for this use case than others: e.g. https://www.ibm.com/developerworks/library/j-multitenant-java/index.html https://www.ibm.com/developerworks/library/j-multitenant-jav...
hibikir 9y ago
It kind of does: Jave EE was all about having single Java Application Servers running on each host, and deploying applications/services on top. In practice every service was built by the same company though, so focused moved toward shared libraries in containers and things like that: Search for OSGI. The issue is both that the security concerns were way softer than they are today, and that all your dependencies better be deplorable in a JVM too. Modern ideas of having databases deployed along with the services that need them don't work quite as well as just using OS level virtualization. That said many a crufry old company still deploys hundreds of services to production by loading .war files into a cluster of servers running JBoss or WebLogic.
- gunnihinn 9y ago
  > and that all your dependencies better be deplorable in a JVM too. Remember: Never use the standard library when a forgotten, half-assed CPAN module you can run via Perlito will kind of do.
- timv 9y ago
  > your dependencies better be deplorable in a JVM too Sadly, many Java dependencies are quite deplorable.
tytytytytytytyt 9y ago
No more so than any other language with a VM, which typically isn't much at all.
fulafel 9y ago
This got me thinking about how it is on Clojure, and turns out there's a library for doing that: https://github.com/RutledgePaulV/clj-embed https://github.com/RutledgePaulV/clj-embed
hinkley 9y ago
The PR for memory protected preemotive multitasking operating systems still sounds remarkably similar to the container sale pitch today (at least the first half of it). I wonder what we’ll be championing in another 10 years.
cuspycode 9y ago
The JVM is a container by itself, and it has been able to isolate dependencies and memory requirements for many years before docker was conceived. The JVM doesn't do everything that docker does, for example it doesn't provide overlay file systems, but this is trivial to add via standard tools (e.g. aufs), and it also doesn't limit CPU usage per container but this is also trivial to add without having to deal with the extra complications of using docker. I think docker is great for non-JVM applications that run into conflicts with other applications, dependency-wise or resource-wise, but I don't see the point of using docker for pure Java applications. It just adds an unnecessary layer that provides no benefit. Edit: If the point is to encapsulate and isolate a Java application as much as possible, I would consider using a unikernel like OSv before considering Docker. This would be more efficient and also more secure.

Edmond 9y ago

The real killer app would be the ability to fully containerize all jvm instances running on any given box. It will be a setup where one jvm instance on the host basically serves the role of "master" in terms of class data and shared object loading while each container instance uses its memory allotment only for running computations specific to the application in that container while sharing memory objects with other containers as much as possible. It is possible to do something similar at the moment but it requires going through a hodge-podge of painful hacks. A seamless solution to this would basically make the jvm an out of the box poor man's polyglot PaaS platform.

jjtheblunt 9y ago
Aren't you describing every OS since Solaris 2.x that uses shared pages across processes as well as Solaris 2.x did?
- Edmond 9y ago
  My OS internals knowledge is rusty so I am not sure they are quite the same but likely similar. If you think of a typical jvm application (true for non-jvm apps as well), a significant chunk of class data will be shared since apps are typically using the same libraries (with deltas in versions), allowing easy reuse of class data across all container instances on a host would be a major scalability advance.
  - karambahh 9y ago
    There is a java feature called class data sharing that does that since Java 5 but I've never seen it in the wild.[0] Java 10 will bring AppCDS via JEP 310[1] [0]https://fosdem.org/2018/schedule/event/class_data_sharing https://fosdem.org/2018/schedule/event/class_data_sharing [1]http://openjdk.java.net/jeps/310 http://openjdk.java.net/jeps/310
bokchoi 9y ago
There was a JSR-121 Application Isolation proposal, that sounds similar to what you're talking about. I'm not sure why it didn't gain traction: http://www.oracle.com/technetwork/articles/javase/mvm-141094.html http://www.oracle.com/technetwork/articles/javase/mvm-141094... https://www.jcp.org/en/jsr/detail?id=121 https://www.jcp.org/en/jsr/detail?id=121

kev009 9y ago

It would be nice to have some kind of cooperative API for runtimes like Java, Go etc to where the OS or container manager can provide watermark hints for the heap size and when to run collections.

techman9 9y ago
How does the JVM determine the number of cores on a Linux system by default?
- kev009 9y ago
  I don't know the precise answer off hand, but probably in an OS dependent code path in OpenJDK. I know for instance OpenJDK uses the global sysctl API to get total RAM on a FreeBSD machine. To use jails as multi-tenancy system you therefore want to fake that API out to the resource controled max for the jail.
- joerg84 9y ago
  One of the authors here: This is the OpenJDK 8 implementation: http://hg.openjdk.java.net/jdk8/jdk8/hotspot/file/tip/src/os/linux/vm/os_linux.cpp#l4967 http://hg.openjdk.java.net/jdk8/jdk8/hotspot/file/tip/src/os...
sitkack 9y ago
Based on request traffic or maximum memory? For the JVM there is -Xmx512m to set the max heap size (512MB in this case). Or are you thinking of something more organic so individual containers could overcommit on memory, and the OOM killer would first send a "collect" signal to all the containers that could GC (and also give the memory back).
- kev009 9y ago
  Yes when I say cooperative I mean nice processes are opting in to being told what and when to do something (change heap size, run a light or heavy collection, etc) apart from their own internal view of affairs.

xcq1 9y ago

Anyone here actually use Java containers in production? Sadly the article mentions very little in terms of practical advice. We've tried running some small Java 8 Spring Boot containers in Kubernetes which are configured to use max ~50M heap and ~150M total off-heap yet decide to use in excess of double that memory so we end up with either a lot of OOMKills or overly large memory limits.

flipp3r 9y ago
Where I work most of our dockerized services are Spring Boot in Kubernetes, they do need more memory than what you've posted and generally run with about 300M ~ 600M usage depending on what they need to do. You can also use smaller frameworks (Vertx? Javalin? possibly Spring Boot 2). I hope that with Java 9 we won't see this amount of memory usage anymore, however our organisation isn't there yet though.
yazr 9y ago
What are the other common JVM Memory offenders ?! (Electron i guess being the obvious JS-to-native example) (And isn't Sprint Boot the light-weight solution to the Sprint memory problem? /s)
maaaats 9y ago
If you ditch Spring Boot and wire Spring yourself the startup-time and footprint should be smaller. But 250M may be a bit optimistic.
jacques_chester 9y ago
Spring Boot is, by design, eager to load everything it sense you might need. Most of the memory usage is not Spring itself, it's the libraries it's pulling in on your behalf. In Spring Boot 2 I'm told you can can use functional beans to cut down on RAM usage. Not sure how it works. But really, it comes down to deciding if you need a dependency or not. As is normal for questions about Spring Boot performance, Dave Syer has done a lot of investigating: https://github.com/dsyer/spring-boot-memory-blog https://github.com/dsyer/spring-boot-memory-blog
oftenwrong 9y ago

ognyankulev 9y ago

The just-released Java 10 has improvements in the container case: https://www.opsian.com/blog/java-on-docker/ https://www.opsian.com/blog/java-on-docker/ edit: at the end of the article, it is acknowledged that there are improvements in Java 10 and the following link is provided: https://bugs.openjdk.java.net/browse/JDK-8146115 https://bugs.openjdk.java.net/browse/JDK-8146115

ChristianGeek 9y ago
There was a discussion about this yesterday on Reddit: https://reddit.com/r/java/comments/85t7dt/java_on_docker_will_no_longer_suck_improvements/ https://reddit.com/r/java/comments/85t7dt/java_on_docker_wil...
joerg84 9y ago
IMO these are a great improvement but one still needs to be aware of the underlying container limits. E.g., understanding why setting MaxRAMFraction=1 might not be a good idea as the container might get killed...

mcgin 9y ago

These options were also backported to Java 8 in update 131 - https://blogs.oracle.com/java-platform-group/java-se-support-for-docker-cpu-and-memory-limits https://blogs.oracle.com/java-platform-group/java-se-support...

r4um 9y ago

Not just the JVM, lot of libraries build thread pools based on number of processors and some of them even hard code the multipliers (X the number of processors etc). Setting up each one becomes a lot of work fast, so we wrote a LD_PRELOAD[1] hook that overrides sysconf[1] _NC_PROCESSORS* call to get number of processors availabale/online to a certain specified value and is baked in the docker image by default during builds. [1] http://man7.org/linux/man-pages/man3/sysconf.3.html http://man7.org/linux/man-pages/man3/sysconf.3.html

_pmf_ 9y ago
That's horrible and irresponsible; I love it.
MaxBarraclough 9y ago
Can we expect that kind of horror to fade away as Java evolves? In the .Net world there's the ThreadPool class to manage the single process-wide thread-pool, and the Task class to enqueue and orchestrate concurrent jobs (it uses ThreadPool and almost completely abstracts away the thread management). (You could write your own thread-pool, of course, but for most code that wouldn't make sense.) As I understand it, the JVM is rather behind in this department. (Not to mention async/await.)
- jacques_chester 9y ago
  Java has these concepts too (ExecutorService since at least Java 5, circa 2004). The problem is not with the JDK libraries, it's been in the JVM's assumption that it's running on bare metal or a VM. Linux containers leak information about the "true" environment in a way that upset JVM assumptions before 9 and 10.
- badpun 9y ago
  > In the .Net world there's the ThreadPool class to manage the single process-wide thread-pool, and the Task class to enqueue and orchestrate concurrent jobs (it uses ThreadPool and almost completely abstracts away the thread management). > (You could write your own thread-pool, of course, but for most code that wouldn't make sense.) Java had thread pools and tasks since 2004 (https://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/ThreadPoolExecutor.html https://docs.oracle.com/javase/1.5.0/docs/api/java/util/conc...). For typical Java programs, no one is writing their own thread pool functionality as far as I know.

jrudolph 9y ago

This. I've seen container builds for Java applications broken in the described ways fail in production over and over again. Just because Docker makes building an image very easy doesn't mean that you will end up with a production ready image using three lines of code in a Dockerfile. Most of the time people don't even bother to use a non-root user to execute the jvm in their container... That's why I feel platforms like Cloud Foundry are a much better fit for teams that don't have tons of container experience but want to get the benefits of a containerized runtime. The CF java buildpack[1] for example automatically handles OOM and heap settings calculation while building your application container. disclaimer: co-founder at meshcloud, we offer a public cloud service for Cloud Foundry and K8s hosted in german datacenters. [1] https://github.com/cloudfoundry/java-buildpack

jacques_chester 9y ago
In particular, the Java buildpack uses a component developed entirely to calculate JVM settings. I am not sure what the plans are for Java 9 and 10 yet. Ben Hale works on JBP pretty much fulltime and the Spring team tend to experiment pretty early on JDKs. So I can't see it falling far behind. [0] https://github.com/cloudfoundry/java-buildpack-memory-calculator https://github.com/cloudfoundry/java-buildpack-memory-calcul...
nerdwaller 9y ago
I wrote up my experience[0] on containerizing JVM based applications a bit ago using the cloud foundry java buildpack’s memory calculator. Fortunately the JVM now has a way to respect cgroup memory[1] making it a bit simpler. [0]: https://medium.com/@matt_rasband/dockerizing-a-spring-boot-application-6ec9b9b41faf https://medium.com/@matt_rasband/dockerizing-a-spring-boot-a... [1]: https://twitter.com/codepitbull/status/934384652806221825?s=20 https://twitter.com/codepitbull/status/934384652806221825?s=...

mboehm 9y ago

>> But the in case of containers with a hard memory limit, the entire container will simply be killed without warning. At least in my experience that is not true. I quite often run into this issue and the OOM-killer will only kill one of the processes inside the container, not the entire container. >> The same is true for default memory limits. The JVM looks at the host overall memory and uses that to set its defaults. Well, I guess if you launch a JVM anywhere without setting appropriate memory settings, you are doing something fundamentally wrong.

rad_gruchalski 9y ago
>> At least in my experience that is not true. I quite often run into this issue and the OOM-killer will only kill one of the processes inside the container, not the entire container. Hence one does not run multiple processes in the container or / and handle crashes of potential child processes correctly.
- mboehm 9y ago
  Well, I guess, sometimes you need to break the rules. But you are right.

matttproud 9y ago

We created a model and applied unsupervised ML to tuning the JVM for execution in containers and eventually open sourced the project: https://github.com/sladeware/groningen https://github.com/sladeware/groningen Unfortunately the experiment state persistence management capability is broken.

allsunny 9y ago

We use shaded/fat jars where I work because it's easier to ensure we never run into issues with missing dependencies. It does however come at the cost of longer build times. I've been hoping that Docker might be able to help with this by allowing us to keep all of the dependencies in the container base image and then just add the new class files in the build process. Is that a reasonable assumption?

rdsubhas 9y ago

Fabric8 has really good base java images[1] with a script that simply sets environment variables with right GC and CPU parameters before launching Java, with nice sane defaults. Heavily encourage anyone running Java in containers to use their base image, or for larger organizations to create standard base image dockerfiles that set these JVM envvar parameters. A simple contract is: ENTRYPOINT belongs to the base image, CMD belongs to downstream application images (unless something else essential). Just don't use vanilla "FROM openjdk:8-jre" and expect it to work. That's the worst way to kill application performance and reliability in a container. 1: https://github.com/fabric8io-images/java/blob/master/images/centos/openjdk8/jre/run-java.sh#L3 https://github.com/fabric8io-images/java/blob/master/images/...

organsnyder 9y ago
+1 for Fabric8. It makes running Java in containers much more pleasant, and provides a complete opinionated ecosystem, if that's desired. I work at an enterprise shop that is a Red Hat customer (so we get commercial support on this stuff), and it has made our lives much easier in many respects.
deleted 9y ago
[deleted]
deepakhj 9y ago
Jdk8u131+ and 9 support detecting cpu and memory limits to set heap and core usage. https://blogs.oracle.com/java-platform-group/java-se-support-for-docker-cpu-and-memory-limits https://blogs.oracle.com/java-platform-group/java-se-support...
- rdsubhas 9y ago
  We tried those flags in the beginning since it was introduced in docker openjdk image[1]. When we dug in further, we find its just not trouble free (i.e. experimental). The default is to use 1/4th of RAM which is entirely inefficient [2]. The "MaxRAMFraction" parameter allows to specify 1/n fraction and not possible to efficiently use 65% or 75% of memory. The only place to start is to set MaxRAMFraction=2 and that already means only 50% of memory is used for heap. That produces a lot of wastage. A lot of resource efficiency is gained by starting with 65% or 80%. OpenJDK 10 is introducing a new option "MaxRAMPercentage" [3] and that goes closer to making a script unnecessary. TL;DR - The default flags are still experimental in JDK 8/9, and deemed to be better on Java 10. A script is just better for consistency. 1: https://github.com/docker-library/docs/pull/900 https://github.com/docker-library/docs/pull/900 2: https://news.ycombinator.com/item?id=16636544 https://news.ycombinator.com/item?id=16636544 3: https://bugs.openjdk.java.net/browse/JDK-8186248 https://bugs.openjdk.java.net/browse/JDK-8186248

stuff4ben 9y ago

Have to say I really liked this article, but most of it was spent on how containers actually work on the backend. Which I think is fantastic as it's one of the most concise and easy to understand articles I've seen on the subject. I forwarded it to my team because a few of them still think Docker is some black-magic voodoo.

jorgec 9y ago

We don't need it. Sheesh with the containers. I can install many versions of JRE/JDK on my machine at the same time and I can decide (with a single environment variable) which version of JRE run. So, why I need a container?. In the same way, I can have installed different Application Server on a single machine, however, I can't run many servers at the same time without changing ports but it's not the point of Docker.

fokinsean 9y ago

What a great read! I'm mostly in application land all day, and even though I have daily interaction with docker containers I don't have a good understanding what is going on under the hood. Now I partially understand why my coworker who manages our K8s cluster added some of the VM arguments to our java application images.

joerg84 9y ago

One of the authors here: First of all thanks for the great disucssion here, it really motivated to write another follow-up article. There were some comments asking for more practical advice which is totally fair as this was supposed to be more informational and creating problem awareness. What else would you like to see covered in a follow-up?

spotlmnop 9y ago
A base Java image and a set of recommendations. Most of us here (I assume) are running containers in K8s or Swarm. Our issue is - there is no base docker image which the java community can embrace. I have pods which restarted 35 times in the last one month.

adrianmonk 9y ago

I wonder if it might be a good general rule to never give just 1 CPU to any multi-threaded application (JVM or otherwise). Often, there's a mix of threads, some of which are doing CPU-intensive stuff and some which just need to do some quick thing to unblock something else, like start some new IO when some IO completes. With 1 CPU, any time any CPU-intensive stuff is happening, these quick things have to wait their turn until the next time slice. With 2 CPUs, you have twice the work and twice the likelihood of having to deal with CPU-intensive stuff, but CPU-intensive moments don't necessarily happen at the same time, so you have better odds of having one of those CPUs immediately available to do the small, quick stuff.