5 ms·

As someone not that familiar with Docker or Go, is this behavior intentional? Could the Go team make it aware of the CGroups limit? Do other runtimes behave sim

by bruh2 3y ago

As someone not that familiar with Docker or Go, is this behavior intentional? Could the Go team make it aware of the CGroups limit? Do other runtimes behave similarly?

eloisant 3y ago
Yes, I've experienced the same problem with the JVM (in Scala).
yjftsjthsd-h 3y ago
I'm fairly certain that that .net had to deal with it and Java had or still has a problem, I forget which. (Or did you mean runtimes like containerd?)
- richdougherty 3y ago
  Supported in Java 10 (and backported to Java 8) since 2018. Not sure about .NET. - "The JVM has been modified to be aware that it is running in a Docker container and will extract container specific configuration information instead of querying the operating system. The information being extracted is the number of CPUs and total memory that have been allocated to the container." https://www.oracle.com/java/technologies/javase/8u191-relnotes.html https://www.oracle.com/java/technologies/javase/8u191-relnot... - Here's a more detailed explanation and even a shared library that can be used to patch container unaware versions of Java. I wonder if the same could be done for Go? "LD_PRELOAD=/path/to/libproccount.so java <args>" https://stackoverflow.com/a/64271429 https://stackoverflow.com/a/64271429 https://gist.github.com/apangin/78d7e6f7402b1a5da0fa3abd93811f36 https://gist.github.com/apangin/78d7e6f7402b1a5da0fa3abd9381... - There are more recent changes to Java container awareness as well: https://developers.redhat.com/articles/2022/04/19/java-17-whats-new-openjdks-container-awareness# https://developers.redhat.com/articles/2022/04/19/java-17-wh...
  - jpolidor 3y ago
    Then in Java, if you don't set the limits, it gets the CPU from the VM via Runtime.getRuntime().availableProcessors()... this method returns the number of CPUs of the VM or the value set as CPU Quota. Starting from Java 11 the -XX:+PreferContainerQuotaForCPUCount is by default true. For Java <= 10 the CPU count is equal to the CPU shares. That method then is used to calculate the GC threads, fork join pool size, compiler threads etc. The solution would be to set -XX:ActiveProcessorCount=X where X is ideally the CPU shares value but as we know shares can change over time, so you would change this value over time... Edit: or set -XX:-PreferContainerQuotaForCPUCount

by bruh2 3y ago

As someone not that familiar with Docker or Go, is this behavior intentional? Could the Go team make it aware of the CGroups limit? Do other runtimes behave similarly?

eloisant 3y ago

Yes, I've experienced the same problem with the JVM (in Scala).

yjftsjthsd-h 3y ago

I'm fairly certain that that .net had to deal with it and Java had or still has a problem, I forget which. (Or did you mean runtimes like containerd?)

richdougherty 3y ago
Supported in Java 10 (and backported to Java 8) since 2018. Not sure about .NET. - "The JVM has been modified to be aware that it is running in a Docker container and will extract container specific configuration information instead of querying the operating system. The information being extracted is the number of CPUs and total memory that have been allocated to the container." https://www.oracle.com/java/technologies/javase/8u191-relnotes.html https://www.oracle.com/java/technologies/javase/8u191-relnot... - Here's a more detailed explanation and even a shared library that can be used to patch container unaware versions of Java. I wonder if the same could be done for Go? "LD_PRELOAD=/path/to/libproccount.so java <args>" https://stackoverflow.com/a/64271429 https://stackoverflow.com/a/64271429 https://gist.github.com/apangin/78d7e6f7402b1a5da0fa3abd93811f36 https://gist.github.com/apangin/78d7e6f7402b1a5da0fa3abd9381... - There are more recent changes to Java container awareness as well: https://developers.redhat.com/articles/2022/04/19/java-17-whats-new-openjdks-container-awareness# https://developers.redhat.com/articles/2022/04/19/java-17-wh...
- jpolidor 3y ago
  Then in Java, if you don't set the limits, it gets the CPU from the VM via Runtime.getRuntime().availableProcessors()... this method returns the number of CPUs of the VM or the value set as CPU Quota. Starting from Java 11 the -XX:+PreferContainerQuotaForCPUCount is by default true. For Java <= 10 the CPU count is equal to the CPU shares. That method then is used to calculate the GC threads, fork join pool size, compiler threads etc. The solution would be to set -XX:ActiveProcessorCount=X where X is ideally the CPU shares value but as we know shares can change over time, so you would change this value over time... Edit: or set -XX:-PreferContainerQuotaForCPUCount