3 ms·

Memory is a difficult one, especially in garbage-collected languages which have a habit of filling up the heap even when it's not used, so it's not always obvio

Memory is a difficult one, especially in garbage-collected languages which have a habit of filling up the heap even when it's not used, so it's not always obvious how much memory is actually being used without having language/runtime specific signals. The mitigation in Cloud Run is both concurrency, and that you're only billed while a request is active. (Disclosure: Google Cloud PM)

twa927 7y ago
> Memory is a difficult one, especially in garbage-collected languages which have a habit of filling up the heap even when it's not used, so it's not always obvious how much memory is actually being used without having language/runtime specific signals. I'm not sure what do you mean here, Cloud Run uses Docker which runs regular processes in a cgroup, so it's sufficient to check the cgroup memory usage, right? Yes, Java can always use large heaps but we're running Python and C++ where a process' memory usage directly relates to what a program allocates (even PyPy with GC has this property). > The mitigation in Cloud Run is both concurrency, and that you're only billed while a request is active. When there are memory peaks, larger deployments without container-level concurreny look better. For my example 16GB of RAM allows running 8 containers to get a chance for a 2GB task to complete, but on average 90% of the memory will be wasted. On a single 16GB server I can run 48 tasks with 40% wasted and a high chance of the 2GB tasks finishing. Yes, in this scenario I must handle tasks killed due to OOM but the difference in throughput is so large that it's worth it.
- chupasaurus 7y ago
  Cloud Run is built upon Knative so it runs inside k8s (which uses CRI API of a backend, Docker or any other alternative) which handles OOM by inner resource manager, not cgroups.
  - chrisbroadfoot 7y ago
    You can use Cloud Run with GCP-managed infrastructure, and also inside your own GKE cluster. The GCP-managed infrastructure exposes the Knative API, but doesn't actually run in GKE/k8s. (I work on GCP but not much/on Cloud Run. Above is correct to my knowledge, but I'm not an expert.)
    - chupasaurus 7y ago
      Good to know, thank you.
- chrisbroadfoot 7y ago
  > Cloud Run uses Docker which runs regular processes in a cgroup, so it's sufficient to check the cgroup memory usage, right? Cloud Run uses gVisor as its container runtime. Disclaimer: I work on GCP but not much on/with Cloud Run.

twa927 7y ago

> Memory is a difficult one, especially in garbage-collected languages which have a habit of filling up the heap even when it's not used, so it's not always obvious how much memory is actually being used without having language/runtime specific signals. I'm not sure what do you mean here, Cloud Run uses Docker which runs regular processes in a cgroup, so it's sufficient to check the cgroup memory usage, right? Yes, Java can always use large heaps but we're running Python and C++ where a process' memory usage directly relates to what a program allocates (even PyPy with GC has this property). > The mitigation in Cloud Run is both concurrency, and that you're only billed while a request is active. When there are memory peaks, larger deployments without container-level concurreny look better. For my example 16GB of RAM allows running 8 containers to get a chance for a 2GB task to complete, but on average 90% of the memory will be wasted. On a single 16GB server I can run 48 tasks with 40% wasted and a high chance of the 2GB tasks finishing. Yes, in this scenario I must handle tasks killed due to OOM but the difference in throughput is so large that it's worth it.

chupasaurus 7y ago
Cloud Run is built upon Knative so it runs inside k8s (which uses CRI API of a backend, Docker or any other alternative) which handles OOM by inner resource manager, not cgroups.
- chrisbroadfoot 7y ago
  You can use Cloud Run with GCP-managed infrastructure, and also inside your own GKE cluster. The GCP-managed infrastructure exposes the Knative API, but doesn't actually run in GKE/k8s. (I work on GCP but not much/on Cloud Run. Above is correct to my knowledge, but I'm not an expert.)
  - chupasaurus 7y ago
    Good to know, thank you.
chrisbroadfoot 7y ago
> Cloud Run uses Docker which runs regular processes in a cgroup, so it's sufficient to check the cgroup memory usage, right? Cloud Run uses gVisor as its container runtime. Disclaimer: I work on GCP but not much on/with Cloud Run.