We run between 200k and 2m processes per beam VM. I don't know how it'd be possible to get as precise metrics as we need just from relying on linux utilities. A

We run between 200k and 2m processes per beam VM. I don't know how it'd be possible to get as precise metrics as we need just from relying on linux utilities. And in the same cluster, some processes although identical in code have dramatically different workloads.