This is really fascinating, and I love PID Controllers. That said, I really don't recommend doing this in production. It is very rare that you can successfully

This is really fascinating, and I love PID Controllers. That said, I really don't recommend doing this in production. It is very rare that you can successfully use this in practice. You either a) Have just one machine, and that machine has a fixed number of cores (or cannot readily swap number of cores on the fly). If your machine is dedicated to this task, it doesn't make much sense to resize your thread pool - Leave it at max. The only way a smaller threadpool would help is if you have noisy neighbors sharing a machine and cores. However, if you have noisy neighbors, your simple queue-length PID controller will start using more cores not just when you have more than one core worth of traffic, but also when you have a lot of noise and are at max threads. It's fine and dandy in the happy case, but it fails pretty spectacularly in the worst case - Each additional thread creates more exposure to your noisy neighbors and possibly even worse global throughput. If you b) Have multiple machines, you don't want to do this at all. The number of replicas (processes, or "pods" as I will call them because anchoring this in real-world K8s experience is helpful) goes up and down, but you need feedback on the number of threads or cores involved at the top level. If the number of threads goes up per process but the number of processes goes down, or the number of threads goes down but the number of processes goes up, you can get really wonky feedback loops.