3 ms·

While neat experiment, I don't know if thread pools are a thing that makes sense to scale up and down? Maybe these days VMs or containers would make more sense

• stevan 4y ago
  I got the idea of scaling thread pools from a paper[0] coauthored by Eric Brewer (of CAP theorem fame and also vice-president of infrastructure at Google according to Wikipedia). The paper was written in 2001, so things might have changed a bit, but I doubt the example is completely irrelevant today. [0]: https://people.eecs.berkeley.edu/~brewer/papers/SEDA-sosp.pdf https://people.eecs.berkeley.edu/~brewer/papers/SEDA-sosp.pd...
• remram 4y ago
  Intuitively, there is a threshold above which increasing the number of threads decreases throughput (for example reaching your total number of cores if you are CPU-bound, might be a different number for different workloads). It would be nice to have the controller try to find that number, this PID approach just tries to scale more if there's more input. It might be more suited to infinitely-scalable situations like cloud VMs, where you are trying to optimize money spent.
  • naasking 4y ago
    > It would be nice to have the controller try to find that number, this PID approach just tries to scale more if there's more input. Right, it should base the feedback on throughput measures instead. If throughput starts dropping, then the threadpool is overcommitted, and should scale back. I believe the SEDA papers mentioned in the README discusses this as backpressure.
• toast0 4y ago
  My experience from apache with forking is if your number of workers scales up and down with load, you get two big problems: 1) memory usage grows significantly with load (but our apache modules were bloaty, maybe your thread pool isn't) and 2) you end up paying the cost to initialize the workers at high load. In that situation, it's best to just figure out your maximum worker number and run that all day. Although, if your OS is smart, it allocates new work to the most recently active process, so some workers stay idle most of the time. If your threads don't block and you have one main application running on the node, you usually just want to run one thread per node and you're done. If you are running a very network heavy load where you can eliminate or highly reduce cross thread communication, you may want one core per NIC tx/rx queue and one thread per core to eliminate cross-core communication; any cores above the number of queues will just be idle, because cross-core communication is more expensive than the work they can do (but that's not a super common scenario). A control system to add and remove nodes makes sense if you're cloudy, though, since there's a cost for running nodes.
  • zokier 4y ago
    > In that situation, it's best to just figure out your maximum worker number and run that all day. Although, if your OS is smart, it allocates new work to the most recently active process, so some workers stay idle most of the time. That would be my intuition too, in particular that usually the cost of idle workers is pretty low, so it's better just to preallocate some fixed max number of workers than try to scale them.
    • toast0 4y ago
      > That would be my intuition too, in particular that usually the cost of idle workers is pretty low, so it's better just to preallocate some fixed max number of workers than try to scale them. I wouldn't necessarily intuit the cost of idle workers is low, more that the non-cpu cost of workers is roughly fixed, and if it's too expensive to run more than you need at idle, it's still going to be too expensive to run that many at full load. Sometimes it's hard to know what the max load capacity is, but Apache configs where the worker count scales in and out are really easy to get into load is high -> spawn more workers -> use too much memory -> pick your poison: evict too much disk cache / swap to death / oom killer