Once the problem has been identified, the solution becomes obvious: make sure the work we push to thread pools describes the resources to acquire before running

Once the problem has been identified, the solution becomes obvious: make sure the work we push to thread pools describes the resources to acquire before running the code in a dedicated thread. My favourite approach assigns one global thread pool (queue) to each function or processing step. The arguments to the functions will change, but the code is always the same, so the resource requirements are also well understood. I think I get what he's saying. To avoid one stage in the multi-step computation acquiring too many resources and starving other stages: - Cap the resources used by each stage. - Create one threadpool per stage. - Include sufficient information in pending jobs that the threadpool can calculate the resources needed to process each job. - Only start a job running when it can be run without exceeding resource cap for the stage. Conceptually, you can think of it as pre-allocating a bundle of resources to each processing stage: threads, memory, etc. Each processing stage then runs as many jobs concurrently as it can with the allocated bundle of resources. That's the idea, but in reality each job allocates its own resources from a common pool that is shared among all the processing stages, so the processing stage has to calculate how much memory, etc. each job will allocate before giving it a thread to run on.