For memory that are actually two different domains, with different problems, different UX requirements, and different solutions: 1. For server workloads, the m

For memory that are actually two different domains, with different problems, different UX requirements, and different solutions: 1. For server workloads, the main issue is usually leaks. Lacking leaks you can just characterize the workload, give it enough memory (usually it's not very much per request), and call it a day. 2. For batch data processing, the main issue is ... using lots of memory to process the data :) Like, loading 4GB of data and doing stuff to it can easily use 20GB of RAM if you're not careful. For the latter case, and specifically for Python, I've written an open source memory profiler (https://pythonspeed.com/fil https://pythonspeed.com/fil) that helps you spot which code allocated the memory responsible for peak memory. You can also use it for leaks, but that's not the main use case. It has some performance overhead, so it's not usable in production, but I'm also working on version that will be fast enough to run in production, at the cost of slightly less accuracy. For batch processing workloads that loss of accuracy isn't meaningful, mostly, who cares if you're off by 1MB on a 20GB peak memory usage. For server workloads with memory leaks... might take a lot longer to catch the problem as a result of reduced accuracy, and a tool designed specifically for server workloads might work better by taking a different implementation approach.