3 ms·
Good. Having spent months (a while back) analyzing where memory leaks were occurring in a fairly big code base I can say that it is no fun without good tools. I
by IgorPartola 13y ago
Good. Having spent months (a while back) analyzing where memory leaks were occurring in a fairly big code base I can say that it is no fun without good tools. In that case the issue was that the underlying homegrown library was based heavily on SQLAlchemy. Our SQLA objects referenced each other and when we freed the session, they were not decoupled automatically.
Also beware crappy C extensions. After much debugging I found a quad tree implementation that leaked like a sieve. From what I understand these tools will not help with that, though it is rare.
- gizmo686 13y ago>Also beware crappy C extensions. After much debugging I found a quad tree implementation that leaked like a sieve. From what I understand these tools will not help with that, though it is rare. As someone who has had to find and resolve memory issues with C extensions, I would suggest using Valgrind [0]. It is not a python tool, but (after compiling Python with to play nice with memory debuggers), it works well for debugging memory in C-extension. It basicly runs your program in a VM, where it monitors memory access and reports leaks and other potential errors (such using unitialized, unallocated or deallocated memory) along with a stack trace. [0] http://valgrind.org/ http://valgrind.org/
- IgorPartola 13y agoYup. Valgrind is the bees knees when it comes to debugging any C program. I do not remember if I used it or not for this particular purpose, but it definitely would have helped.
- ogrisel 13y agomemory_profiler [0] mentioned in the post actual can detect memory leaks in compiled C extensions as it uses the OS through the psutil module to measure the aggregate memory usage of the Python process rather than asking the Python runtime itself. It can therefore take into account memory that was not allocated by the Python malloc. [0] https://github.com/fabianp/memory_profiler https://github.com/fabianp/memory_profiler