4 ms·

I recently went through a similar exercise using the same tools on a large open source Python codebase. The solution is the same as the author found: don't ke

I recently went through a similar exercise using the same tools on a large open source Python codebase. The solution is the same as the author found: don't keep Exception objects around past the actual lifetime of the exception. An Exception has the traceback, and the traceback has all the frames of the call stack, and each frame has a reference to the local variables in that frame. Keeping an Exception around past its time can yield a huge tangle of circular references that puts a lot of pressure on the GC. I reduced memory utilization dramatically by deleting 1 line of code.

mikepurvis 6y ago
Does it do the right thing if you unset the traceback portion of the Exception object?
- jnwatson 6y ago
  Sure. However, instead of reaching into objects to prune them, the better approach is to ask why you're keeping the object around at all. Extracting just the parts you want into your own structure is usually the better way to go.
  - mikepurvis 6y ago
    A fair point. The scenario I'm thinking of is with APIs like asyncio.gather, which can be asked to return exception objects rather than re-raise them, for coroutines which fail/raise. This isn't in and of itself a case of long-lived exception objects, but I could see how this approach might lead some users to treat the exception as a long-lived results object, perform type checks against it, etc. If you were in the position of having an app that was set up like this, it would be nice to know you had the option of trimming the Exception's footprint while still allowing it to retain its Exception-ness.
trupper-se 6y ago
> 'don't keep Exception objects around past the actual lifetime of the exception' But, does it generate more read/write-cycles on your local SSD/Flash-Memory ? Maybe my main intention was: "Does it make 'computing' longer lasting -or does it support obsolence", or 'is it a typical language processing idea' ? ˁ ˆ͜ᴥˆ ˀ
- seabrookmx 6y ago
  This impacts memory (RAM) usage and has nothing to do with storage (SSD).

mikepurvis 6y ago

Does it do the right thing if you unset the traceback portion of the Exception object?

jnwatson 6y ago
Sure. However, instead of reaching into objects to prune them, the better approach is to ask why you're keeping the object around at all. Extracting just the parts you want into your own structure is usually the better way to go.
- mikepurvis 6y ago
  A fair point. The scenario I'm thinking of is with APIs like asyncio.gather, which can be asked to return exception objects rather than re-raise them, for coroutines which fail/raise. This isn't in and of itself a case of long-lived exception objects, but I could see how this approach might lead some users to treat the exception as a long-lived results object, perform type checks against it, etc. If you were in the position of having an app that was set up like this, it would be nice to know you had the option of trimming the Exception's footprint while still allowing it to retain its Exception-ness.

trupper-se 6y ago

> 'don't keep Exception objects around past the actual lifetime of the exception' But, does it generate more read/write-cycles on your local SSD/Flash-Memory ? Maybe my main intention was: "Does it make 'computing' longer lasting -or does it support obsolence", or 'is it a typical language processing idea' ? ˁ ˆ͜ᴥˆ ˀ

seabrookmx 6y ago
This impacts memory (RAM) usage and has nothing to do with storage (SSD).