3 ms·
Some of our tools do so much allocation that capturing all the information (using MTuner) to disk and later loading require by itself hundreths of GB. Instead
by malkia 3y ago
Some of our tools do so much allocation that capturing all the information (using MTuner) to disk and later loading require by itself hundreths of GB.
Instead I've added random sampling - e.g. if ptr % modulo > level - output it or not.
Another factor that slows down is doing a callstack capture. It's not for free at all, like on Windows it has to go through the exception handlers, etc. I think perfetto simply captures the whole stack (need to check again), and then offline decodes it - or something like this.
Windows ETW Tracing can also capture the stack, but I guess it'll incur also some penalty - it can't come for free.
I also wish there was some kind of standard binary format for emitting alloc/free sequences with callstack/etc.
- kouteiheika 3y ago> Instead I've added random sampling - e.g. if ptr % modulo > level - output it or not. A good alternative that also cuts down on the amount of data significantly is to strip away temporary allocations, say, only emit those allocations which were alive for at least X seconds.
- malkia 3y agoAh good idea! It kind of calls for out-of-process to filter this out, or post-process (maybe there is common tooling around).
- milianw 3y agoHeaptrack can handle millions of allocations per second, and thanks to the deduplication built into the trace file format combined with zstd compression, the overhead is pretty manageable. It's pretty hard to get "hundreds of GB" of data recorded. Another thing: you can always just runtime attach and profile a partial time to reduce the amount of data recorded. Finally: if your tools do so many allocations, maybe you should consider optimizing them...
- malkia 3y agoThanks! Should look into heaptrack - reading that it's linux only, but probably can be adopted for Windows?
- milianw 3y agoMTuner exists for Windows which is similar but different. Porting heaptrack is not straight forward, as it heavily relies on libraries that do not work on Windows or do not support Windows executables and debugging information, such as libunwind and elfutils.