3 ms·
Lol I interned inside pytorch a few years ago (you and I even met/talked about tangential things :)) and worked on tracking such allocations (although I didn't
by almostgotcaught 2y ago
Lol I interned inside pytorch a few years ago (you and I even met/talked about tangential things :)) and worked on tracking such allocations (although I didn't hook it up to profiler). Spoiler alert: you can't track such provenance because everything gets muddled in the dispatcher.
EDIT: not completely accurate to say you can't do it. I prototyped a little allocator that would stamp every allocation (the pointer itself, in the unused bits, a trick I learned from zach) with the thread id and a timestamp (just an incrementing counter) and then percolate that up to the surface. Obv that didn't land lol.
- brrrrrm 2y agoI wonder if you could track provenance by operating at the highest layer of the dispatcher and capture any calls to GPU operations (ala Cuda Graph)? > in the unused bits I feel like this is a PT rite of passage :P
- chillee 2y agoWe actually do track such provenance now (https://pytorch.org/blog/understanding-gpu-memory-1/ https://pytorch.org/blog/understanding-gpu-memory-1/) - works pretty well I think :)
- almostgotcaught 2y agowell you should tell bram then :p but also while generally "in all things i defer to horace" (ok not really) so maybe i'm not looking closely enough (and missed it) but the bottom of that stack shows (roughly) the autograd dispatch key and not the python call site (or some such). and maybe it's a pedantic difference (depends on what bram wants) but i wanted provenance back to the TS op so that i could then do static memory allocation things with that representation (now i've probably fully de-anonymized myself...) and for that use-case, even what you have now, isn't enough (you can't get a total sum for how much each TS op or whatever allocates and when the corresponding free happens).