3 ms·

It's nice to see how many different approaches to profiling there are these days in Python. I work on another (commercial but with free plan) Python profiler, S

It's nice to see how many different approaches to profiling there are these days in Python. I work on another (commercial but with free plan) Python profiler, Sciagraph: https://sciagraph.com https://sciagraph.com. The main use case is data science and other long-running batch jobs. Some differences: 1. It does memory profiling at basically no performance overhead; sounds like for FunctionTrace it's high overhead so off by default. And it catches _all_ memory allocations, not just Python API ones. This is based on using sampling, so it's not useful for profiling tiny functions (but for data science/scientific computing it'll work just fine). 2. Uses sampling for performance profiling, unlike FunctionTrace. Again, perfectly fine for any non-micro-benchmark data science program. 3. Also has a timeline view, without having to upload your data anywhere. 4. No native stacks yet. 5. Shows you if you're using CPU or I/O for every particular sample.

Veserv 3y ago
You should really not randomly speculate when doing adversarial comparisons. Note: I am not a author of FunctionTrace. 1. You claim ~5% overhead [1] and it is not useful for profiling tiny functions. They claim ~10% overhead when handling tiny functions and, who I believe to be the author of the package below, said it averages low single digit percents (i.e <5%) on normal non-worst cases. This is comparable or less overhead than you claim. 2. Sampling is so much worse than a full execution trace from a performance optimization and observability perspective it is ridiculous. Yes, a magnifying glass and a microscope are both okay for looking at ants, but only one is good for looking at cells. People prefer sampling because it was traditionally lower overhead, but for the same cost full tracing is so much better it is not even worth comparing, the result is patently obvious. 3. No “upload” is done. You are not sending data to them, everything is local. They are just opening the profiling data using Firefox to use it as a viewer like how you might open a PDF in Firefox. [1] https://www.sciagraph.com/docs/understanding/fil/ https://www.sciagraph.com/docs/understanding/fil/

Veserv 3y ago

You should really not randomly speculate when doing adversarial comparisons. Note: I am not a author of FunctionTrace. 1. You claim ~5% overhead [1] and it is not useful for profiling tiny functions. They claim ~10% overhead when handling tiny functions and, who I believe to be the author of the package below, said it averages low single digit percents (i.e <5%) on normal non-worst cases. This is comparable or less overhead than you claim. 2. Sampling is so much worse than a full execution trace from a performance optimization and observability perspective it is ridiculous. Yes, a magnifying glass and a microscope are both okay for looking at ants, but only one is good for looking at cells. People prefer sampling because it was traditionally lower overhead, but for the same cost full tracing is so much better it is not even worth comparing, the result is patently obvious. 3. No “upload” is done. You are not sending data to them, everything is local. They are just opening the profiling data using Firefox to use it as a viewer like how you might open a PDF in Firefox. [1] https://www.sciagraph.com/docs/understanding/fil/ https://www.sciagraph.com/docs/understanding/fil/