4 ms·

Advanced Python Mastery: https://news.ycombinator.com/item?id=36785005 https://news.ycombinator.com/item?id=36785005 Book: High Performance Python

• IanOzsvald 3y ago
  I'm the co-author of High Performance Python, Micha and I are working on the 3rd ed (for 2025). Lots of bits of the book came from my past conference talks, they're available here (and the public talks will generally be on youtube): https://speakerdeck.com/ianozsvald https://speakerdeck.com/ianozsvald Mostly that content has a scientific focus but the obvious thing that carries over to any part of Python is _profiling_ to figure out what's slow. Top tools I'd recommend are: * https://pypi.org/project/scalene/ https://pypi.org/project/scalene/ combined cpu+memory+gpu profiling * https://github.com/gaogaotiantian/viztracer https://github.com/gaogaotiantian/viztracer get a timeline of execution vs call-stack (great to discover what's happening deep inside pandas) * my https://pypi.org/project/ipython-memory-usage/ https://pypi.org/project/ipython-memory-usage/ if you're in Jupyter Notebooks (built on https://github.com/pythonprofilers/memory_profiler https://github.com/pythonprofilers/memory_profiler which sadly is unmaintained) * https://github.com/pyutils/line_profiler/ https://github.com/pyutils/line_profiler/
  • randlet 3y ago
    Ooh that ipython extension is nice. Thanks!
    • IanOzsvald 3y ago
      Thanks :-) I use it for all my talks and finally decided I'd better start sharing it a bit. It really is useful to understand the memory cost of things like Pandas operations