4 ms·
Every Python programmer should be thinking about far more important things than low level performance minutiae. Great reference but practically irrelevant excep
by willseth 9mo ago
Every Python programmer should be thinking about far more important things than low level performance minutiae. Great reference but practically irrelevant except in rare cases where optimization is warranted. If your workload grows to the point where this stuff actually matters, great! Until then it’s a distraction.
- amelius 9mo agoYeah, if you hit limits just look for a module that implements the thing in C (or write it). This is how it was always done in Python.
- willseth 9mo agoSometimes it’s as simple as finding the hotspot with a profiler and making a simple change to an algorithm or data structure, just like you would do in any language. The amount of handwringing people do about building systems with Python is silly.
- ryandrake 9mo agoI am currently (as we type actually LOL) doing this exact thing in a hobby GIS project: Python got me a prototype and proof of concept, but now that I am scaling the data processing to worldwide, it is obviously too slow so I'm rewriting it (with LLM assistance) in C. The huge benefit of Python is that I have a known working (but slow) "reference implementation" to test against. So I know the C version works when it produces identical output. If I had a known-good Python version of past C, C++, Rust, etc. projects I worked on, it would have been most beneficial when it came time to test and verify.
- pjmlp 9mo agoI rather have a JIT that avoids the "rewrite in C", unless there is no way around it, after heroic optimizations.
- kc0bfv 9mo agoI agree - however, that has mostly been a feeling for me for years. Things feel fast enough and fine. This page is a nice reminder of the fact, with numbers. For a while, at least, I will Know, instead of just feel, like I can ignore the low level performance minutiae.
- HendrikHensen 9mo agoHaving general knowledge about the tools you're working with is not a distraction, it's an intellectual enrichment in any case, and can be a valuable asset in specific cases.
- willseth 9mo agoKnowing that an empty string is 41 bytes or how many ns it takes to do arithmetic operations is not general knowledge.
- oivey 9mo agoHow is it not general knowledge? How do you otherwise gauge if your program is taking a reasonable amount of time, and, if not, how do you figure out how to fix it?
- willseth 9mo agoYou gauge with metrics and profiles, if necessary, and address as needed. You don’t scrutinize every line of code over whether it’s “reasonable” in advance instead of doing things that actually move the needle.
- oivey 9mo agoThese are the metrics underneath it all. Profiles tell you what parts are slow relative to others and time your specific implementation. How long should it take to sum together a million integers?