5 ms·

Counterintuitively: program in python only if you can get away without knowing these numbers. When this starts to matter, python stops being the right tool for

• MontyCarloHall 10mo ago
  Exactly. If you're working on an application where these numbers matter, Python is far too high-level a language to actually be able to optimize them.
• libraryofbabel 10mo ago
  Or keep your Python scaffolding, but push the performance-critical bits down into a C or Rust extension, like numpy, pandas, PyTorch and the rest all do. But I agree with the spirit of what you wrote - these numbers are interesting but aren’t worth memorizing. Instead, instrument your code in production to see where it’s slow in the real world with real user data (premature optimization is the root of all evil etc), profile your code (with pyspy, it’s the best tool for this if you’re looking for cpu-hogging code), and if you find yourself worrying about how long it takes to add something to a list in Python you really shouldn’t be doing that operation in Python at all.
  • eichin 10mo ago
    "if you're not measuring, you're not optimizing"
• Demiurge 10mo ago
  I agree. I've been living off Python for 20 years and have never needed to know any of these numbers, nor do I need them now, for my work, contrary to the title. I also regularly use profiling for performance optimization and opt for Cython, SWIG, JIT libraries, or other tools as needed. None of these numbers would ever factor into my decision-making.
  • AtlasBarfed 10mo ago
    ..... You don't see any value in knowing that numbers?
    • TuringTest 10mo ago
      As others have pointed out, Python is better used in places where those numbers aren't relevant. If they start becoming relevant, it's usually a sign that you're using the language in a domain where a duck-typed bytecode scripting-glue language is not well-suited.
    • Demiurge 10mo ago
      That's what I just said. There is zero value to me knowing these numbers. I assume that all python built in methods are pretty much the same speed. I concentrate on IO being slow, minimizing these operations. I think about CPU intensive loops that process large data, and I try to use libraries like numpy, DuckDB, or other tools to do the processing. If I have a more complicated system, I profile its methods, and optimize tight loops based on PROFILING. I don't care what the numbers in the article are, because I PROFILE, and I optimize the procedures that are the slowest, for example, using cython. Which part of what I am saying does not make sense?
      • KeplerBoy 10mo ago
        That makes perfect sense. Especially since those numbers can change with new python versions.
• Quothling 10mo ago
  Why? I've build some massive analytic data flows in Python with turbodbc + pandas which are basically C++ fast. It uses more memory which supports your point, but on the flip-side we're talking $5-10 extra cost a year. It could frankly be $20k a year and still be cheaper than staffing more people like me to maintain these things, rather than having a couple of us and then letting the BI people use the tools we provide for them. Similarily when we do embeded work, micro-python is just so much easier to deal with for our engineering staff. The interoperability between C and Python makes it great, and you need to know these numbers on Python to know when to actually build something in C. With Zig getting really great interoperability, things are looking better than ever. Not that you're wrong as such. I wouldn't use Python to run an airplane, but I really don't see why you wouldn't care about the resources just because you're working with an interpreted or GC language.
  • fooker 10mo ago
    > you need to know these numbers on Python to know when to actually build something in C People usually approach this the other way, use something like pandas or numpy from the beginning if it solves your problem. Do not write matrix multiplications or joins in python at all. If there is no library that solves your problem, it's a great indication that you should avoid python. Unless you are willing to spend 5 man-years writing a C or C++ library with good python interop.
    • oivey 10mo ago
      People generally aren’t rolling their own matmuls or joins or whatever in production code. There are tons of tools like Numba, Jax, Triton, etc that you can use to write very fast code for new, novel, and unsolved problems. The idea that “if you need fast code, don’t write Python” has been totally obsolete for over a decade.
      • fooker 10mo ago
        Yes, that's what I said. If you are writing performance sensitive code that is not covered by a popular Python library, don't do it unless you are a megacorp that can put a team to write and maintain a library.
• bathtub365 10mo ago
  These basically seem like numbers of last resort. After you’ve profiled and ruled out all of the usual culprits (big disk reads, network latency, polynomial or exponential time algorithms, wasteful overbuilt data structures, etc) and need to optimize at the level of individual operations.
• BiteCode_dev 10mo ago
  Not at all. Some of those number are very important: - Set membership check is 19.0 ns, list is 3.85 μs. Knowing what data structure to use for the job is paramount. - Write 1KB file is 35.1 μs but 1MB file is only 207 μs. Knowing the implications of I/O trade off is essential. - sum() 1,000 integers is only 1,900 ns: Knowing to leverage the stdlib makes all the difference compared to manual loop. Etc. A few years ago I did a Python rewrite of a big clients code base. They had a massive calculation process that took 6 servers 2 hours. We got it down to 1 server, 10 minutes, and it was not even the goal of the mission, just the side effect of using Python correctly. In the end, quadratic behavior is quadratic behavior.
  • deleted 10mo ago
    [deleted]
  • fooker 10mo ago
    List membership check being significantly slower than set membership check is freshman computer science 101.