3 ms·

The thing where people quote optimisers as only eeking out tiny percentages of performance over decades really annoys me. It's true for the numerical kernels wr

The thing where people quote optimisers as only eeking out tiny percentages of performance over decades really annoys me. It's true for the numerical kernels written in C and Fortran where the code is essentially optimal already, and all the compiler has to do is not screw it up. It's grossly false for the vast majority of code, where the sludge written by thousands of engineers gets completely rewritten by the compiler before it executes. Compilers are the reason unskilled devs manage to get tolerable performance out of frameworks. The cpython is slow complaints? That's what life without a good compiler looks like.

klodolph 2y ago
“CPython is slow” is kind of a complaint about the Python object model. I’m not sure that I would pin this one on the compiler, per se. Everything you do in Python involves chasing down pointers and method tables, looking up strings in dictionaries, and that sort of thing. Maybe it’s splitting hairs to talk about whether this is down to the compiler or the language. > It's grossly false for the vast majority of code, where the sludge written by thousands of engineers gets completely rewritten by the compiler before it executes. It used to be true, for sure. What has changed since then is not the sludge or bloat or many cooks. Instead, the major change is that we have written a bunch of code on top of optimizing compilers with the assumption that these optimizations are happening. For example, nowadays, you might write C++ code with a deep call stack and lots of small functions (e.g. through templates), and it’s fast because the compiler inlines and then optimizes the result. Back in the 1990s, you would not have written code that way because the compiler would have choked on it. You see a lot more macro use in code from the 1990s rather than functions, because programmers didn’t trust that the functions will get inlined.
- arka2147483647 2y ago
  > Everything you do in Python involves chasing down pointers and method tables, looking up strings in dictionaries, and that sort of thing. One of the things that a Cpp Compiler can do is De-Virtualize Function Calls. That is; solve those function pointer tables at Compile Time. Why could not a good compiler for Python do the same?
  - nine_k 2y ago
    C++ has an explicit compilation phase after which things like classes remain stable. Python only has a byte-compilation phase that turns the parse tree into executable format. Everything past that, including the creation if classes, imports, etc is runtime. You can pick a class and patch it. You can create classes and functions at runtime; in fact, this happens all the time, and not only for lambdas, but this is how decorators work. A JIT compiler could detect actual usage patterns and replace the code with more efficient versions, until a counter-example is found, and the original "de-optimized" code is run. This is how JavaScript JITs generally work.
  - klodolph 2y ago
    It’s a lot easier in C++ or Java. In C++, you make the virtual calls to some base class, and all you need to demonstrate is that only one concrete class is being used at that place. For example, you might see a List<T> in Java, and then the JIT compiler figures out that the field is always assigned = new ArrayList<T>(), which allows it to devirtualize. In Python, the types are way more general. Basically, every method call is being made to “object”. Every field has a value of type “object”. This makes it much more difficult for the compiler to devirtualize anything. It might be much harder to track which code assigns values to a particular field, because objects can easily escape to obscure parts of your codebase and be modified without knowing their type at all. This happens even if you write very boring, Java-like code in Python.
- nine_k 2y ago
  The point is that Python object model prevents most optimizations, and thus makes the interpreter to executive code exactly as written. All inefficiencies become more obvious this way, while an optimizing compiler could have detected and eliminated many of them in a different language.
  - klodolph 2y ago
    Yes, that’s a good point. But I think you can already see the difference in a lot of C++ codebases when you compile at -O0. Until recently, std::move() was a function call, and at -O0, this meant that calls which should be basically a memcpy() ended up having thousands of little function calls in it to std::move().

klodolph 2y ago

“CPython is slow” is kind of a complaint about the Python object model. I’m not sure that I would pin this one on the compiler, per se. Everything you do in Python involves chasing down pointers and method tables, looking up strings in dictionaries, and that sort of thing. Maybe it’s splitting hairs to talk about whether this is down to the compiler or the language. > It's grossly false for the vast majority of code, where the sludge written by thousands of engineers gets completely rewritten by the compiler before it executes. It used to be true, for sure. What has changed since then is not the sludge or bloat or many cooks. Instead, the major change is that we have written a bunch of code on top of optimizing compilers with the assumption that these optimizations are happening. For example, nowadays, you might write C++ code with a deep call stack and lots of small functions (e.g. through templates), and it’s fast because the compiler inlines and then optimizes the result. Back in the 1990s, you would not have written code that way because the compiler would have choked on it. You see a lot more macro use in code from the 1990s rather than functions, because programmers didn’t trust that the functions will get inlined.

arka2147483647 2y ago
> Everything you do in Python involves chasing down pointers and method tables, looking up strings in dictionaries, and that sort of thing. One of the things that a Cpp Compiler can do is De-Virtualize Function Calls. That is; solve those function pointer tables at Compile Time. Why could not a good compiler for Python do the same?
- nine_k 2y ago
  C++ has an explicit compilation phase after which things like classes remain stable. Python only has a byte-compilation phase that turns the parse tree into executable format. Everything past that, including the creation if classes, imports, etc is runtime. You can pick a class and patch it. You can create classes and functions at runtime; in fact, this happens all the time, and not only for lambdas, but this is how decorators work. A JIT compiler could detect actual usage patterns and replace the code with more efficient versions, until a counter-example is found, and the original "de-optimized" code is run. This is how JavaScript JITs generally work.
- klodolph 2y ago
  It’s a lot easier in C++ or Java. In C++, you make the virtual calls to some base class, and all you need to demonstrate is that only one concrete class is being used at that place. For example, you might see a List<T> in Java, and then the JIT compiler figures out that the field is always assigned = new ArrayList<T>(), which allows it to devirtualize. In Python, the types are way more general. Basically, every method call is being made to “object”. Every field has a value of type “object”. This makes it much more difficult for the compiler to devirtualize anything. It might be much harder to track which code assigns values to a particular field, because objects can easily escape to obscure parts of your codebase and be modified without knowing their type at all. This happens even if you write very boring, Java-like code in Python.
nine_k 2y ago
The point is that Python object model prevents most optimizations, and thus makes the interpreter to executive code exactly as written. All inefficiencies become more obvious this way, while an optimizing compiler could have detected and eliminated many of them in a different language.
- klodolph 2y ago
  Yes, that’s a good point. But I think you can already see the difference in a lot of C++ codebases when you compile at -O0. Until recently, std::move() was a function call, and at -O0, this meant that calls which should be basically a memcpy() ended up having thousands of little function calls in it to std::move().