3 ms·

While optimizing Python code is indeed really hard, this is not a good example of why. It uses implementation-specific details which are outside of the scope o

While optimizing Python code is indeed really hard, this is not a good example of why. It uses implementation-specific details which are outside of the scope of anything to do with Python semantics. It's roughly equivalent to: #include <stdio.h> int nine = 9; int main(void) { printf("nine = %d\n", nine); return 0; } /* in a library */ __attribute__((constructor)) static void sneaky(void) { int *n = (int *) &nine; *n = 8; } Your hyperbole simply isn't true, as demonstrated by the many static code analysis tools for Python. They can't handle all cases, certainly, but they demonstrate it's mostly possible to give semantics for Python code without running it.

derriz 3y ago
I dunno, the fact that integers are by (often mutable) references has to make it really difficult for optimization. You don't have to be "sneaky" for this to bite you with Python. Maybe it looks obvious when stated in a bare-bones fashion but this bug was not easy to track down in a larger code base: i = 1 incr_by_1 = lambda x : x + i i = 4 incr_by_4 = lambda x : x + i i is a reference in both incr_by_1 and incr_by_4 are equivalent at this point. If anyone assigns to i, then their behaviour will change. In most languages, integers are values so an optimiser has a chance to (for example) replace incr1 by a single CPU increment instruction but can't do it here as the value "pointed to" by i needs to be fetched according to Python semantics.
- eesmith 3y ago
  Agreed! Optimizing Python code is indeed really hard, and the lack of const and ability to describe capture semantics don't help. To be fair, the equivalent in C++ is: #include <iostream> int main() { int i; i = 1; auto incr_by_1 = [&](int x) {return x + i;}; i = 4; auto incr_by_4 = [&](int x) {return x + i;}; std::cout << incr_by_1(0) << " and " << incr_by_4(0) << std::endl; return 0; } which prints "4 and 4". Replace the first [&] with [i] and it prints "1 and 4". A Python implementation also can't replace incr1 by a single CPU increment instruction because it doesn't know the type of x. That's still a far cry from being unable to give semantics for Python code without running it.

derriz 3y ago

I dunno, the fact that integers are by (often mutable) references has to make it really difficult for optimization. You don't have to be "sneaky" for this to bite you with Python. Maybe it looks obvious when stated in a bare-bones fashion but this bug was not easy to track down in a larger code base: i = 1 incr_by_1 = lambda x : x + i i = 4 incr_by_4 = lambda x : x + i i is a reference in both incr_by_1 and incr_by_4 are equivalent at this point. If anyone assigns to i, then their behaviour will change. In most languages, integers are values so an optimiser has a chance to (for example) replace incr1 by a single CPU increment instruction but can't do it here as the value "pointed to" by i needs to be fetched according to Python semantics.

eesmith 3y ago
Agreed! Optimizing Python code is indeed really hard, and the lack of const and ability to describe capture semantics don't help. To be fair, the equivalent in C++ is: #include <iostream> int main() { int i; i = 1; auto incr_by_1 = [&](int x) {return x + i;}; i = 4; auto incr_by_4 = [&](int x) {return x + i;}; std::cout << incr_by_1(0) << " and " << incr_by_4(0) << std::endl; return 0; } which prints "4 and 4". Replace the first [&] with [i] and it prints "1 and 4". A Python implementation also can't replace incr1 by a single CPU increment instruction because it doesn't know the type of x. That's still a far cry from being unable to give semantics for Python code without running it.