3 ms·

And this is why optimizing Python code is really hard. When at runtime you can change almost any aspect of the language it's virtually impossible to give a sema

And this is why optimizing Python code is really hard. When at runtime you can change almost any aspect of the language it's virtually impossible to give a semantics for Python code beyond "run it and find out".

stuaxo 3y ago
Ok, though most of the time this isn't the kind of thing that changes.
eesmith 3y ago
While optimizing Python code is indeed really hard, this is not a good example of why. It uses implementation-specific details which are outside of the scope of anything to do with Python semantics. It's roughly equivalent to: #include <stdio.h> int nine = 9; int main(void) { printf("nine = %d\n", nine); return 0; } /* in a library */ __attribute__((constructor)) static void sneaky(void) { int *n = (int *) &nine; *n = 8; } Your hyperbole simply isn't true, as demonstrated by the many static code analysis tools for Python. They can't handle all cases, certainly, but they demonstrate it's mostly possible to give semantics for Python code without running it.
- derriz 3y ago
  I dunno, the fact that integers are by (often mutable) references has to make it really difficult for optimization. You don't have to be "sneaky" for this to bite you with Python. Maybe it looks obvious when stated in a bare-bones fashion but this bug was not easy to track down in a larger code base: i = 1 incr_by_1 = lambda x : x + i i = 4 incr_by_4 = lambda x : x + i i is a reference in both incr_by_1 and incr_by_4 are equivalent at this point. If anyone assigns to i, then their behaviour will change. In most languages, integers are values so an optimiser has a chance to (for example) replace incr1 by a single CPU increment instruction but can't do it here as the value "pointed to" by i needs to be fetched according to Python semantics.
  - eesmith 3y ago
    Agreed! Optimizing Python code is indeed really hard, and the lack of const and ability to describe capture semantics don't help. To be fair, the equivalent in C++ is: #include <iostream> int main() { int i; i = 1; auto incr_by_1 = [&](int x) {return x + i;}; i = 4; auto incr_by_4 = [&](int x) {return x + i;}; std::cout << incr_by_1(0) << " and " << incr_by_4(0) << std::endl; return 0; } which prints "4 and 4". Replace the first [&] with [i] and it prints "1 and 4". A Python implementation also can't replace incr1 by a single CPU increment instruction because it doesn't know the type of x. That's still a far cry from being unable to give semantics for Python code without running it.
formerly_proven 3y ago
Not really. Stuff like this is shown around from time to time as a massive "gotcha" kind of thing for a few languages, but it's really just the nature of boxed primitives and interning (=literal bindings).

stuaxo 3y ago

Ok, though most of the time this isn't the kind of thing that changes.

eesmith 3y ago

While optimizing Python code is indeed really hard, this is not a good example of why. It uses implementation-specific details which are outside of the scope of anything to do with Python semantics. It's roughly equivalent to: #include <stdio.h> int nine = 9; int main(void) { printf("nine = %d\n", nine); return 0; } /* in a library */ __attribute__((constructor)) static void sneaky(void) { int *n = (int *) &nine; *n = 8; } Your hyperbole simply isn't true, as demonstrated by the many static code analysis tools for Python. They can't handle all cases, certainly, but they demonstrate it's mostly possible to give semantics for Python code without running it.

derriz 3y ago
I dunno, the fact that integers are by (often mutable) references has to make it really difficult for optimization. You don't have to be "sneaky" for this to bite you with Python. Maybe it looks obvious when stated in a bare-bones fashion but this bug was not easy to track down in a larger code base: i = 1 incr_by_1 = lambda x : x + i i = 4 incr_by_4 = lambda x : x + i i is a reference in both incr_by_1 and incr_by_4 are equivalent at this point. If anyone assigns to i, then their behaviour will change. In most languages, integers are values so an optimiser has a chance to (for example) replace incr1 by a single CPU increment instruction but can't do it here as the value "pointed to" by i needs to be fetched according to Python semantics.
- eesmith 3y ago
  Agreed! Optimizing Python code is indeed really hard, and the lack of const and ability to describe capture semantics don't help. To be fair, the equivalent in C++ is: #include <iostream> int main() { int i; i = 1; auto incr_by_1 = [&](int x) {return x + i;}; i = 4; auto incr_by_4 = [&](int x) {return x + i;}; std::cout << incr_by_1(0) << " and " << incr_by_4(0) << std::endl; return 0; } which prints "4 and 4". Replace the first [&] with [i] and it prints "1 and 4". A Python implementation also can't replace incr1 by a single CPU increment instruction because it doesn't know the type of x. That's still a far cry from being unable to give semantics for Python code without running it.

formerly_proven 3y ago

Not really. Stuff like this is shown around from time to time as a massive "gotcha" kind of thing for a few languages, but it's really just the nature of boxed primitives and interning (=literal bindings).