4 ms·

Interesting! Same here. Also with pypy3, putting it in a function about doubles the speed: time pypy3 min_in_function.py 4999874750 real 0m0.0

by mg 2y ago

Interesting! Same here. Also with pypy3, putting it in a function about doubles the speed: time pypy3 min_in_function.py 4999874750 real 0m0.045s

eesmith 2y ago
Back in Python 1.x, function locals used a dictionary, just like module globals. This made it possible do to: def f(): from math import * return cos(3.0) https://peps.python.org/pep-0227/#import-used-in-function-scope https://peps.python.org/pep-0227/#import-used-in-function-sc... notes "The language reference specifies that import * may only occur in a module scope. (Sec. 6.11) The implementation of C Python has supported import * at the function scope." Python 2.1 added static nested scopes (see the PEP 227 link), which enabled a much faster locals lookup. With that one case excluded, all of the locals can be determined at parse/byte-compile time, stored in a simple vector, and referenced by offset: >>> z = 0 >>> def f(x): ... y = x + z ... return y * 2 ... >>> import dis >>> dis.dis(f) 1 0 RESUME 0 2 2 LOAD_FAST 0 (x) 4 LOAD_GLOBAL 0 (z) 14 BINARY_OP 0 (+) 18 STORE_FAST 1 (y) 3 20 LOAD_FAST 1 (y) 22 LOAD_CONST 1 (2) 24 BINARY_OP 5 (*) 28 RETURN_VALUE The "LOAD_FAST" uses offset 0 to store "x" and offset 1 to store "y", while "z" required a LOAD_GLOBAL to find z in the globals() dictionary.
- mg 2y ago
  Yes, I stumbled across this quite recently: https://x.com/marekgibney/status/1817495719520940236 https://x.com/marekgibney/status/1817495719520940236 I was surprised that you cannot access dynamically created local variables even though they exist. Looks like the local dictionary still exists. Just that whether accessing a local variable looks it up in the local or global dictionary is determined at compile time.
  - eesmith 2y ago
    See PEP 667, "Consistent views of namespaces" https://peps.python.org/pep-0667/ https://peps.python.org/pep-0667/ . In Python 3.12, quoting https://docs.python.org/dev/whatsnew/3.13.html https://docs.python.org/dev/whatsnew/3.13.html > PEP 667: The locals() builtin now has defined semantics when mutating the returned mapping. Python debuggers and similar tools may now more reliably update local variables in optimized scopes even during concurrent code execution. with more details at https://docs.python.org/dev/whatsnew/3.13.html#whatsnew313-locals-semantics https://docs.python.org/dev/whatsnew/3.13.html#whatsnew313-l... , which adds: > To ensure debuggers and similar tools can reliably update local variables in scopes affected by this change, FrameType.f_locals now returns a write-through proxy to the frame’s local and locally referenced nonlocal variables in these scopes
    - mg 2y ago
      Interesting. As I understand it, it will not affect my example: def f(): exec('y = 7') # will still create a local variable print(locals()['y']) # will still print this local variable print(y) # will still try to access a global variable 'y'
      - eesmith 2y ago
        That does not work for me in Python 3.10 or 3.12: Python 3.12.2 ... Type "help", "copyright", "credits" or "license" for more information. >>> def f(): ... exec('y = 7') # will still create a local variable ... print(locals()['y']) # will still print this local variable ... print(y) # will still try to access a global variable 'y' ... >>> f() 7 Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 4, in f NameError: name 'y' is not defined It does print 7 in Python 2.7.
        mg 2y ago
        That's what I meant. It tries to access a global variable 'y' and fails because there is none. See also: https://x.com/marekgibney/status/1817495719520940236 https://x.com/marekgibney/status/1817495719520940236 and https://x.com/marekgibney/status/1817495721974583569 https://x.com/marekgibney/status/1817495721974583569
deleted 2y ago
[deleted]
cb321 2y ago
I got very little speed-up on pypy from being in a function (0.043 vs 0.033), but I got a 1.8X speed-up for py2 vs py3 (0.7686/.4322 = 1.78). (All programs emit 4999874750 as expected.) EDIT: This uses a Nim program https://github.com/c-blake/bu/blob/main/doc/ru.md https://github.com/c-blake/bu/blob/main/doc/ru.md run as `ru -t`, but for the very fast variants you can get a more precise wall time from https://github.com/c-blake/bu/blob/main/doc/tim.md https://github.com/c-blake/bu/blob/main/doc/tim.md pypy3_10-7.3.16_p1 - (first form,2nd form) x same in a func TM 0.045421363 wall 0.034517 usr 0.010855 sys 99.9 % 56476 mxRS minCall TM 0.043447211 wall 0.029492 usr 0.013763 sys 99.6 % 56124 mxRS ifStmt TM 0.033408064 wall 0.020605 usr 0.012758 sys 99.9 % 55960 mxRS minCallInFunc TM 0.033115636 wall 0.026059 usr 0.007018 sys 99.9 % 55956 mxRS ifStmtInFunc python-2.7.18_p16 TM 1.755861105 wall 1.751740 usr 0.001991 sys 99.9 % 5944 mxRS TM 0.940162912 wall 0.935362 usr 0.003988 sys 99.9 % 5916 mxRS TM 1.155755936 wall 1.151529 usr 0.002993 sys 99.9 % 5928 mxRS TM 0.432215672 wall 0.430851 usr 0.000998 sys 99.9 % 5800 mxRS python-3.11.9_p1 TM 2.678891504 wall 2.675141 usr 0.000994 sys 99.9 % 7992 mxRS TM 1.348009364 wall 1.344612 usr 0.001995 sys 99.9 % 7864 mxRS TM 2.018986091 wall 2.014702 usr 0.001994 sys 99.9 % 7972 mxRS TM 0.768633122 wall 0.766915 usr 0.000997 sys 99.9 % 7980 mxRS I recall when Python3 was first going through its growing pains in the mid noughties that promises of eventually clawing back performance. This clawback now seems to have been a fantasy (or perhaps they have just piled on so many new features that they clawed back and then regressed?). Anyway, Nim is even faster than PyPy and uses less memory than either version of CPython: doIt.nim: proc doIt() = var i = 10000000 var r = 0 while i > 0: i = i - 1 r += min(i, 500) echo r doIt() #TM 0.024735 wall 0.024743 usr 0.000000 sys 100.0% 1.504 mxRM In terms of "how Python-like is Nim", all I did was change `def` to `proc` and add the 2 `var`s and change `print` -> `echo`. { EDIT: though if you love Py print(), there is always https://github.com/c-blake/cligen/blob/master/cligen/print.nim https://github.com/c-blake/cligen/blob/master/cligen/print.n... or some other roll-your-own idea. Then instead of py2/py3 print x,y vs print(x,y) you can actually do either one in Nim since its call syntax is so flexible. } It is perhaps noteworthy that, if the values are realized in some kind of array rather than generated by a loop, that modern CPUs can do this kind of calculation well with SIMD and compilers like gcc can even recognize the constructs and auto-vectorize. Of course, needing to load data costs memory bandwidth which may not be great compared to SIMD instruction throughput at scales past 80 MiB as in this problem.
igouy 2y ago
too simple https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/toosimple.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

by mg 2y ago

Interesting! Same here. Also with pypy3, putting it in a function about doubles the speed: time pypy3 min_in_function.py 4999874750 real 0m0.045s

eesmith 2y ago

Back in Python 1.x, function locals used a dictionary, just like module globals. This made it possible do to: def f(): from math import * return cos(3.0) https://peps.python.org/pep-0227/#import-used-in-function-scope https://peps.python.org/pep-0227/#import-used-in-function-sc... notes "The language reference specifies that import * may only occur in a module scope. (Sec. 6.11) The implementation of C Python has supported import * at the function scope." Python 2.1 added static nested scopes (see the PEP 227 link), which enabled a much faster locals lookup. With that one case excluded, all of the locals can be determined at parse/byte-compile time, stored in a simple vector, and referenced by offset: >>> z = 0 >>> def f(x): ... y = x + z ... return y * 2 ... >>> import dis >>> dis.dis(f) 1 0 RESUME 0 2 2 LOAD_FAST 0 (x) 4 LOAD_GLOBAL 0 (z) 14 BINARY_OP 0 (+) 18 STORE_FAST 1 (y) 3 20 LOAD_FAST 1 (y) 22 LOAD_CONST 1 (2) 24 BINARY_OP 5 (*) 28 RETURN_VALUE The "LOAD_FAST" uses offset 0 to store "x" and offset 1 to store "y", while "z" required a LOAD_GLOBAL to find z in the globals() dictionary.

mg 2y ago
Yes, I stumbled across this quite recently: https://x.com/marekgibney/status/1817495719520940236 https://x.com/marekgibney/status/1817495719520940236 I was surprised that you cannot access dynamically created local variables even though they exist. Looks like the local dictionary still exists. Just that whether accessing a local variable looks it up in the local or global dictionary is determined at compile time.
- eesmith 2y ago
  See PEP 667, "Consistent views of namespaces" https://peps.python.org/pep-0667/ https://peps.python.org/pep-0667/ . In Python 3.12, quoting https://docs.python.org/dev/whatsnew/3.13.html https://docs.python.org/dev/whatsnew/3.13.html > PEP 667: The locals() builtin now has defined semantics when mutating the returned mapping. Python debuggers and similar tools may now more reliably update local variables in optimized scopes even during concurrent code execution. with more details at https://docs.python.org/dev/whatsnew/3.13.html#whatsnew313-locals-semantics https://docs.python.org/dev/whatsnew/3.13.html#whatsnew313-l... , which adds: > To ensure debuggers and similar tools can reliably update local variables in scopes affected by this change, FrameType.f_locals now returns a write-through proxy to the frame’s local and locally referenced nonlocal variables in these scopes
  - mg 2y ago
    Interesting. As I understand it, it will not affect my example: def f(): exec('y = 7') # will still create a local variable print(locals()['y']) # will still print this local variable print(y) # will still try to access a global variable 'y'
    - eesmith 2y ago
      That does not work for me in Python 3.10 or 3.12: Python 3.12.2 ... Type "help", "copyright", "credits" or "license" for more information. >>> def f(): ... exec('y = 7') # will still create a local variable ... print(locals()['y']) # will still print this local variable ... print(y) # will still try to access a global variable 'y' ... >>> f() 7 Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 4, in f NameError: name 'y' is not defined It does print 7 in Python 2.7.
      - mg 2y ago
        That's what I meant. It tries to access a global variable 'y' and fails because there is none. See also: https://x.com/marekgibney/status/1817495719520940236 https://x.com/marekgibney/status/1817495719520940236 and https://x.com/marekgibney/status/1817495721974583569 https://x.com/marekgibney/status/1817495721974583569

deleted 2y ago

[deleted]

cb321 2y ago

I got very little speed-up on pypy from being in a function (0.043 vs 0.033), but I got a 1.8X speed-up for py2 vs py3 (0.7686/.4322 = 1.78). (All programs emit 4999874750 as expected.) EDIT: This uses a Nim program https://github.com/c-blake/bu/blob/main/doc/ru.md https://github.com/c-blake/bu/blob/main/doc/ru.md run as `ru -t`, but for the very fast variants you can get a more precise wall time from https://github.com/c-blake/bu/blob/main/doc/tim.md https://github.com/c-blake/bu/blob/main/doc/tim.md pypy3_10-7.3.16_p1 - (first form,2nd form) x same in a func TM 0.045421363 wall 0.034517 usr 0.010855 sys 99.9 % 56476 mxRS minCall TM 0.043447211 wall 0.029492 usr 0.013763 sys 99.6 % 56124 mxRS ifStmt TM 0.033408064 wall 0.020605 usr 0.012758 sys 99.9 % 55960 mxRS minCallInFunc TM 0.033115636 wall 0.026059 usr 0.007018 sys 99.9 % 55956 mxRS ifStmtInFunc python-2.7.18_p16 TM 1.755861105 wall 1.751740 usr 0.001991 sys 99.9 % 5944 mxRS TM 0.940162912 wall 0.935362 usr 0.003988 sys 99.9 % 5916 mxRS TM 1.155755936 wall 1.151529 usr 0.002993 sys 99.9 % 5928 mxRS TM 0.432215672 wall 0.430851 usr 0.000998 sys 99.9 % 5800 mxRS python-3.11.9_p1 TM 2.678891504 wall 2.675141 usr 0.000994 sys 99.9 % 7992 mxRS TM 1.348009364 wall 1.344612 usr 0.001995 sys 99.9 % 7864 mxRS TM 2.018986091 wall 2.014702 usr 0.001994 sys 99.9 % 7972 mxRS TM 0.768633122 wall 0.766915 usr 0.000997 sys 99.9 % 7980 mxRS I recall when Python3 was first going through its growing pains in the mid noughties that promises of eventually clawing back performance. This clawback now seems to have been a fantasy (or perhaps they have just piled on so many new features that they clawed back and then regressed?). Anyway, Nim is even faster than PyPy and uses less memory than either version of CPython: doIt.nim: proc doIt() = var i = 10000000 var r = 0 while i > 0: i = i - 1 r += min(i, 500) echo r doIt() #TM 0.024735 wall 0.024743 usr 0.000000 sys 100.0% 1.504 mxRM In terms of "how Python-like is Nim", all I did was change `def` to `proc` and add the 2 `var`s and change `print` -> `echo`. { EDIT: though if you love Py print(), there is always https://github.com/c-blake/cligen/blob/master/cligen/print.nim https://github.com/c-blake/cligen/blob/master/cligen/print.n... or some other roll-your-own idea. Then instead of py2/py3 print x,y vs print(x,y) you can actually do either one in Nim since its call syntax is so flexible. } It is perhaps noteworthy that, if the values are realized in some kind of array rather than generated by a loop, that modern CPUs can do this kind of calculation well with SIMD and compilers like gcc can even recognize the constructs and auto-vectorize. Of course, needing to load data costs memory bandwidth which may not be great compared to SIMD instruction throughput at scales past 80 MiB as in this problem.

igouy 2y ago

too simple https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/toosimple.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...