5 ms·
Accessing a global object as the most simple benchmark that in fact exercises the locks still shows a massive slowdown that is not offset by the moderate genera
by Zsfe510asG 1y ago
Accessing a global object as the most simple benchmark that in fact exercises the locks still shows a massive slowdown that is not offset by the moderate general speedups since 3.9:
x = 0
def f():
global x
for i in range(100000000):
x += i
f()
print(x)
Results:
3.9: 7.1s
3.11: 5.9s
3.14: 6.5s
3.14-nogil: 8.4s
That is a NOGIL slowdown of 18% compared to 3.9, 44% compared to 3.11 and 30% compared to 3.14. These numbers are in line with previous attempts at GIL removal that were rejected because they didn't come from Facebook.
Please do not complain about the global object. Using a pure function would obviously be a useless benchmark for locking and real world Python code bases have far more intricate access patterns.
- logicchains 1y ago>Please do not complain about the global object. Using a pure function would obviously be a useless benchmark for locking and real world Python code bases have far more intricate access patterns. Just because there's a lot of shit Python code out there, doesn't mean people who want to write clean, performant Python code should suffer for it.
- deleted 1y ago[deleted]
- nine_k 1y agoThose who's going to suffer are the people who inherited a ton of legacy Python code written in this style.
- NeutralForest 1y agoAs mentioned in the article, others might have different constraints that make the GIL worth it for them; since both versions of Python are available anyways, it's a win in my book.
- mynewaccount00 1y agoyou realize this is not a concern because nobody in the last 20 years uses global?
- kg 1y agoAn incrementing global counter is a pretty common scenario if your goal is to have guaranteed-unique IDs assigned to objects within a process, especially if you want them to be sequential too. I've got counters like that in various parts of code I've shipped, typically incremented using atomics.
- sillythrowaway7 1y agoIt is a pretty common scenario if you don’t follow almost two decades worth of best practices and suggested alternatives and advice
- immibis 1y agoCan you show us some of this advice?
- mynewaccount00 1y ago... Not using globals? Pass the value around, use recursion, and keep state as local as possible.
- immibis 1y agoI'd like you to show: How to write Python code with no globals And a source other than you saying you should do it this way
- pphysch 1y agoIs that global counter really the only dependency? No database connections or anything that would warrant some proper dependency injection?
- adamzwasserman 1y agoI will complain about the global object. Even though technically, everything in Python is an object, I feel strongly that programmers should avoid OOP in Python like the plague. Every object is a petri dish for state corruption. Thee is a very solid list of reasons to use pure functions with explicit passing wherever humanly possible, and I personally believe there is no comparable list of reason to use OOP. * Stack-allocated primitives need no refcounting * Immutable structures reduce synchronization * Data locality improves when you pass arrays/structs rather than object graphs * Pure functions can be parallelized without locks
- nine_k 1y agoI'm 100% on board regarding the use of pure functions and eschewing shared mutable state wherever practical. (Local mutable state is fine.) But OOP does not necessitate mutable state; you can do OOP with immutable objects and pure methods (except the constructor). Objects are collections of partially-applied functions (which is explicit in Python) that also conceal internal details within its own namespace. It is convenient in certain cases.
- adamzwasserman 1y agoI agree. I sometimes do that. But very rarely.
- int_19h 1y agoPython doesn't even have frozendicts out of the box. If you want immutability as a lynchpin, you need to look at a different language. This one is very much not designed for it.
- wenc 1y agoGlobals are not commonly used except in retrofit cases of legacy code or in beginner code. I haven’t seen or used a global more than once in my 20 years of writing Python.
- nilamo 1y agoFlask's request context is a global object. And since this is specifically about web services, flask seems highly relevant to the conversation.
- treyd 1y agoThat's correct. Flask has a global request context object, so by design it can only safely handle a single request at a time per Python interpreter. If you want to parallelize multiple Flask servers, you spin up multiple interpreters. Web services in Python that want to handle multiple comcurrent requests in the same interpreter should be using a web framework that is designed around that expectation and don't use a global request context object, such as FastAPI.
- formerly_proven 1y agoNo, Flask has global objects wrapping state objects in contextvars.
- nine_k 1y agoThis is even more funny, because now you need to switch the global reference according to the context. With GIL, it is easy; without it,...
- formerly_proven 1y agoYou misunderstand. The "request" or "g" objects in Flask are proxies which access the actual objects through contextvars, which are effectively thread-local storage with some extra sugar. The context stack of a contextvar is already within the TLS and therefore always bound to a specific thread.
- immibis 1y ago> Using a pure function would obviously be a useless benchmark for locking But isn't that the point? Previously, pure functions would all lock, so you could only run one pure function at a time. Now they don't.
- byroot 1y agoI haven’t seriously used Python in over 15 years, but I assume the comparison is against using a preforking server with 1+ process per core. The question is whether 1+ thread per core with GIL free Python perform as well as 1+ process per core with GIL. My understanding is that this global is just a way to demonstrate that the finely grained locking in the GIL free version may make it so that preforking servers may still be more performant.
- greatgib 1y agoSad that you did not test with 3.12 that is a lot better then 3.11!
- josefx 1y agoIf you are accessing a global in a loop you might want to assign it to a local variable first. From what I remember that should result in a speedup in all python versions.