5 ms·
It's concurrent not parallel. The switch won't happen inside the execution of one opcode including some dictionary update operations so it's safe in many cases
by lvass 5y ago
It's concurrent not parallel. The switch won't happen inside the execution of one opcode including some dictionary update operations so it's safe in many cases where parallel execution isn't.
- zelphirkalt 5y agoIt is probably a bad practice to not acquire a mutex for that concurrent dictionary update. The code should be improved in that regard, with or without any potential Python language change.
- lvass 5y agoIf needed, I'll probably just change it to asyncio, probably saner than inspecting everything for new parallelism bugs which can be incredibly subtle.
- ajkjk 5y agoIf performance isn't hugely important you could make blanket-locking wrappers around common data structures and swap them in-place for all of your global state. .. but, as I said, removing the GIL will almost certainly be opt-in.
- ajkjk 5y agoYes, single-instruction operations would be fine, but if you're writing multithreaded code you are probably doing things that the GIL doesn't protect all the time. Like dict-updates on classes that implement __set__, or `if not a[x]: a[x] = y` sorts of two-phased checks, or just like, anything else. You can't get very far with global state without reckoning with concurrency, GIL or not. I assume that a change to relax the GIL will both allow you to opt-out of it, and allow you to use locking versions of primitive data-structures, anyway; it's not like it's going to just vanish overnight with no guardrails.
- colinmhayes 5y agoSeems like checking the docs for atomicity of every operation is a huge pain in the ass.
- ajkjk 5y agoIt really is, they don't make it clear at all. Every time I have to ask the question of "is this atomic under the GIL" I struggle to find the right answer.
- kzrdude 5y agoBest to avoid sharing data, and never mutating shared data! Rust has a great rule: Sharing XOR Mutation. Python is higher level, so message passing and passing "owned" values between threads is all the more feasible and sensible.
- ynik 5y agoCurrently, dictionary updates are atomic only if the keys are primitive types. `dict.update()` will call methods like `__eq__`, and those methods (if implemented in Python) may temporarily release the GIL.
- kzrdude 5y agoUnfortunately the Python docs says that dict.update() is atomic. It's being fixed though... It came out during these discussions.