4 ms·
Good article. Basics that everyone can benefit from knowing. Just one nit/warning... breaking coarse locks into fine-grained locks can be taken too far. Ther
by dbcurtis 7y ago
Good article. Basics that everyone can benefit from knowing.
Just one nit/warning... breaking coarse locks into fine-grained locks can be taken too far. There is a point of diminishing returns where you end up spending increased time acquiring/releasing/waiting-for locks. At some point you want to clump together under a single lock resources that tend to often be used together, even if you often end up locking an extra resource or two unnecessarily. As always, benchmark workloads are your friends.
- hinkley 7y agoNot just performance, but the moment there are two locks required for any task, you may have the Dining Philosophers problem.
- jacobush 7y agoTaken to the extreme, ONE lock in Python. :)
- josalhor 7y agoI know this is a joke, but you still need locks in Python
- bch 7y agoTheoretically not necessarily the referenced GIL, though. There are conceptually alternative models; see Microsoft doc[0] on apartment threading model, or Tcl[1]. [0] https://docs.microsoft.com/en-us/windows/win32/com/processes--threads--and-apartments https://docs.microsoft.com/en-us/windows/win32/com/processes... [1] https://stackoverflow.com/questions/45799121/runtimeerror-calling-tcl-from-different-appartment-tkinter-and-threading#45803955 https://stackoverflow.com/questions/45799121/runtimeerror-ca...
- ajross 7y agoI didn't read it as a joke, you're just operating at a different abstraction level. The CPython interpreter famously uses a single global interpreter lock to protect the language internals and runtime, so it has trouble scaling beyond a single CPU on interpreter-heavy workloads. You're saying that threads in python scripts can be arbitrarily preempted, and so locking is required to protect them against each other, which is also true.
- dbcurtis 7y agoHa Ha! Yes. The gillectomy project collected some interesting data on that a couple of years ago, creating very fine-grained locks and measuring. Not unexpectedly, with a zillion locks performance suffered. The reason though, had much to do with how much it thrashed the cache on a modern CPU. Lots of cache invalidation traffic back and forth between cores. The C-Python implementation is architected around the GIL, and that is a tough problem to crack. My favorite too-many-locks story goes back quite a long time, a former coworker is an old-time Unix guru, who was at Sequent back in the day when 8 CPU's was kind of a big deal. He spent about 9 months on a project putting lots of fine-grained locks into their Unix kernel. After shipping that, his next project was 6 months taking about 25% of them out :)
- hinkley 7y ago75% correct is not a bad ratio.