3 ms·

An other component of the Gil story is that removing the Gil require adding fine grained locks, which (aside from making VM development more complicated) signif

An other component of the Gil story is that removing the Gil require adding fine grained locks, which (aside from making VM development more complicated) significantly increases lock traffic and thus runtime costs, which noticeably impacts single-threaded performance, which is of major import. Postgres starts from a share-nothing architecture, it’s quite a bit easier to evaluate the addition of sharing.

ComputerGuru 3y ago
> which (aside from making VM development more complicated) significantly increases lock traffic and thus runtime costs, which noticeably impacts single-threaded performance, which is of major import. I don't think that's a fair characterization of the trade offs. Acquiring uncontended mutexes is basically free (and fairly side-effect free) so single-threaded performance will not be noticeably impacted. Every large C project I'm aware of (read: kernels) that has publicly switched from coarse locks to fine-grained locks has considered it to be a huge win with little to no impact on single-threaded performance. You can even gain performance if you chop up objects or allocations into finer-grained blobs to fit your finer-grained locking strategy because it can play nicer with cache friendliness (accessing one bit of code doesn't kick the other bits of code out of the cache).
bsder 3y ago
> which noticeably impacts single-threaded performance, which is of major import. 1) I don't buy this a priori. Almost everybody who removed a gigantic lock suddenly realizes that there was more contention than they thought and that atomizing it made performance improve. 2) Had Python bitten the bullet and removed the GIL back at Python 3.0, the performance would likely already be back to normal or better. You can't optimize hypothetically. Optimization on something like Python is an accumulation of lots of small wins.
- masklinn 3y ago
  > I don't buy this a priori. You don’t have to buy anything, that’s been the result of every attempt so far and a big reason for their rejection. The latest effort only gained some traction because the backers also did optimisation work which compensated (and then was merged separately). > Almost everybody who removed a gigantic lock See that’s the issue with your response, you’re not actually reading the comment you’re replying to. And the “almost” is a big tell. > suddenly realizes that there was more contention than they thought and that atomizing it made performance improve. There is no contention on the gil in single threaded workloads. > Had Python bitten the bullet and removed the GIL back at Python 3.0 It would have taken several more years and been completely DOA.
anarazel 3y ago
Postgres already shares a lot of state between processes via shared memory. There's not a whole lot that would initially change from a concurrency perspective.

ComputerGuru 3y ago

> which (aside from making VM development more complicated) significantly increases lock traffic and thus runtime costs, which noticeably impacts single-threaded performance, which is of major import. I don't think that's a fair characterization of the trade offs. Acquiring uncontended mutexes is basically free (and fairly side-effect free) so single-threaded performance will not be noticeably impacted. Every large C project I'm aware of (read: kernels) that has publicly switched from coarse locks to fine-grained locks has considered it to be a huge win with little to no impact on single-threaded performance. You can even gain performance if you chop up objects or allocations into finer-grained blobs to fit your finer-grained locking strategy because it can play nicer with cache friendliness (accessing one bit of code doesn't kick the other bits of code out of the cache).

bsder 3y ago

> which noticeably impacts single-threaded performance, which is of major import. 1) I don't buy this a priori. Almost everybody who removed a gigantic lock suddenly realizes that there was more contention than they thought and that atomizing it made performance improve. 2) Had Python bitten the bullet and removed the GIL back at Python 3.0, the performance would likely already be back to normal or better. You can't optimize hypothetically. Optimization on something like Python is an accumulation of lots of small wins.

masklinn 3y ago
> I don't buy this a priori. You don’t have to buy anything, that’s been the result of every attempt so far and a big reason for their rejection. The latest effort only gained some traction because the backers also did optimisation work which compensated (and then was merged separately). > Almost everybody who removed a gigantic lock See that’s the issue with your response, you’re not actually reading the comment you’re replying to. And the “almost” is a big tell. > suddenly realizes that there was more contention than they thought and that atomizing it made performance improve. There is no contention on the gil in single threaded workloads. > Had Python bitten the bullet and removed the GIL back at Python 3.0 It would have taken several more years and been completely DOA.

anarazel 3y ago

Postgres already shares a lot of state between processes via shared memory. There's not a whole lot that would initially change from a concurrency perspective.