5 ms·

Call me optimistically skeptical. I share similar reservations about GIL obsession with the original comment author, but if this is true: > The overall effect

Call me optimistically skeptical. I share similar reservations about GIL obsession with the original comment author, but if this is true: > The overall effect of this change, and a number of others with it, actually boosts single-threaded performance slightly—by around 10% Then it sounds like having the cake and eating it too (optimism). Although my experience keeps nagging at me with, "there is not such thing as a free lunch" (skepticism).

comex 5y ago
Perhaps better coverage on LWN: https://lwn.net/Articles/872869/ https://lwn.net/Articles/872869/ The no-GIL version is actually about 8% slower on single-threaded performance than the GIL version, but the author bundled in some unrelated performance improvements that make the no-GIL version overall 10% faster than today's Python.
- qeternity 5y ago
  This suggests that the unrelated patches improve perf by 20% (0.92 * 1.20 ~= 1.10) I would love to be proven wrong but I am skeptical.
  - selcuka 5y ago
    That is already explained by the author [1]: > The resulting interpreter is about 9% faster than the no-GIL proof-of-concept (or ~19% faster than CPython 3.9.0a3). That 9% difference between the “nogil” interpreter and the stripped-down “nogil” interpreter can be thought of as the “cost” of the major GIL-removal changes. [1] https://docs.google.com/document/d/18CXhDb1ygxg-YXNBJNzfzZsDFosB5e6BfnXLlejd9l0/edit https://docs.google.com/document/d/18CXhDb1ygxg-YXNBJNzfzZsD...
    - qeternity 5y ago
      Thanks for this. It’s interesting because I think many people (myself included) would be far more interested in the perf patches than the GILectomy.
  - guenthert 5y ago
    Why? It's not like CPython is a speed daemon. I'd think there are some low hanging fruits, simply because performance is such a low priority for the maintainers. It doesn't even do TCO after all.
- vitus 5y ago
  Subscriber link from Twitter if you (like me) ran into a paywall: https://lwn.net/SubscriberLink/872869/0e62bba2db51ec7a/ https://lwn.net/SubscriberLink/872869/0e62bba2db51ec7a/
  - ignoramous 5y ago
    also: https://archive.is/1gLVY https://archive.is/1gLVY
- Ph0X 5y ago
  Right, the 20% boost is unrelated to the Gilectomy. > though, as Guido van Rossum noted, the Python developers could always just take the performance improvements without the concurrency work and be even faster yet. Why be 10% faster single threaded when you can be 20% faster single threaded!
atoav 5y ago
> "there is not such thing as a free lunch" (skepticism). When you carry a heavy suitcase filled with lead and you drop it, things get lighter for free. You paid for it by carrying the damn thing around with you for the whole time.
thaumasiotes 5y ago
> Although my experience keeps nagging at me with, "there is not such thing as a free lunch" (skepticism). Well, yeah, someone had to make the changes. That's the cost that was paid. You can get a mass-produced machete that is cheaper and higher-quality than a 7th-century sword. It's easy for one thing to be better than another thing across several dimensions simultaneously. That's why certain technologies go out of use -- they have negative value compared to other technologies. But that has nothing to do with the principle that there's no such thing as a free lunch.
- gshulegaard 5y ago
  I feel like you aren't well informed on why removing the GIL results in a single-threaded performance hit. And while I think it's always nice to keep in mind the developer effort required, it's not the only cost as GIL removal has been done before (several times, even as far back as Python 1.5 [1]). The crux of the issue (as I understand it) is that the GIL absolves the Python interpreter of downstream memory access control. You can replace the GIL with memory access controls of various strategies, but the overhead of that access control is just that: overhead. In a multi-threaded program the concurrency gains should outweigh that overhead, but in a single-threaded one it's just extra work that wasn't being done before. Which brings us back to no free lunch. It turns out that the claim "10%" faster without the GIL is actually a result of Gross (GIL removal author) doing a multitude of unrelated performance improvements. These performance improvements increase performance enough that the performance of single-threaded no GIL code (with overhead) is ~10% higher than today. But as Guido pointed out, the core developers could upstream the performance improvements without the GIL removal: > To be clear, Sam’s basic approach is a bit slower for single-threaded code, and he admits that. But to sweeten the pot he has also applied a bunch of unrelated speedups that make it faster in general, so that overall it’s always a win. But presumably we could upstream the latter easily, separately from the GIL-freeing part. [2] [1] https://docs.python.org/3/faq/library.html#can-t-we-get-rid-of-the-global-interpreter-lock https://docs.python.org/3/faq/library.html#can-t-we-get-rid-... [2] https://lwn.net/ml/python-dev/CAP7+vJJ1hzXiyDwVs6-eXed+DtodHJ==Ck680DxAyoiBSXZDQQ@mail.gmail.com/ https://lwn.net/ml/python-dev/CAP7+vJJ1hzXiyDwVs6-eXed+DtodH...
  - 5y ago

comex 5y ago

Perhaps better coverage on LWN: https://lwn.net/Articles/872869/ https://lwn.net/Articles/872869/ The no-GIL version is actually about 8% slower on single-threaded performance than the GIL version, but the author bundled in some unrelated performance improvements that make the no-GIL version overall 10% faster than today's Python.

qeternity 5y ago
This suggests that the unrelated patches improve perf by 20% (0.92 * 1.20 ~= 1.10) I would love to be proven wrong but I am skeptical.
- selcuka 5y ago
  That is already explained by the author [1]: > The resulting interpreter is about 9% faster than the no-GIL proof-of-concept (or ~19% faster than CPython 3.9.0a3). That 9% difference between the “nogil” interpreter and the stripped-down “nogil” interpreter can be thought of as the “cost” of the major GIL-removal changes. [1] https://docs.google.com/document/d/18CXhDb1ygxg-YXNBJNzfzZsDFosB5e6BfnXLlejd9l0/edit https://docs.google.com/document/d/18CXhDb1ygxg-YXNBJNzfzZsD...
  - qeternity 5y ago
    Thanks for this. It’s interesting because I think many people (myself included) would be far more interested in the perf patches than the GILectomy.
- guenthert 5y ago
  Why? It's not like CPython is a speed daemon. I'd think there are some low hanging fruits, simply because performance is such a low priority for the maintainers. It doesn't even do TCO after all.
vitus 5y ago
Subscriber link from Twitter if you (like me) ran into a paywall: https://lwn.net/SubscriberLink/872869/0e62bba2db51ec7a/ https://lwn.net/SubscriberLink/872869/0e62bba2db51ec7a/
- ignoramous 5y ago
  also: https://archive.is/1gLVY https://archive.is/1gLVY
Ph0X 5y ago
Right, the 20% boost is unrelated to the Gilectomy. > though, as Guido van Rossum noted, the Python developers could always just take the performance improvements without the concurrency work and be even faster yet. Why be 10% faster single threaded when you can be 20% faster single threaded!

atoav 5y ago

> "there is not such thing as a free lunch" (skepticism). When you carry a heavy suitcase filled with lead and you drop it, things get lighter for free. You paid for it by carrying the damn thing around with you for the whole time.

thaumasiotes 5y ago

> Although my experience keeps nagging at me with, "there is not such thing as a free lunch" (skepticism). Well, yeah, someone had to make the changes. That's the cost that was paid. You can get a mass-produced machete that is cheaper and higher-quality than a 7th-century sword. It's easy for one thing to be better than another thing across several dimensions simultaneously. That's why certain technologies go out of use -- they have negative value compared to other technologies. But that has nothing to do with the principle that there's no such thing as a free lunch.

gshulegaard 5y ago
I feel like you aren't well informed on why removing the GIL results in a single-threaded performance hit. And while I think it's always nice to keep in mind the developer effort required, it's not the only cost as GIL removal has been done before (several times, even as far back as Python 1.5 [1]). The crux of the issue (as I understand it) is that the GIL absolves the Python interpreter of downstream memory access control. You can replace the GIL with memory access controls of various strategies, but the overhead of that access control is just that: overhead. In a multi-threaded program the concurrency gains should outweigh that overhead, but in a single-threaded one it's just extra work that wasn't being done before. Which brings us back to no free lunch. It turns out that the claim "10%" faster without the GIL is actually a result of Gross (GIL removal author) doing a multitude of unrelated performance improvements. These performance improvements increase performance enough that the performance of single-threaded no GIL code (with overhead) is ~10% higher than today. But as Guido pointed out, the core developers could upstream the performance improvements without the GIL removal: > To be clear, Sam’s basic approach is a bit slower for single-threaded code, and he admits that. But to sweeten the pot he has also applied a bunch of unrelated speedups that make it faster in general, so that overall it’s always a win. But presumably we could upstream the latter easily, separately from the GIL-freeing part. [2] [1] https://docs.python.org/3/faq/library.html#can-t-we-get-rid-of-the-global-interpreter-lock https://docs.python.org/3/faq/library.html#can-t-we-get-rid-... [2] https://lwn.net/ml/python-dev/CAP7+vJJ1hzXiyDwVs6-eXed+DtodHJ==Ck680DxAyoiBSXZDQQ@mail.gmail.com/ https://lwn.net/ml/python-dev/CAP7+vJJ1hzXiyDwVs6-eXed+DtodH...
- 5y ago