5 ms·

So glad they are doing this and it’s just the start from what I understand. We also may have actual concurrency in python soon.

gshulegaard 4y ago
Can you elaborate on what you mean by "actual concurrency in python soon"?
- AlphaSite 4y ago
  There was an experimental branch where someone reimplemented Python 3.9 sans the Gil and managed to maintain performance parity with the Gil’ed version. It’s likely this performance uplift is through the various optimisations that PR implemented.
  - bratao 4y ago
    The majority of the improvements come from the Faster-CPython (https://github.com/faster-cpython/ideas https://github.com/faster-cpython/ideas) project, sponsored by Microsoft (Thanks!!)
- samwillis 4y ago
  I believe some of these improvements were part of the same work to eventually remove the GIL (global interpreter lock), albeit not tied to it. However, the final decision on removing the GIL hasn't been made.
  - gshulegaard 4y ago
    Ah yes, this is what I thought was the current state. I was curious if a GIL removal decision had been made that I had missed, but seems like it's still a soft "maybe": https://lukasz.langa.pl/5d044f91-49c1-4170-aed1-62b6763e6ad0/ https://lukasz.langa.pl/5d044f91-49c1-4170-aed1-62b6763e6ad0... Edit: And to be fair, you can do "true" concurrency in Python you just have to eject to Multi-processing (with the tradeoffs that implies), so I didn't want to assume the comment was about GIL removal.
- axblount 4y ago
  Sam Gross has been working on a branch that removes the GIL and has a few other performance improvements. More details are here on the python mailing list: https://mail.python.org/archives/list/python-dev@python.org/thread/ABR2L6BENNA6UPSPKV474HCS4LWT26GY/ https://mail.python.org/archives/list/python-dev@python.org/... There's a link to his design overview document in the thread.
onphonenow 4y ago
A lot of credit for these improvements come from that gil removal effort. I very very much hope we get it or a well supported branch with those changes.
dekhn 4y ago
I think that's actually "actual parallelism", not concurrency. Concurrency is multiple processes making forward progress using a single processor and some form of blocking/switching, while parallelism takes advantage of multiple cores. Personally, I consider concurrency a degenerate form of parallelism limited to multiplexing processes on a single core.
- pertymcpert 4y ago
  Concurrency is really a higher level of abstraction than parallelism. It's more of a program structuring concept.

gshulegaard 4y ago

Can you elaborate on what you mean by "actual concurrency in python soon"?

AlphaSite 4y ago
There was an experimental branch where someone reimplemented Python 3.9 sans the Gil and managed to maintain performance parity with the Gil’ed version. It’s likely this performance uplift is through the various optimisations that PR implemented.
- bratao 4y ago
  The majority of the improvements come from the Faster-CPython (https://github.com/faster-cpython/ideas https://github.com/faster-cpython/ideas) project, sponsored by Microsoft (Thanks!!)
samwillis 4y ago
I believe some of these improvements were part of the same work to eventually remove the GIL (global interpreter lock), albeit not tied to it. However, the final decision on removing the GIL hasn't been made.
- gshulegaard 4y ago
  Ah yes, this is what I thought was the current state. I was curious if a GIL removal decision had been made that I had missed, but seems like it's still a soft "maybe": https://lukasz.langa.pl/5d044f91-49c1-4170-aed1-62b6763e6ad0/ https://lukasz.langa.pl/5d044f91-49c1-4170-aed1-62b6763e6ad0... Edit: And to be fair, you can do "true" concurrency in Python you just have to eject to Multi-processing (with the tradeoffs that implies), so I didn't want to assume the comment was about GIL removal.
axblount 4y ago
Sam Gross has been working on a branch that removes the GIL and has a few other performance improvements. More details are here on the python mailing list: https://mail.python.org/archives/list/python-dev@python.org/thread/ABR2L6BENNA6UPSPKV474HCS4LWT26GY/ https://mail.python.org/archives/list/python-dev@python.org/... There's a link to his design overview document in the thread.

onphonenow 4y ago

A lot of credit for these improvements come from that gil removal effort. I very very much hope we get it or a well supported branch with those changes.

dekhn 4y ago

I think that's actually "actual parallelism", not concurrency. Concurrency is multiple processes making forward progress using a single processor and some form of blocking/switching, while parallelism takes advantage of multiple cores. Personally, I consider concurrency a degenerate form of parallelism limited to multiplexing processes on a single core.

pertymcpert 4y ago
Concurrency is really a higher level of abstraction than parallelism. It's more of a program structuring concept.