3 ms·

>Gil free python would break a lot of oss packages written with the C API that are only thread safe because if the GIL. So make it a runtime option. It's gett

by lzauz 4y ago

>Gil free python would break a lot of oss packages written with the C API that are only thread safe because if the GIL. So make it a runtime option. It's getting tiresome that python performance is suffering because some can't be bothered to write thread-safe software. In 2022.

coldtea 4y ago
In C which Python extensions use, it hasen't gotten any easier to write thread-safe software "in 2022"
- ahartmetz 4y ago
  But it has become more necessary and useful, so even C programmers better get used to it (and some tooling to help catch most mistakes).
  - coldtea 4y ago
    >so even C programmers better get used to it Or else? It's not like they're not trying their best - or don't spend the level of effort that they and their companies are willing to take...
    - ahartmetz 4y ago
      I guess it would have made more sense for me to say "Python C extension developers". I don't find multithreading in languages without particular support easy at all, but I have become better at it. It is possible and sometimes necessary. It seems like the prevailing attitude in the Python ecosystem is weird, a kind of sour grapes thing, i.e. "Python doesn't have good multithreading support, but multithreading is ugly and error-prone anyway and the alternatives are almost as good or better".
AdamN 4y ago
Usually when I've needed more parallelization I've allowed more processes and for slow methods, there is threading available (this doesn't overcome the GIL but allows those methods to independently operate). It seems like the biggest reasons to focus on removing the GIL are single-process applications or machines where memory is constrained (so you don't want tons of processes consuming it all). Are you in one of those situations or is there another scenario that is impacted by the GIL?
kortex 4y ago
This comment demonstrates a tremendous amount of naivete about the Cpython runtime. After PyObject itself, the GIL mutex is probably the next most important data structure in the entire codebase. It's not "someone not being bothered to write thread-safe software." It's not something you can hide behind a flag. It's central to the entire cpython data model and any library which relies on releasing the GIL. The closest anyone has come to removing the GIL is the Gilectomy project by Larry Hastings, and it's unlikely to ever be upstreamed unless it could be somehow made to work with libraries that rely on assumptions about GIL mechanics (eg numpy).
- mixmastamyk 4y ago
  > The closest anyone has come to removing the GIL is the Gilectomy project by Larry Hastings It was Sam Gross and he more or less achieved it: https://mail.python.org/archives/list/python-dev@python.org/thread/ABR2L6BENNA6UPSPKV474HCS4LWT26GY/ https://mail.python.org/archives/list/python-dev@python.org/...

by lzauz 4y ago

coldtea 4y ago

In C which Python extensions use, it hasen't gotten any easier to write thread-safe software "in 2022"

ahartmetz 4y ago
But it has become more necessary and useful, so even C programmers better get used to it (and some tooling to help catch most mistakes).
- coldtea 4y ago
  >so even C programmers better get used to it Or else? It's not like they're not trying their best - or don't spend the level of effort that they and their companies are willing to take...
  - ahartmetz 4y ago
    I guess it would have made more sense for me to say "Python C extension developers". I don't find multithreading in languages without particular support easy at all, but I have become better at it. It is possible and sometimes necessary. It seems like the prevailing attitude in the Python ecosystem is weird, a kind of sour grapes thing, i.e. "Python doesn't have good multithreading support, but multithreading is ugly and error-prone anyway and the alternatives are almost as good or better".

AdamN 4y ago

Usually when I've needed more parallelization I've allowed more processes and for slow methods, there is threading available (this doesn't overcome the GIL but allows those methods to independently operate). It seems like the biggest reasons to focus on removing the GIL are single-process applications or machines where memory is constrained (so you don't want tons of processes consuming it all). Are you in one of those situations or is there another scenario that is impacted by the GIL?

kortex 4y ago

This comment demonstrates a tremendous amount of naivete about the Cpython runtime. After PyObject itself, the GIL mutex is probably the next most important data structure in the entire codebase. It's not "someone not being bothered to write thread-safe software." It's not something you can hide behind a flag. It's central to the entire cpython data model and any library which relies on releasing the GIL. The closest anyone has come to removing the GIL is the Gilectomy project by Larry Hastings, and it's unlikely to ever be upstreamed unless it could be somehow made to work with libraries that rely on assumptions about GIL mechanics (eg numpy).

mixmastamyk 4y ago
> The closest anyone has come to removing the GIL is the Gilectomy project by Larry Hastings It was Sam Gross and he more or less achieved it: https://mail.python.org/archives/list/python-dev@python.org/thread/ABR2L6BENNA6UPSPKV474HCS4LWT26GY/ https://mail.python.org/archives/list/python-dev@python.org/...