4 ms·

What are advantages of non-GIL Python?

KMnO4 3y ago
GIL is a promise from the internal implementation (eg CPython) of Python that things will happen atomically within the Python interpreter. This means that when multiple threads try to access and modify Python objects at the same time, the GIL ensures that only one thread can execute Python bytecode at any given moment, preventing potential conflicts and ensuring data integrity. However, this comes at the cost of limiting the full utilization of multiple CPU cores for certain CPU-bound tasks. Non-GIL adds some complexity to the implementation and some risk when writing multithreaded code at the benefit of improving performance.
- fbdab103 3y ago
  At the current implementation, GILless is a performance regression[0], of 5-7% [0] https://peps.python.org/pep-0703/#performance https://peps.python.org/pep-0703/#performance
- slt2021 3y ago
  But people are not doing cpu bound tasks in native python (performance is a joke), it all comes down to calling a C library that is optimized for compute - what will be the change that GIL brings here?
  - AlphaSite 3y ago
    There is a JIT being built for Python, so performance is being attacked from 2 avenues, single threaded and multithreaded.
lostdog 3y ago
In GIL Python, you might think you could speed something up by multithreading, but it turns out you can't. The GIL will just run it serially anyways. No-GIL means it is possible to run things in parallel (without resorting to fancy C extensions).
- miraculixx 3y ago
  In theory it's easy. In practice you need to design programs very carefully to leverage multiple threads. The same is true for multiple processes, yet that is much easier to reason about and a lot more likely to get it right.
  - uneekname 3y ago
    > and a lot more likely to get it right I get that it's a hard problem, but it isn't a guessing game. Concurrency has always required one to carefully design their program. We will see how Python implements these new APIs, but I trust it will be approachable to those who want to do it.
    - miraculixx 3y ago
      It's not about the API. The multithreading API (in any language) is easy: 1. Start a thread 2. Give it work to do (data + code) The problems start when multiple threads work on the same data and compete for the same system resources. Agree, it's not a guessing game. But it is a huge state machine with unobvious hidden states. The cure are locks everywhere, and the magic is to find the least number and places of locks to still make it fast and correct, in any and all situations aka race conditions. It's a much harder problem to reason about than most people realize.

GIL is a promise from the internal implementation (eg CPython) of Python that things will happen atomically within the Python interpreter. This means that when multiple threads try to access and modify Python objects at the same time, the GIL ensures that only one thread can execute Python bytecode at any given moment, preventing potential conflicts and ensuring data integrity. However, this comes at the cost of limiting the full utilization of multiple CPU cores for certain CPU-bound tasks. Non-GIL adds some complexity to the implementation and some risk when writing multithreaded code at the benefit of improving performance.

fbdab103 3y ago
At the current implementation, GILless is a performance regression[0], of 5-7% [0] https://peps.python.org/pep-0703/#performance https://peps.python.org/pep-0703/#performance
slt2021 3y ago
But people are not doing cpu bound tasks in native python (performance is a joke), it all comes down to calling a C library that is optimized for compute - what will be the change that GIL brings here?
- AlphaSite 3y ago
  There is a JIT being built for Python, so performance is being attacked from 2 avenues, single threaded and multithreaded.

lostdog 3y ago

In GIL Python, you might think you could speed something up by multithreading, but it turns out you can't. The GIL will just run it serially anyways. No-GIL means it is possible to run things in parallel (without resorting to fancy C extensions).

miraculixx 3y ago
In theory it's easy. In practice you need to design programs very carefully to leverage multiple threads. The same is true for multiple processes, yet that is much easier to reason about and a lot more likely to get it right.
- uneekname 3y ago
  > and a lot more likely to get it right I get that it's a hard problem, but it isn't a guessing game. Concurrency has always required one to carefully design their program. We will see how Python implements these new APIs, but I trust it will be approachable to those who want to do it.
  - miraculixx 3y ago
    It's not about the API. The multithreading API (in any language) is easy: 1. Start a thread 2. Give it work to do (data + code) The problems start when multiple threads work on the same data and compete for the same system resources. Agree, it's not a guessing game. But it is a huge state machine with unobvious hidden states. The cure are locks everywhere, and the magic is to find the least number and places of locks to still make it fast and correct, in any and all situations aka race conditions. It's a much harder problem to reason about than most people realize.