6 ms·

how about getting rid of GIL?

gautamdivgi 5y ago
I agree. At least an annotation based method of disabling the GIL. If I'm running multi-processing code and its a thread per process then I really don't need the GIL - I should be able to disable it.
fastball 5y ago
If you get rid of the GIL you have to replace one big lock with a bunch of very small locks, which actually hurts performance for programs which only need 1 core, which is most programs.
- throwaway894345 5y ago
  > If you get rid of the GIL you have to replace one big lock with a bunch of very small locks, which actually hurts performance for programs which only need 1 core, which is most programs. Python could do what most other languages do and allow programmers to explicitly add locks to the code that needs to be locked, rather than locking everything, all the time.
  - qbasic_forever 5y ago
    Python has a robust and very capable set of locks and multiprocessing primitives. The issue is that the GIL has been giving you atomic/serialized access to a lot of primitives for free since the inception of the language, so no one has really felt the need to do things like lock access to their basic dicts or lists. You can't just take that code and remove the GIL locking without causing chaos. And as we saw with the decade plus transition from Python 2->3 you can't just tell people to go fix all their old code.
    - throwaway894345 5y ago
      Most code doesn't actually run in a shared memory parallel context precisely because of the GIL, so I think the "chaos" would be minimal. It will still be painful, and would require the ecosystem to adapt (and therefore will never happen because the Python ecosystem lacks the necessary leadership), but it's technically feasible and it would be politically feasible for other language communities.
- prabhu-yu 5y ago
  Ok, can we do this? Python interpreter can understand if it is running on 1 core processor (assume 1 core = 1 HW thread) or multi core processor. Then use GIL taht suits. ie, if the code uses multiple SW threads, use GIL that suits needs... something like possible? Any reason why experts are not doing this? -p
  - overgard 5y ago
    It's been done; it just was never accepted into the core. And it never will be.
deleted 5y ago
[deleted]
dekhn 5y ago
it's never going to happen in mainline cpython.
qbasic_forever 5y ago
a lot of python code has subtle race conditions where they assume some operations like dictionary access, manipulation, etc. are atomic and the GIL makes it all fine. remove the GIL and there can be a huge explosion of subtle and difficult to find bugs in even simple code. people have been working on trying to remove the GIL for a decade or more now and I don't expect any magic fix soon.

I agree. At least an annotation based method of disabling the GIL. If I'm running multi-processing code and its a thread per process then I really don't need the GIL - I should be able to disable it.

fastball 5y ago

If you get rid of the GIL you have to replace one big lock with a bunch of very small locks, which actually hurts performance for programs which only need 1 core, which is most programs.

throwaway894345 5y ago
> If you get rid of the GIL you have to replace one big lock with a bunch of very small locks, which actually hurts performance for programs which only need 1 core, which is most programs. Python could do what most other languages do and allow programmers to explicitly add locks to the code that needs to be locked, rather than locking everything, all the time.
- qbasic_forever 5y ago
  Python has a robust and very capable set of locks and multiprocessing primitives. The issue is that the GIL has been giving you atomic/serialized access to a lot of primitives for free since the inception of the language, so no one has really felt the need to do things like lock access to their basic dicts or lists. You can't just take that code and remove the GIL locking without causing chaos. And as we saw with the decade plus transition from Python 2->3 you can't just tell people to go fix all their old code.
  - throwaway894345 5y ago
    Most code doesn't actually run in a shared memory parallel context precisely because of the GIL, so I think the "chaos" would be minimal. It will still be painful, and would require the ecosystem to adapt (and therefore will never happen because the Python ecosystem lacks the necessary leadership), but it's technically feasible and it would be politically feasible for other language communities.
prabhu-yu 5y ago
Ok, can we do this? Python interpreter can understand if it is running on 1 core processor (assume 1 core = 1 HW thread) or multi core processor. Then use GIL taht suits. ie, if the code uses multiple SW threads, use GIL that suits needs... something like possible? Any reason why experts are not doing this? -p
- overgard 5y ago
  It's been done; it just was never accepted into the core. And it never will be.

deleted 5y ago

[deleted]

dekhn 5y ago

it's never going to happen in mainline cpython.

qbasic_forever 5y ago

a lot of python code has subtle race conditions where they assume some operations like dictionary access, manipulation, etc. are atomic and the GIL makes it all fine. remove the GIL and there can be a huge explosion of subtle and difficult to find bugs in even simple code. people have been working on trying to remove the GIL for a decade or more now and I don't expect any magic fix soon.