4 ms·

Can someone explain this part: > What happens if multiple threads try to access / edit the same object at the same time? Imagine one thread is trying to add to

Can someone explain this part: > What happens if multiple threads try to access / edit the same object at the same time? Imagine one thread is trying to add to a dict while another is trying to read from it. There are two options here Why not just ignore this fact, like C and C++? Worst case this is a datarace, best case the programmer either puts the lock or writes a thread safe dict themselves? What am I missing here?

tliltocatl 2y ago
Memory safety, heap integrity and GC correctness i guess. If you ignore facts like C your language will be as safe as C, except it's worse because at least C doesn't silently rearrange your heap in background.
Numerlor 2y ago
There is an expectation of not dealing with data races from Python code. Apart from being a language where people expect these things to not be an issue it is also the behaviour with thw Gil in place and would be breaking
deleted 2y ago
[deleted]
0xFEE1DEAD 2y ago
Let me preface this by saying I have no source to prove what I’m about to say, but Guido van Rossum aimed to create a programming language that feels more like a natural language without being just a toy language. He envisioned a real programming language that could be used by non-programmers, and for this to work it couldn’t contain the usual footguns. One could argue that he succeeded, considering how many members of the scientific community, who don’t primarily see themselves as programmers, use Python.
- William_BB 2y ago
  All the answers were good, but I think this explained it the best. Thank you
- dawnofdusk 2y ago
  But the performance motivations for removing the GIL seem at odds with this. I feel like the subset of Python users who care about the GIL and the subset who are "non-programmers" are entirely disjoint.
- charlieyu1 2y ago
  I guess I am one of these people and have zero idea on how to write code in parallel programming. Would GIL removal benefit me, given that it seems like it would hurt performance?
LudwigNagasena 2y ago
Why not just use C and C++ at that point? People use Python because they don't want to manage data races or re-implement a hash table.
- William_BB 2y ago
  This was not a gotcha, sorry if it came across that way. It was a genuine question
fulafel 2y ago
We don't want Python programs to become the same kind of security swiss cheese as C/C++ programs.
oconnor663 2y ago
It's harder to ignore the problem in Python, because reference counting turns every read into a write, incrementing and then decrementing the recount of the object you read. For example, calling "my_mutex.lock()" has already messed with the recount on my_mutex before any locking happens. If races between threads could corrupt those refcounts, there's no way you could code around that. Right now the GIL protects those refcounts, so without a GIL you need a big change to make them all atomic.
- William_BB 2y ago
  Oh wow, I haven't thought about refcounts on mutexes
masklinn 2y ago
> Worst case this is a datarace [... w]hat am I missing here? That the worst case being memory unsafety and a compromised VM is not acceptable? Especially for a language as open to low-skill developers as Python?
- William_BB 2y ago
  Hm I get the memory safety part, but could you elaborate on the compromised VM part? I'm not sure I understand that.. Specifically I don't understand what you mean by VM here
  - masklinn 2y ago
    Once you compromise memory safety you have access to the entire Python runtime, there is no separation.
micheles 2y ago
I am scientific Python programmer. 99% to 100% of my programs require parallelism, but it is ALWAYS embarrassingly trivial parallelism, nothing is ever mutated and I never need locks. Right now I am forced to use multiprocessing to get the performance, with all problems of multiprocessing, the major one being that I need to use more memory. For me using multithreading could mean the difference between running out of memory and not running out memory. The GIL removal matters for people like me, the proponents of the GIL removal comes from the scientific community.

tliltocatl 2y ago

Memory safety, heap integrity and GC correctness i guess. If you ignore facts like C your language will be as safe as C, except it's worse because at least C doesn't silently rearrange your heap in background.

Numerlor 2y ago

There is an expectation of not dealing with data races from Python code. Apart from being a language where people expect these things to not be an issue it is also the behaviour with thw Gil in place and would be breaking

deleted 2y ago

[deleted]

0xFEE1DEAD 2y ago

Let me preface this by saying I have no source to prove what I’m about to say, but Guido van Rossum aimed to create a programming language that feels more like a natural language without being just a toy language. He envisioned a real programming language that could be used by non-programmers, and for this to work it couldn’t contain the usual footguns. One could argue that he succeeded, considering how many members of the scientific community, who don’t primarily see themselves as programmers, use Python.

William_BB 2y ago
All the answers were good, but I think this explained it the best. Thank you
dawnofdusk 2y ago
But the performance motivations for removing the GIL seem at odds with this. I feel like the subset of Python users who care about the GIL and the subset who are "non-programmers" are entirely disjoint.
charlieyu1 2y ago
I guess I am one of these people and have zero idea on how to write code in parallel programming. Would GIL removal benefit me, given that it seems like it would hurt performance?

LudwigNagasena 2y ago

Why not just use C and C++ at that point? People use Python because they don't want to manage data races or re-implement a hash table.

William_BB 2y ago
This was not a gotcha, sorry if it came across that way. It was a genuine question

fulafel 2y ago

We don't want Python programs to become the same kind of security swiss cheese as C/C++ programs.

oconnor663 2y ago

It's harder to ignore the problem in Python, because reference counting turns every read into a write, incrementing and then decrementing the recount of the object you read. For example, calling "my_mutex.lock()" has already messed with the recount on my_mutex before any locking happens. If races between threads could corrupt those refcounts, there's no way you could code around that. Right now the GIL protects those refcounts, so without a GIL you need a big change to make them all atomic.

William_BB 2y ago
Oh wow, I haven't thought about refcounts on mutexes

masklinn 2y ago

> Worst case this is a datarace [... w]hat am I missing here? That the worst case being memory unsafety and a compromised VM is not acceptable? Especially for a language as open to low-skill developers as Python?

William_BB 2y ago
Hm I get the memory safety part, but could you elaborate on the compromised VM part? I'm not sure I understand that.. Specifically I don't understand what you mean by VM here
- masklinn 2y ago
  Once you compromise memory safety you have access to the entire Python runtime, there is no separation.

micheles 2y ago

I am scientific Python programmer. 99% to 100% of my programs require parallelism, but it is ALWAYS embarrassingly trivial parallelism, nothing is ever mutated and I never need locks. Right now I am forced to use multiprocessing to get the performance, with all problems of multiprocessing, the major one being that I need to use more memory. For me using multithreading could mean the difference between running out of memory and not running out memory. The GIL removal matters for people like me, the proponents of the GIL removal comes from the scientific community.