4 ms·

Thanks for the clarification. It's interesting that we see the same definition so differently. The Python example has two concurrent accesses of the same memor

Thanks for the clarification. It's interesting that we see the same definition so differently. The Python example has two concurrent accesses of the same memory (the global counter) where one of them is a write. That seems to meet the definition. We're also seeing the telltale symptoms of a data race. Note that we don't need to see slicing to qualify as a data race; as plenty cache-line-aligned data races don't exhibit slicing (not implying you were claiming this, just mentioning it for anyone curious). The Rustonomicon definition also adds "they are unsynchronized", perhaps that's the part we disagree on. I see this code as unsynchronized; there is no mutex or atomic increment happening around this operation. One could reasonably disagree, depending on what level they're looking at: statement-level, instruction-level, opcode-level, assembly level, uop level, or a higher "logical" level. Either way, the point of the article is to help debunk a common misperception that the GIL helps users with their concurrency in some way.

Kranar 5y ago
The GIL helps users protect against data races, it does not protect against race conditions. In Python, due to the GIL, it is not possible to concurrently access a memory location and hence there is no concurrent access to x. There are concurrent threads of execution, but they are interrupted either before accessing x, or after accessing x. At no point is a thread of execution interrupted right in the middle of an access to x.
- verdagon 5y ago
  These things are running concurrently with each other though. Concurrent operations can happen one after the other, and Python's threads are more about concurrency than parallelism. If these definitions instead mentioned "simultaneous" or "in parallel", I could agree with your belief. (Edited in response to your edit)
  - Kranar 5y ago
    There need not be any parallelism or simultaneity for a data race to occur. What matters is that writes occur concurrently with another read/write, not whether they occur in parallel. For example it's possible on a single core CPU to have one thread operate on the higher bits of a 32-bit memory location then yield to second thread that operates on the lower 32-bits of the same memory location so that the end result is a clobbered write. At no point do the two threads run simultaneously and yet this would still be an example of a data race. Python's GIL makes such an access impossible on any platform it runs on.
mirashii 5y ago
> The Python example has two concurrent accesses of the same memory (the global counter) where one of them is a write. That seems to meet the definition. Ah, but no, there is never concurrent access to that memory address, that's what the GIL prevents. The loads/stores to that memory address happen only while the GIL is held. The "concurrency" being used here is much more narrow than the dictionary definition of concurrency and the way it is usually discussed in the Python world.
- verdagon 5y ago
  Two different kinds of concurrency, a twist! If you have a link to enlighten, I'd love to read more about this.
  - Kranar 5y ago
    The precise definition of a concurrent access is with respect to a programming language's memory model. Now Python does not provide formal semantics for its memory model (neither does Rust for that matter or C++ prior to C++11), instead Python is defined by its reference implementation where individual reads and writes are protected by the GIL. So while threads in Python run concurrently as in individual operations interleave with one another, there is never a concurrent read or write operation. Every individual operation performed by the Python bytecode interpreter is protected by the GIL, ie. atomic. That said, reads and writes can be atomic and still subject to race conditions which is what your example demonstrates. All your article shows is that data race freedom does not imply race condition freedom and this true not only of Python but of all programming languages.
    - verdagon 5y ago
      Two operations can be concurrent, even if they are run on the same thread, as long as they are unsequenced, in other words, there is nothing making one happen before the other. That's what the C++ standard says, linked from the wikipedia article you linked. I realize that C++ may not be completely applicable to Python, but this fits my prior understanding of the topic. Since there's no formal semantics to consult here, no actual alternate definition of concurrency, and the definition of data race fits, I can't say that the article is incorrect. This is going long and I need to get dinner, but thank you for the great discussion!
oefrha 5y ago
Just look at the bytecode. >>> import dis >>> def increment_counter(): ... counter = counter + 1 >>> dis.dis(increment_counter) 2 0 LOAD_FAST 0 (counter) 2 LOAD_CONST 1 (1) 4 BINARY_ADD 6 STORE_FAST 0 (counter) 8 LOAD_CONST 0 (None) 10 RETURN_VALUE A data race looks like LOAD_FAST from thread 1 begins STORE_FAST from thread 2 begins memory is partially overwritten LOAD_FAST from thread 1 finishes (got partially written garbage) STORE_FAST from thread 2 finishes This is not a data race: LOAD_FAST from thread 1 LOAD_FAST from thread 2 STORE_FAST from thread 1 STORE_FAST from thread 2 There's no LOAD_FAST + STORE_FAST "transaction" here, GIL has no such guarantee. Edit: You also need to realize that in Python, that BINARY_ADD can be anything: import time class SlowCounter(int): def __add__(self, other): time.sleep(1) return SlowCounter(super().__add__(other)) counter = SlowCounter() counter += 1 The GIL isn't going to wait around for the __add__.

Kranar 5y ago

The GIL helps users protect against data races, it does not protect against race conditions. In Python, due to the GIL, it is not possible to concurrently access a memory location and hence there is no concurrent access to x. There are concurrent threads of execution, but they are interrupted either before accessing x, or after accessing x. At no point is a thread of execution interrupted right in the middle of an access to x.

verdagon 5y ago
These things are running concurrently with each other though. Concurrent operations can happen one after the other, and Python's threads are more about concurrency than parallelism. If these definitions instead mentioned "simultaneous" or "in parallel", I could agree with your belief. (Edited in response to your edit)
- Kranar 5y ago
  There need not be any parallelism or simultaneity for a data race to occur. What matters is that writes occur concurrently with another read/write, not whether they occur in parallel. For example it's possible on a single core CPU to have one thread operate on the higher bits of a 32-bit memory location then yield to second thread that operates on the lower 32-bits of the same memory location so that the end result is a clobbered write. At no point do the two threads run simultaneously and yet this would still be an example of a data race. Python's GIL makes such an access impossible on any platform it runs on.

mirashii 5y ago

> The Python example has two concurrent accesses of the same memory (the global counter) where one of them is a write. That seems to meet the definition. Ah, but no, there is never concurrent access to that memory address, that's what the GIL prevents. The loads/stores to that memory address happen only while the GIL is held. The "concurrency" being used here is much more narrow than the dictionary definition of concurrency and the way it is usually discussed in the Python world.

verdagon 5y ago
Two different kinds of concurrency, a twist! If you have a link to enlighten, I'd love to read more about this.
- Kranar 5y ago
  The precise definition of a concurrent access is with respect to a programming language's memory model. Now Python does not provide formal semantics for its memory model (neither does Rust for that matter or C++ prior to C++11), instead Python is defined by its reference implementation where individual reads and writes are protected by the GIL. So while threads in Python run concurrently as in individual operations interleave with one another, there is never a concurrent read or write operation. Every individual operation performed by the Python bytecode interpreter is protected by the GIL, ie. atomic. That said, reads and writes can be atomic and still subject to race conditions which is what your example demonstrates. All your article shows is that data race freedom does not imply race condition freedom and this true not only of Python but of all programming languages.
  - verdagon 5y ago
    Two operations can be concurrent, even if they are run on the same thread, as long as they are unsequenced, in other words, there is nothing making one happen before the other. That's what the C++ standard says, linked from the wikipedia article you linked. I realize that C++ may not be completely applicable to Python, but this fits my prior understanding of the topic. Since there's no formal semantics to consult here, no actual alternate definition of concurrency, and the definition of data race fits, I can't say that the article is incorrect. This is going long and I need to get dinner, but thank you for the great discussion!

oefrha 5y ago

Just look at the bytecode. >>> import dis >>> def increment_counter(): ... counter = counter + 1 >>> dis.dis(increment_counter) 2 0 LOAD_FAST 0 (counter) 2 LOAD_CONST 1 (1) 4 BINARY_ADD 6 STORE_FAST 0 (counter) 8 LOAD_CONST 0 (None) 10 RETURN_VALUE A data race looks like LOAD_FAST from thread 1 begins STORE_FAST from thread 2 begins memory is partially overwritten LOAD_FAST from thread 1 finishes (got partially written garbage) STORE_FAST from thread 2 finishes This is not a data race: LOAD_FAST from thread 1 LOAD_FAST from thread 2 STORE_FAST from thread 1 STORE_FAST from thread 2 There's no LOAD_FAST + STORE_FAST "transaction" here, GIL has no such guarantee. Edit: You also need to realize that in Python, that BINARY_ADD can be anything: import time class SlowCounter(int): def __add__(self, other): time.sleep(1) return SlowCounter(super().__add__(other)) counter = SlowCounter() counter += 1 The GIL isn't going to wait around for the __add__.