4 ms·

I think Java got this mostly right. On the threading front, very little is thread-safe or atomic (x += 1 is not thread-safe), so as soon as you expose something

I think Java got this mostly right. On the threading front, very little is thread-safe or atomic (x += 1 is not thread-safe), so as soon as you expose something to threads, you have to think about safe access. For interacting with C code, your choices are either shared buffers or copying data between C and Java. It's painful, but it's needed for memory safety.

colonCapitalDee 1y ago
Is compound assignment atomic in any major language?
- Groxx 1y ago
  it has been in Python due to the GIL.
  - i80and 1y ago
    It's not atomic even with the GIL, though: another thread can run in between the bytecode's load and increment, right? The GIL's guarantees didn't extend to this.
    - deleted 1y ago
      [deleted]
    - Groxx 1y ago
      There is NB_INPLACE_ADD... but I'm struggling to find enough details to be truly confident :\ possibly its existence is misleading other people (thus me) to think += is a single operation in bytecode. Or, on further reading, maybe it applies to anything that implements `_iadd_` in C. Which does not appear to include native longs: https://github.com/python/cpython/blob/main/Objects/longobject.c#L6621 https://github.com/python/cpython/blob/main/Objects/longobje...
- arccy 1y ago
  no... but some languages may disallow simultaneously holding a reference in different execution threads
- delusional 1y ago
  Python and Javascript (in the browser) due to their single threaded nature. C++ too as long as you have a std::atomic on the left hand side (since they overload the operator).
rogerbinns 1y ago
The core Python data structures are atomic to Python developers. eg there is no way you can corrupt a list or dictionary no matter how much concurrency you try to use. This was traditionally done under the protection of the global interpreter lock which ensured that only one piece of C code at a time was operating with the internals of those objects. C code can also release the GIL eg during I/O, or operations in other libraries that aren't interacting with Python objects, allowing concurrency. The free threaded implementation adds what amounts to individual object locks at the C level (critical sections). This still means developers writing Python code can do whatever they want, and they will not experience corruption or crashes. The base objects have all been updated. Python is popular because of many extensions written in C, including many in the standard library. Every single piece of that code must be updated to operate correctly in free threaded mode. That is a lot of work and is still in progress in the standard library. But in order to make the free threaded interpreter useful at this point, some have been marked as free thread safe, when that is not the case.
- hunterpayne 1y ago
  So its the worst of all possible worlds then. It has the poorest performance due to forced locking even when not necessary and if you load a library in another language (C), then you can still get corruptions. If you really care about performance, probably best to avoid Python entirely, even when its compiled like it is in CPython. PS For extra fun, learn what the LD_PRELOAD environmental variable does and how it can be used to abuse CPython (or other things that dynamically load shared objects).
  - AlphaSite 1y ago
    It’s another step in the right direction. These things take time.
    - ReflectedImage 1y ago
      Arguably, it's a step in the wrong direction. Share memory by communicating is already doable in Python with Pipe() and Queue() and side steps the issue entirely.
hunterpayne 1y ago
> x += 1 is not thread-safe Nit, that's true iff x is a primitive without the volatile modifier. That's not true for a volatile primitive.
- deleted 1y ago
  [deleted]
- wmanley 1y ago
  Even with volatile it’s a load and then a store no? It may not be undefined behaviour, but I don’t think it will be atomic.
  - diek 1y ago
    You're correct. If you have: public int someField; public void inc() { someField += 1; } that still compiles down to: GETFIELD [someField] ICONST_1 IADD PUTFIELD [somefield] whether 'someField' is volatile or not. The volatile just affects the load/store semantics of the GETFIELD/PUTFIELD ops. For atomic increment you have to go through something like AtomicInteger that will internally use an Unsafe instance to ensure it emits a platform-specific atomic increment instruction.

colonCapitalDee 1y ago

Is compound assignment atomic in any major language?

Groxx 1y ago
it has been in Python due to the GIL.
- i80and 1y ago
  It's not atomic even with the GIL, though: another thread can run in between the bytecode's load and increment, right? The GIL's guarantees didn't extend to this.
  - deleted 1y ago
    [deleted]
  - Groxx 1y ago
    There is NB_INPLACE_ADD... but I'm struggling to find enough details to be truly confident :\ possibly its existence is misleading other people (thus me) to think += is a single operation in bytecode. Or, on further reading, maybe it applies to anything that implements `_iadd_` in C. Which does not appear to include native longs: https://github.com/python/cpython/blob/main/Objects/longobject.c#L6621 https://github.com/python/cpython/blob/main/Objects/longobje...
arccy 1y ago
no... but some languages may disallow simultaneously holding a reference in different execution threads
delusional 1y ago
Python and Javascript (in the browser) due to their single threaded nature. C++ too as long as you have a std::atomic on the left hand side (since they overload the operator).

rogerbinns 1y ago

The core Python data structures are atomic to Python developers. eg there is no way you can corrupt a list or dictionary no matter how much concurrency you try to use. This was traditionally done under the protection of the global interpreter lock which ensured that only one piece of C code at a time was operating with the internals of those objects. C code can also release the GIL eg during I/O, or operations in other libraries that aren't interacting with Python objects, allowing concurrency. The free threaded implementation adds what amounts to individual object locks at the C level (critical sections). This still means developers writing Python code can do whatever they want, and they will not experience corruption or crashes. The base objects have all been updated. Python is popular because of many extensions written in C, including many in the standard library. Every single piece of that code must be updated to operate correctly in free threaded mode. That is a lot of work and is still in progress in the standard library. But in order to make the free threaded interpreter useful at this point, some have been marked as free thread safe, when that is not the case.

hunterpayne 1y ago
So its the worst of all possible worlds then. It has the poorest performance due to forced locking even when not necessary and if you load a library in another language (C), then you can still get corruptions. If you really care about performance, probably best to avoid Python entirely, even when its compiled like it is in CPython. PS For extra fun, learn what the LD_PRELOAD environmental variable does and how it can be used to abuse CPython (or other things that dynamically load shared objects).
- AlphaSite 1y ago
  It’s another step in the right direction. These things take time.
  - ReflectedImage 1y ago
    Arguably, it's a step in the wrong direction. Share memory by communicating is already doable in Python with Pipe() and Queue() and side steps the issue entirely.

hunterpayne 1y ago

> x += 1 is not thread-safe Nit, that's true iff x is a primitive without the volatile modifier. That's not true for a volatile primitive.

deleted 1y ago
[deleted]
wmanley 1y ago
Even with volatile it’s a load and then a store no? It may not be undefined behaviour, but I don’t think it will be atomic.
- diek 1y ago
  You're correct. If you have: public int someField; public void inc() { someField += 1; } that still compiles down to: GETFIELD [someField] ICONST_1 IADD PUTFIELD [somefield] whether 'someField' is volatile or not. The volatile just affects the load/store semantics of the GETFIELD/PUTFIELD ops. For atomic increment you have to go through something like AtomicInteger that will internally use an Unsafe instance to ensure it emits a platform-specific atomic increment instruction.