4 ms·

> Your goals are either memory isolation for better distribution/concurrency/clustering/fault tolerance/garbage collection or shared memory for ability to work

by zzzcpan 10y ago

> Your goals are either memory isolation for better distribution/concurrency/clustering/fault tolerance/garbage collection or shared memory for ability to work with large datasets more efficiently. The isolation is not physical, but logical. Implementations are free to use zero-copying and make everything just as efficient. Theoretically compiler could even optimize message passing overhead away for systems with shared memory in some cases. The opposite is also true, shared memory is also logical and there is a lot of room for a lot of clever things, like eliminating races.

brightball 10y ago
How can you eliminate races with shared memory in a concurrent environment?
- dbaupp 10y ago
  Rust's "sharing xor mutability" guarantee eliminates data races by (usually statically but also dynamically) ensuring that a memory location won't be mutated when multiple places/threads can access it. Of course, the machinery required for this in Rust is relatively complicated (e.g. as a side-effect, it allows one to also forgo a GC without losing safety).
  - brightball 10y ago
    How is that different from a mutex lock?
    - sinhpham 10y ago
      It's being done at compile time.
    - dbaupp 10y ago
      Rust lets you be precise about placing the locks exactly where they are needed, with compile time errors if a lock isn't locked when accessing data, or a lock is missing. Fixing data races with locks for arbitrary code, without more static guarantees in the language, essentially means locks have to be held for most memory accesses: it's hard/impossible to tell if that location will be accessed concurrently. (This is why CPython has a GIL: it's easier to only let one piece of code execute at a time than to handle data races in arbitrary code.) https://blog.rust-lang.org/2015/04/10/Fearless-Concurrency.html https://blog.rust-lang.org/2015/04/10/Fearless-Concurrency.h... is interesting reading.
    - kazagistar 10y ago
      For example, in Rust, it will stop you from accessing the value protected by a Mutex after the lock is freed, and do so at compile time, even through function calls, pointers hidden in data structures, etc. Another example: I can take a value and mutate it locally, then pass it immutably to a number of threads which use it to compute some results, and once they are all complete, I can mutate the data structure again. If I try to write mutation code before all the threads are re-joined, the compiler will tell me that I can't mutate while there are outstanding immutable borrows. At compile time, at no runtime overhead or locking.
- ergl 10y ago
  Look into Pony's reference capabilities. To put it shortly, the compiler will guarantee that there are no data races through this system, and therefore the runtime is able to use shared memory and pointers for efficiency.

by zzzcpan 10y ago

brightball 10y ago

How can you eliminate races with shared memory in a concurrent environment?

dbaupp 10y ago
Rust's "sharing xor mutability" guarantee eliminates data races by (usually statically but also dynamically) ensuring that a memory location won't be mutated when multiple places/threads can access it. Of course, the machinery required for this in Rust is relatively complicated (e.g. as a side-effect, it allows one to also forgo a GC without losing safety).
- brightball 10y ago
  How is that different from a mutex lock?
  - sinhpham 10y ago
    It's being done at compile time.
  - dbaupp 10y ago
    Rust lets you be precise about placing the locks exactly where they are needed, with compile time errors if a lock isn't locked when accessing data, or a lock is missing. Fixing data races with locks for arbitrary code, without more static guarantees in the language, essentially means locks have to be held for most memory accesses: it's hard/impossible to tell if that location will be accessed concurrently. (This is why CPython has a GIL: it's easier to only let one piece of code execute at a time than to handle data races in arbitrary code.) https://blog.rust-lang.org/2015/04/10/Fearless-Concurrency.html https://blog.rust-lang.org/2015/04/10/Fearless-Concurrency.h... is interesting reading.
  - kazagistar 10y ago
    For example, in Rust, it will stop you from accessing the value protected by a Mutex after the lock is freed, and do so at compile time, even through function calls, pointers hidden in data structures, etc. Another example: I can take a value and mutate it locally, then pass it immutably to a number of threads which use it to compute some results, and once they are all complete, I can mutate the data structure again. If I try to write mutation code before all the threads are re-joined, the compiler will tell me that I can't mutate while there are outstanding immutable borrows. At compile time, at no runtime overhead or locking.
ergl 10y ago
Look into Pony's reference capabilities. To put it shortly, the compiler will guarantee that there are no data races through this system, and therefore the runtime is able to use shared memory and pointers for efficiency.