5 ms·

I think it makes the most sense for Google, considering the ridiculous amount of C++ code. Even Spanner alone is larger than most company's codebases, and rewri

I think it makes the most sense for Google, considering the ridiculous amount of C++ code. Even Spanner alone is larger than most company's codebases, and rewriting it in Rust would take decades, especially because of the paradigm mismatch (C++ embraces shared mutability, Rust rejects it). It also makes sense because in Spanner, doing any minor change required many months of review, because it was such critical infrastructure. Refactoring was a non-starter in a lot of cases. So, a more gradual approach, just adding annotations that don't themselves affect the behavior of the program (just assist in static analysis) makes much more sense.

zozbot234 5y ago
Rust does not reject shared mutability. Rather, it is explicitly reified via the Cell<>, RefCell<>, etc. patterns. Even unsafe shared mutability ala idiomatic C++ is just an UnsafeCell<> away.
- verdagon 5y ago
  In theory yes, but when you look at the average Rust program, those mechanisms are avoided in favor of more idiomatic approaches.
  - hgomersall 5y ago
    Interior mutability is used all over the place. It's absolutely necessary. The thing is you are forced to be clear about the runtime safety mechanism you're using. In general, it gets abstracted away behind a safe API.
    - verdagon 5y ago
      Indeed it's present under the hood, we are operating on a CPU after all, which treats all of RAM as one giant shared-mutable blob of data. It will always be there, under some abstraction. The point I'm trying to communicate is that in practice, for various reasons, Rust programs do not use shared mutable access to objects to the same extent that C++ programs do. For example, C++ programs use observers, and dependency injection (the pattern, not the framework) to have member pointers to mutable subsystems, and we just don't often see that in Rust programs. This is the paradigm mismatch I'm highlighting: to rewrite a C++ program in Rust often requires a large paradigm shift. The pain is particularly felt when making them interoperate in a gradual refactoring. This is IMO one the bigger reasons that big rewrites to new languages fail, and why new languages benefit from being multi-paradigm, so that there's no paradigm mismatch to impede migrations.
      - westurner 5y ago
        IDK why the private, internal paradigms of e.g. Spanner would even need to be reimplemented in a clone instead of a port? Looks like CockroachDB (Go), TiDB (Go, Rust), and YugabyteDB (C++) are Distributed SQL alternatives to Spanner. ... Which may or may not be able to rely upon inexpensive Lamport clocks: https://en.wikipedia.org/wiki/Lamport_timestamp#Lamport's_logical_clock_in_distributed_systems https://en.wikipedia.org/wiki/Lamport_timestamp#Lamport's_lo...
        ceeplusplus 5y ago
        All those alternatives you mentioned are nowhere close to being on the level of Spanner in reliability or performance, particularly in a high contention scenario.
        westurner 5y ago
        According to which objective metrics?
        westurner 5y ago
        From https://westurner.github.io/hnlog/#comment-30603322 https://westurner.github.io/hnlog/#comment-30603322 re https://en.wikipedia-on-ipfs.org/wiki/Database_index https://en.wikipedia-on-ipfs.org/wiki/Database_index : https://github.com/wilsonzlin/edgesearch https://github.com/wilsonzlin/edgesearch : > * Serverless full-text search with Cloudflare Workers, WebAssembly, and Roaring Bitmaps * > "Edgesearch builds a reverse index by mapping terms to a compressed bit set (using Roaring Bitmaps) of IDs of documents containing the term, and creates a custom worker script and data to upload to Cloudflare Workers" WASM or [C++] to WASM? TIL about Roaring Bitmaps: /?q=roaring+bitmap https://medium.com/@amit.desai03/roaring-bitmaps-fast-data-structure-for-inverted-indexes-5490fa4d1b27 https://medium.com/@amit.desai03/roaring-bitmaps-fast-data-s...
- deleted 5y ago
  [deleted]
- celeritascelery 5y ago
  Interestingly unsafecell only allows half the shared mutability story. It allows immutable references to be mutated. But it does not allow mutable references to alias. So rust can’t (yet) express all the patterns in C++.
  - Rusky 5y ago
    That's backwards. If you can mutate through a shared reference, then there is no need to alias `&mut T`s in the first place! It is C++ that is missing the ability to express unique references like `&mut T`.
    - celeritascelery 5y ago
      > If you can mutate through a shared reference, then there is no need to alias `&mut T`s in the first place! Unfortunately that is not true. See [1] as an example is core where mutable aliasing is needed (async generators). In this case they just added to compiler hack [2][3], but this really needs proper support in rust. Something like “AliasCell”. [1] https://github.com/rust-lang/rust/issues/63818 https://github.com/rust-lang/rust/issues/63818 [2] https://github.com/rust-lang/rust/pull/82834 https://github.com/rust-lang/rust/pull/82834 [3] https://github.com/tokio-rs/tokio/pull/3654 https://github.com/tokio-rs/tokio/pull/3654
      - Rusky 5y ago
        I'm familiar with this issue. But it's a self-imposed problem, not a fundamental limitation relative to C++: the "same people" decided (at different points in time, to be fair) to make the `Future` trait use `&mut Self`, and to let specific futures hold self references. This is a very different issue than the actual ability to express shared mutability in the language. With enough foresight, Rust could have "simply" used `&Self` and `UnsafeCell` for futures (and the current compiler hack with `!Unpin` is basically just replacing `&mut Self` with `&UnsafeCell<Self>` under the hood).

zozbot234 5y ago

Rust does not reject shared mutability. Rather, it is explicitly reified via the Cell<>, RefCell<>, etc. patterns. Even unsafe shared mutability ala idiomatic C++ is just an UnsafeCell<> away.

verdagon 5y ago
In theory yes, but when you look at the average Rust program, those mechanisms are avoided in favor of more idiomatic approaches.
- hgomersall 5y ago
  Interior mutability is used all over the place. It's absolutely necessary. The thing is you are forced to be clear about the runtime safety mechanism you're using. In general, it gets abstracted away behind a safe API.
  - verdagon 5y ago
    Indeed it's present under the hood, we are operating on a CPU after all, which treats all of RAM as one giant shared-mutable blob of data. It will always be there, under some abstraction. The point I'm trying to communicate is that in practice, for various reasons, Rust programs do not use shared mutable access to objects to the same extent that C++ programs do. For example, C++ programs use observers, and dependency injection (the pattern, not the framework) to have member pointers to mutable subsystems, and we just don't often see that in Rust programs. This is the paradigm mismatch I'm highlighting: to rewrite a C++ program in Rust often requires a large paradigm shift. The pain is particularly felt when making them interoperate in a gradual refactoring. This is IMO one the bigger reasons that big rewrites to new languages fail, and why new languages benefit from being multi-paradigm, so that there's no paradigm mismatch to impede migrations.
    - westurner 5y ago
      IDK why the private, internal paradigms of e.g. Spanner would even need to be reimplemented in a clone instead of a port? Looks like CockroachDB (Go), TiDB (Go, Rust), and YugabyteDB (C++) are Distributed SQL alternatives to Spanner. ... Which may or may not be able to rely upon inexpensive Lamport clocks: https://en.wikipedia.org/wiki/Lamport_timestamp#Lamport's_logical_clock_in_distributed_systems https://en.wikipedia.org/wiki/Lamport_timestamp#Lamport's_lo...
      - ceeplusplus 5y ago
        All those alternatives you mentioned are nowhere close to being on the level of Spanner in reliability or performance, particularly in a high contention scenario.
        westurner 5y ago
        According to which objective metrics?
      - westurner 5y ago
        From https://westurner.github.io/hnlog/#comment-30603322 https://westurner.github.io/hnlog/#comment-30603322 re https://en.wikipedia-on-ipfs.org/wiki/Database_index https://en.wikipedia-on-ipfs.org/wiki/Database_index : https://github.com/wilsonzlin/edgesearch https://github.com/wilsonzlin/edgesearch : > * Serverless full-text search with Cloudflare Workers, WebAssembly, and Roaring Bitmaps * > "Edgesearch builds a reverse index by mapping terms to a compressed bit set (using Roaring Bitmaps) of IDs of documents containing the term, and creates a custom worker script and data to upload to Cloudflare Workers" WASM or [C++] to WASM? TIL about Roaring Bitmaps: /?q=roaring+bitmap https://medium.com/@amit.desai03/roaring-bitmaps-fast-data-structure-for-inverted-indexes-5490fa4d1b27 https://medium.com/@amit.desai03/roaring-bitmaps-fast-data-s...
deleted 5y ago
[deleted]
celeritascelery 5y ago
Interestingly unsafecell only allows half the shared mutability story. It allows immutable references to be mutated. But it does not allow mutable references to alias. So rust can’t (yet) express all the patterns in C++.
- Rusky 5y ago
  That's backwards. If you can mutate through a shared reference, then there is no need to alias `&mut T`s in the first place! It is C++ that is missing the ability to express unique references like `&mut T`.
  - celeritascelery 5y ago
    > If you can mutate through a shared reference, then there is no need to alias `&mut T`s in the first place! Unfortunately that is not true. See [1] as an example is core where mutable aliasing is needed (async generators). In this case they just added to compiler hack [2][3], but this really needs proper support in rust. Something like “AliasCell”. [1] https://github.com/rust-lang/rust/issues/63818 https://github.com/rust-lang/rust/issues/63818 [2] https://github.com/rust-lang/rust/pull/82834 https://github.com/rust-lang/rust/pull/82834 [3] https://github.com/tokio-rs/tokio/pull/3654 https://github.com/tokio-rs/tokio/pull/3654
    - Rusky 5y ago
      I'm familiar with this issue. But it's a self-imposed problem, not a fundamental limitation relative to C++: the "same people" decided (at different points in time, to be fair) to make the `Future` trait use `&mut Self`, and to let specific futures hold self references. This is a very different issue than the actual ability to express shared mutability in the language. With enough foresight, Rust could have "simply" used `&Self` and `UnsafeCell` for futures (and the current compiler hack with `!Unpin` is basically just replacing `&mut Self` with `&UnsafeCell<Self>` under the hood).