19 ms·

When Rust hurts

296 points by nequo 4y ago

swsieber 4y ago
I believe there is a group working on "safe transmute", which would address the newtype issues... safely. Not just memory safety, but invariant safe, to prevent transmutes in safe rust when the newtype wrapper wants to protect its own invariants.
- shirogane86x 4y ago
  This sounds really interesting (and also eerily similar to haskell's Coercible and roles), is there any material on the matter somewhere? I'd love to see what solution they're thinking of working towards, seeing as this is a pain point that I encounter somewhat frequently.
  - burntsushi 4y ago
    RFC PR: https://github.com/rust-lang/rfcs/pull/2981 https://github.com/rust-lang/rfcs/pull/2981 RFC text: https://github.com/jswrenn/rfcs/blob/safer-transmute/text/0000-safer-transmute.md https://github.com/jswrenn/rfcs/blob/safer-transmute/text/00... I don't know what the status quo is, but it is very exciting work. There are also crates in this domain that work today. The two I'm aware of are https://docs.rs/zerocopy/latest/zerocopy/ https://docs.rs/zerocopy/latest/zerocopy/ and https://docs.rs/bytemuck/latest/bytemuck/ https://docs.rs/bytemuck/latest/bytemuck/
hardwaregeek 4y ago
This is a very nice article! And a very nicely designed blog. I’ve run into almost all of these issues myself. Only in Rust have I had to respond to someone’s PR feedback with “yes, that would be better but the compiler won’t let me for xyz reasons”. I do wonder if someone could design a language with a little more built in logic that would be able to know that an Arc won’t be moved, or that can understand String can be matched to a str, if that could function as the often requested “higher level Rust”
- swsieber 4y ago
  There are a few crates that abstract the unsafe setup you need for a self referential struct, but I'm not sure which or their status. And they might employ "Pin" for their abstraction, I'm not sure. Pin is basically a wrapper type to keep an object from being moved. The interesting thing is that Pin doesn't receive special compiler treatment for its implementation. It's implemented in normal rust (IIRC).
- estebank 4y ago
  > or that can understand String can be matched to a str That's called "deref patterns" and it's been made to work on nightly only for String for now, but 1) the implementation is currently a huge hack, 2) development stalled over the holidays but there's multiple interested parties to see this implemented and stabilized and 3) the idea is to make these "just work" (without any extra sigils or keywords) for specific Deref impls, maybe with a new marker trait DerefPure, which might or might not be available for end users to (similar to trait Try). DerefMut is IIRC still an open question.
- dontlaugh 4y ago
  That language is probably Swift.
mjb 4y ago
> Remember that paths are raw pointers, even in Rust. Most file operations are inherently unsafe and can lead to data races (in a broad sense) if you do not correctly synchronize file access. This is a reality that our operating systems have chosen for us. It's bad. The way Unix does filesystem stuff is both too high level for great performance, and too low level for developer convenience and to make correct programs easy. What would it look like to go higher level? For example: - Operating systems could support START TRANSACTION on filesystem operations, allowing multiple operations to be performed atomically and with isolation. No more having to reason carefully about which posix file operations are atomic, no more having to worry about temp file TOCTOU etc. - fopen(..., 'r') could, by default, operate on a COW snapshot of a file rather than on the raw file on the filesystems. No more having to worry about races with other processes. - Temp files, like database temp tables, could by default only be visible to the current process (and optionally its children) or even the current thread. No more having to worry about temp file races and related issues. That sort of thing. Maybe implemented like DBOS: https://dbos-project.github.io/ https://dbos-project.github.io/. Or, you know, just get rid of the whole idea of processes for server-side applications. Was that really a good idea to begin with? > However, if the locks you use are not re-entrant (and Rust's locks are not), having a single lock is enough to cause a deadlock. I'm a fan of Java's synchronized, and sometimes wish that Rust had something higher-level object-level synchronization primitive that was safer than messing with raw mutexes (which never seems to end well).
- elij 4y ago
  same issue with networking (specifically per packet heuristics) and even Java hasn't got that right either.
- surajrmal 4y ago
  Per application isolated storage is becoming more and more popular. Docker containers, flatpaks, android apps, etc are all providing isolated storage by default. Recursive locks are a nightmare. You cannot reason about lock acquisition order and have to invent new ways to make it safe.
- eska 4y ago
  The filesystem is a database. Databases like Postgres implement such a transaction not through a lock, but by keeping multiple versions of the data/file until the transactions using them closed. Yet other databases operate on the stream of changes and the current state of data is merely the application of all changes until a certain time, allowing multiple transactions to use different snapshots without blocking each other (you can parse a file while somebody else edits it and you won’t be interrupted). I read about various filesystems offering some of these features, but not in IO APIs.
Aissen 4y ago
This is a quite nice article. Do not stop at the first example that shows something passed by value that would probably better be passed by reference instead of always cloning. I really like how the author used green/red backgrounds to show code that doesn't compile.
tipsytoad 4y ago
Also, leaking memory with reference cycles from shared ownership
- dcow 4y ago
  This seems to be a problem that even Swift can’t solve. The only languages that avoid this issue are ones with active GCs. Not that I don't agree and wouldn't love to see efforts made here.
  - swsieber 4y ago
    It'd be possible to add an active GC to Rust. There are some library attempts at that: - https://lib.rs/crates/gc https://lib.rs/crates/gc - https://github.com/withoutboats/shifgrethor https://github.com/withoutboats/shifgrethor But they don't look particularly usable. Beyond that, yeah, it sucks.
- naasking 4y ago
  There are local cycle collection algorithms, but Rust doesn't expose enough metadata to write them in Rust itself. They're also not cheap to run.
ufo 4y ago
What's the idiomatic way to move the self.x+=1 into a separate function? (From the "Function Abstraction" example)
- edflsafoiewq 4y ago
  Construct references to the fields of self inc needs at the call site and pass those instead of all of self, eg. inc(&mut self.x).
- duped 4y ago
  https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=b94daa9a2c2a32b39149b33855c9fa86 https://play.rust-lang.org/?version=stable&mode=debug&editio... But to be honest that's bad code.
- winstonewert 4y ago
  I've found success in such cases by splitting the struct in question. This example is a bit contrived, but you could do something like: self.counter.inc(); Where `self.counter` is a new struct that holds `x`. Because `counter` is its own struct field, I can call mutable methods on it while holding mutable references to other fields. What I've found is that often the current "object" naturally decomposes into different parts each of which can have their own methods that only need to work with their particular subset of the fields.
dcow 4y ago
I really resonate with the async complaints in the article. Rust concurrency is not fearless. I fear writing async code every time I have to do it. I have to think so much harder which ultimately means opportunity for errors. Even though Rust does slightly skirt the colored function issue, my code still bifurcates because truly correct async code needs to use async variants of the standard lib for file access. Knowing when something you call blocks, or grabs a lock, etc. is something which the compiler doesn’t help you with but should. And then how the hell are you supposed to use any code sanely when the entire community is using different async runtimes? Why can’t the runtime be part of the core language with hooks into parts that might need to be customized in hyper special cases? It really feels like we lost the steam behind async Rust and we’ve got this partially complete thing that’s neither high level enough to be convenient nor low level enough to be fearlessly correct. Async traits still aren't a thing and it’s been years.
- MuffinFlavored 4y ago
  > I fear writing async code every time I have to do it. Devil's advocate: can you give an example of something complicated you ran into that wasn't "simply" solved by Arc<Mutex<T>>?
  - dxuh 4y ago
    Probably any IO heavy workload, which usually requires async if it should be performant (like in all other programming languages too).
  - dcow 4y ago
    Everything in my example. Scratching my head about whether I can call this part of my code from an async function because it might do filesystem operations or make a blocking network request. If I can’t, jumping through hoops to spawn an async task that can. Oh, now everything has to be static. Oh god async block syntax. I run into this a lot working on an API server written in Rust. It’s part of a larger project that has a CLI and lib crates. So I’ve run into everything I mentioned from poor language support for expressing async traits, to mixing and matching async runtimes and calling conventions.
gardaani 4y ago
I have been hurt by issues similar to inc(&mut self) in the "Functional abstraction" section so many times. I have been considering using macros as a workaround, but so far I have just written the same code multiple times. DRY isn't easy in Rust.
- edflsafoiewq 4y ago
  Me too. You can make the references at the caller and pass them individually to the function, like inc(&mut self.x), but that's also not very nice. It's weird how this is at odds with the purpose of grouping variables into a logical struct in the first place.
netr0ute 4y ago
The part about the filesystem is completely ironic, because the C++ version of the same API uses no pointers at all.
- ynik 4y ago
  I think you misunderstood the article. That section is not talking about actual pointers. Filenames are like pointers, opening a file is like dereferencing the filename to get the actual file. Any other process could "free your pointer" (by deleting the file) at any point in time. That's true no matter which language you use.
  - charcircuit 4y ago
    >Any other process could "free your pointer" (by deleting the file) at any point in time. Files are garbage collected. There is no use after free if someone deletes your file because the OS sees that you are still using it
    - hot_gril 4y ago
      Yeah, common confusion to be out of free space, rm some huge file, and still be out of space because something has the file open.
    - deleted 4y ago
      [deleted]
  - mappu 4y ago
    There's a TOCTTOU going from the filename to the file handle, but once you have a handle, - on Windows, another process deleting the file is prevented (unless you explicitly opted-in to cooperative FILE_SHARE_DELETE); and - on Linux, another process deleting the file only unlinks its name, and your handle continues working with the nameless file and its content
Ciantic 4y ago
For intro and info about Software Transactional Memory which allows real fearless concurrency: "Beyond Locks: Software Transactional Memory" by Bartosz Milewski https://bartoszmilewski.com/2010/09/11/beyond-locks-software-transactional-memory/ https://bartoszmilewski.com/2010/09/11/beyond-locks-software... I wonder how far it can be implemented in Rust? There are people trying: https://github.com/aakoshh/async-stm-rs https://github.com/aakoshh/async-stm-rs (updated just days ago)
Georgelemental 4y ago
Nice article. - The "functional abstraction" issues could some day be adressed with view types: https://smallcultfollowing.com/babysteps//blog/2021/11/05/view-types/ https://smallcultfollowing.com/babysteps//blog/2021/11/05/vi... - "Newtype abstraction" can be made nicer with third-party crates, or maybe in the future with project safe transmute - "Views and bundles" I run into this one all the time, it's annoying - "Deref patterns" could allow pattern matching to see through boxes one day, but there are unresolved design issues
hkalbasi 4y ago
Among things mentioned in the article, pattern matching of strings is already available in the nightly.
pornel 4y ago
Haskell also solves data races and use-after-free (with immutability and GC), so it's not surprising the author isn't impressed by Rust's fearless concurrency. But debugging deadlocks in threaded code is a piece of cake compared to the hell of data races (attach debugger, and stack traces will show what is deadlocking vs races happening randomly, often impossible to reproduce in a debugger).
zamalek 4y ago
Some of these are more "When a linear type system hurts:" > Common expression elimination I feel as though there should be a warning when a parameter that could be a reference is not. I usually run into this when a function that I've written is incorrectly taking ownership of a parameter. > Newtype abstraction The Rust compiler could implicitly call `into` and `inner` (and then possibly optimize those away), but this (explicit or implicit) is required for linear types. What could work is: `Hex(&bytes)`.
alcover 4y ago
fn f<T: Into<MyType>>(t: T) -> MyType { t.into() } Picturing a group of nuclear-winter survivors trying to restart a coal plant. Found the control room. Inside, a working diesel generator, a computer and a Rust manual.
- nequo 4y ago
  The binary would most likely work correctly so they could restart the coal plant, and read the Rust manual later to understand the source code.
  - badrequest 4y ago
    And who wouldn't want to run a binary that compiles to some unknown effect after a devastating apocalypse?
  - alcover 4y ago
    I agree my fiction is leaky, but it sounds like a nice prompt.
- Klonoar 4y ago
  Are you just being snarky or are you actually assuming that is code you'd write?
  - alcover 4y ago
    Being gently snarky. The code is from the article and frankly looks an irradiated still-born.
    - tialaramex 4y ago
      I find Rust way more readable than the similarly powerful C++. fn f<T: Into<MyType>>(t: T) -> MyType { t.into() } We've got an introducer keyword 'fn'. We're defining a fn function named f. Cool, in C++ I might have gotten a long way before I found out what the function's name was. Hopefully this is an example name, obviously don't really just name your functions "f" <T: Into<MyType>> means there's a type parameter here, our f function can work with different types, T, however, it also constrains the type parameter by saying it must implement a Trait, named Into<MyType>. In Rust we're allowed to also write these type constraints separately, and if they're complicated it's bad style to put them here, but this constraint was very simple. (t: T) means the f function takes a single parameter, named t (again, please give things better names in real life people) of type T, that T is the one we just met, it's some type which implements Into<MyType> -> MyType means the function returns a value of MyType. { t.into() } is the entire function definition. This function just calls the method into() on the parameter t. How does it know it can call into() ? If you're a C++ programmer you might assume it's just duck typed, C++ just does text replacement here which is how you get those horrible multi-page scrolling template errors, but nope, in Rust it knew because the thing we know about t is that its type is T, and the only thing we know about T is that it implements Into<MyType> which is a trait with exactly this into() method defined, where the return type of that into() method will be MyType. We actually didn't know how to do almost anything else with t except call into()
retrocat 4y ago
> We know that moving `Arc<Db>` will not change the location of the `Db` object, but there is no way to communicate this information to rustc. There is! `std::pin`[0] does just this (and `Arc<T>` even implements `Unpin`, so the overall struct can be `Unpin` if everything else is `Unpin`). The page even includes a self-referential struct example, though it does use `unsafe`. [0]: https://doc.rust-lang.org/std/pin/index.html https://doc.rust-lang.org/std/pin/index.html
mmastrac 4y ago
I think this article is missing a mention of #[repr(transparent)], which is supposed to help bridge the problem of orphan rules, etc. I'm not sure why there isn't a guarantee that Rust will treat a wrapped type with a single field in the same way as a naked type, but I suspect it's because they Just Haven't Gotten To It Yet. https://www.reddit.com/r/rust/comments/kbyb6z/what_is_the_purpose_of_reprtransparent/ https://www.reddit.com/r/rust/comments/kbyb6z/what_is_the_pu... Also, the "reference" issue described in the article later is painful, but can be solved w/the Yoke or Rental crates right now. I hope those make it into the stdlib, however.
- tialaramex 4y ago
  > I'm not sure why there isn't a guarantee that Rust will treat a wrapped type with a single field in the same way as a naked type, but I suspect it's because they Just Haven't Gotten To It Yet. What do you mean by "treat it in the same way" ? #[repr(transparent)] does promise you something with identical layout and alignment. If whatever is in there is actually the same kind of thing then the transmute will work, but the problem is - what if it's not? Take nook::BalancedI8 and std::num::NonZeroI8 these both have the same representation as Rust's i8 built-in type, they're one byte signed integers. But, wait a minute Option<BalancedI8> and Option<NonZeroI8> also have the same representation. Now, if that single byte is the value 9, we're fine, all five of these types agree that's just 9, no problem. Same for -106 or 35 or most other values. But, for 0 it's a problem. i8, BalancedI8 and Option<BalancedI8> all agree that's zero. However NonZeroI8 can't conceive of this value, we've induced Undefined Behaviour, and Option<NonZeroI8> says this is None. Similarly at -128, i8, NonZeroI8 and Option<NonZeroI8> all agree that's -128, their minimum possible value. However the minimum value of BalancedI8 is -127 so this representation is Undefined Behaviour, and in Option<BalanacedI8> that's None.
bornfreddy 4y ago
I only wrote a short "helloworld.rs" in Rust, but it looks way more complex than any other language I have tried. Which is a shame because I really like the idea of safe(r) C. Is there really no simpler way to achieve memory safety without GC? I'm not talking about the borrower, but the whole language... In other words, is the complexity of Rust a consequence of memory handling or of the design decisions?
- mmastrac 4y ago
  Are you doing more than this? pub fn main() { println!("Hello, world!"); } If you're hitting the borrow checker and you haven't found yourself at that point because a profiler or benchmark suggested it, you might just be trying to access too much of Rust's power for what you need. You can always just throw everything inside of Rc<RefCell<...>> if you're using a single thread, or Arc<Mutex<...>>/Arc<RwLock<...>> if you're doing multi-threading. It looks expensive, but what you end up with is approx. the same accounting you find in higher-level languages, but the same guarantees.
  - mr_00ff00 4y ago
    Isn’t it kinda a catch-22 with Arc? That’s considered a form of garbage collection. It’s like, if you can use Arc than you can use garbage collection, in which case why battle the borrow checker and just use Elixir or something which better guarantees and an easier time.
wheelerof4te 4y ago
And this is why Python, Java and Javascript rule modern day programming landscape. Those languages let you just do the frickin' work TM.
- vgatherps 4y ago
  If Python or Javascript are even potential candidates for what you should build something in, it’s questionable whether you should use Rust in the first place.
topaz0 4y ago
I have mainly read about rust, not worked with it, but the section about: let x = compute_x(); f(x.clone()); g(x); was confusing/surprising to me. Isn't the whole point of default-immutable that you can use x without owning a copy? Forgive me if this is a stupid question.
- hawski 4y ago
  I'm no Rust expert, but it would be ok if you would do borrows there. let x = compute_x(); f(&x); g(&x); Otherwise you have moves there. Now it depends on the API, it may want the value and not the reference (a borrow).
  - deleted 4y ago
    [deleted]
  - topaz0 4y ago
    I see. Is it not common practice for functions to expect references when they can (i.e. when they don't need to modify the argument)?
    - tialaramex 4y ago
      Not just when they don't need to modify the argument, when they don't need to consume it. The way these parameters are written takes ownership of x, it's gone, not just modified. For a non-trivial object (which apparently x is), you would always prefer to write &x (immutable reference to x) or at least &mut x (mutable reference, not applicable here because x is not defined as a mutable thing). So apparently these functions intend to consume x. That's not common but there are a reasonable number of Rust APIs which consume larger things on purpose.
      - topaz0 4y ago
        > So apparently these functions intend to consume x. I guess in that case I don't really get the complaint. If there's a good reason for f or g to consume x, I would think I would always want the copy to be explicit. But I could still be missing some subtlety I guess.
pjmlp 4y ago
Found a soul mate regarding my point of view on fearless concurrency.
up2isomorphism 4y ago
The closest thing to fearless concurrency is the Unix os shell. Are you afraid of grep asdf 1.txt | gzip ? Probably not, you probably don’t even noticed this is a parallel execution.

296 points by nequo 4y ago

I believe there is a group working on "safe transmute", which would address the newtype issues... safely. Not just memory safety, but invariant safe, to prevent transmutes in safe rust when the newtype wrapper wants to protect its own invariants.

shirogane86x 4y ago
This sounds really interesting (and also eerily similar to haskell's Coercible and roles), is there any material on the matter somewhere? I'd love to see what solution they're thinking of working towards, seeing as this is a pain point that I encounter somewhat frequently.
- burntsushi 4y ago
  RFC PR: https://github.com/rust-lang/rfcs/pull/2981 https://github.com/rust-lang/rfcs/pull/2981 RFC text: https://github.com/jswrenn/rfcs/blob/safer-transmute/text/0000-safer-transmute.md https://github.com/jswrenn/rfcs/blob/safer-transmute/text/00... I don't know what the status quo is, but it is very exciting work. There are also crates in this domain that work today. The two I'm aware of are https://docs.rs/zerocopy/latest/zerocopy/ https://docs.rs/zerocopy/latest/zerocopy/ and https://docs.rs/bytemuck/latest/bytemuck/ https://docs.rs/bytemuck/latest/bytemuck/

hardwaregeek 4y ago

This is a very nice article! And a very nicely designed blog. I’ve run into almost all of these issues myself. Only in Rust have I had to respond to someone’s PR feedback with “yes, that would be better but the compiler won’t let me for xyz reasons”. I do wonder if someone could design a language with a little more built in logic that would be able to know that an Arc won’t be moved, or that can understand String can be matched to a str, if that could function as the often requested “higher level Rust”

swsieber 4y ago
There are a few crates that abstract the unsafe setup you need for a self referential struct, but I'm not sure which or their status. And they might employ "Pin" for their abstraction, I'm not sure. Pin is basically a wrapper type to keep an object from being moved. The interesting thing is that Pin doesn't receive special compiler treatment for its implementation. It's implemented in normal rust (IIRC).
estebank 4y ago
> or that can understand String can be matched to a str That's called "deref patterns" and it's been made to work on nightly only for String for now, but 1) the implementation is currently a huge hack, 2) development stalled over the holidays but there's multiple interested parties to see this implemented and stabilized and 3) the idea is to make these "just work" (without any extra sigils or keywords) for specific Deref impls, maybe with a new marker trait DerefPure, which might or might not be available for end users to (similar to trait Try). DerefMut is IIRC still an open question.
dontlaugh 4y ago
That language is probably Swift.

mjb 4y ago

> Remember that paths are raw pointers, even in Rust. Most file operations are inherently unsafe and can lead to data races (in a broad sense) if you do not correctly synchronize file access. This is a reality that our operating systems have chosen for us. It's bad. The way Unix does filesystem stuff is both too high level for great performance, and too low level for developer convenience and to make correct programs easy. What would it look like to go higher level? For example: - Operating systems could support START TRANSACTION on filesystem operations, allowing multiple operations to be performed atomically and with isolation. No more having to reason carefully about which posix file operations are atomic, no more having to worry about temp file TOCTOU etc. - fopen(..., 'r') could, by default, operate on a COW snapshot of a file rather than on the raw file on the filesystems. No more having to worry about races with other processes. - Temp files, like database temp tables, could by default only be visible to the current process (and optionally its children) or even the current thread. No more having to worry about temp file races and related issues. That sort of thing. Maybe implemented like DBOS: https://dbos-project.github.io/ https://dbos-project.github.io/. Or, you know, just get rid of the whole idea of processes for server-side applications. Was that really a good idea to begin with? > However, if the locks you use are not re-entrant (and Rust's locks are not), having a single lock is enough to cause a deadlock. I'm a fan of Java's synchronized, and sometimes wish that Rust had something higher-level object-level synchronization primitive that was safer than messing with raw mutexes (which never seems to end well).

elij 4y ago
same issue with networking (specifically per packet heuristics) and even Java hasn't got that right either.
surajrmal 4y ago
Per application isolated storage is becoming more and more popular. Docker containers, flatpaks, android apps, etc are all providing isolated storage by default. Recursive locks are a nightmare. You cannot reason about lock acquisition order and have to invent new ways to make it safe.
eska 4y ago
The filesystem is a database. Databases like Postgres implement such a transaction not through a lock, but by keeping multiple versions of the data/file until the transactions using them closed. Yet other databases operate on the stream of changes and the current state of data is merely the application of all changes until a certain time, allowing multiple transactions to use different snapshots without blocking each other (you can parse a file while somebody else edits it and you won’t be interrupted). I read about various filesystems offering some of these features, but not in IO APIs.

Aissen 4y ago

This is a quite nice article. Do not stop at the first example that shows something passed by value that would probably better be passed by reference instead of always cloning. I really like how the author used green/red backgrounds to show code that doesn't compile.

tipsytoad 4y ago

Also, leaking memory with reference cycles from shared ownership

dcow 4y ago
This seems to be a problem that even Swift can’t solve. The only languages that avoid this issue are ones with active GCs. Not that I don't agree and wouldn't love to see efforts made here.
- swsieber 4y ago
  It'd be possible to add an active GC to Rust. There are some library attempts at that: - https://lib.rs/crates/gc https://lib.rs/crates/gc - https://github.com/withoutboats/shifgrethor https://github.com/withoutboats/shifgrethor But they don't look particularly usable. Beyond that, yeah, it sucks.
naasking 4y ago
There are local cycle collection algorithms, but Rust doesn't expose enough metadata to write them in Rust itself. They're also not cheap to run.

ufo 4y ago

What's the idiomatic way to move the self.x+=1 into a separate function? (From the "Function Abstraction" example)

edflsafoiewq 4y ago
Construct references to the fields of self inc needs at the call site and pass those instead of all of self, eg. inc(&mut self.x).
duped 4y ago
https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=b94daa9a2c2a32b39149b33855c9fa86 https://play.rust-lang.org/?version=stable&mode=debug&editio... But to be honest that's bad code.
winstonewert 4y ago
I've found success in such cases by splitting the struct in question. This example is a bit contrived, but you could do something like: self.counter.inc(); Where `self.counter` is a new struct that holds `x`. Because `counter` is its own struct field, I can call mutable methods on it while holding mutable references to other fields. What I've found is that often the current "object" naturally decomposes into different parts each of which can have their own methods that only need to work with their particular subset of the fields.

dcow 4y ago

I really resonate with the async complaints in the article. Rust concurrency is not fearless. I fear writing async code every time I have to do it. I have to think so much harder which ultimately means opportunity for errors. Even though Rust does slightly skirt the colored function issue, my code still bifurcates because truly correct async code needs to use async variants of the standard lib for file access. Knowing when something you call blocks, or grabs a lock, etc. is something which the compiler doesn’t help you with but should. And then how the hell are you supposed to use any code sanely when the entire community is using different async runtimes? Why can’t the runtime be part of the core language with hooks into parts that might need to be customized in hyper special cases? It really feels like we lost the steam behind async Rust and we’ve got this partially complete thing that’s neither high level enough to be convenient nor low level enough to be fearlessly correct. Async traits still aren't a thing and it’s been years.

MuffinFlavored 4y ago
> I fear writing async code every time I have to do it. Devil's advocate: can you give an example of something complicated you ran into that wasn't "simply" solved by Arc<Mutex<T>>?
- dxuh 4y ago
  Probably any IO heavy workload, which usually requires async if it should be performant (like in all other programming languages too).
- dcow 4y ago
  Everything in my example. Scratching my head about whether I can call this part of my code from an async function because it might do filesystem operations or make a blocking network request. If I can’t, jumping through hoops to spawn an async task that can. Oh, now everything has to be static. Oh god async block syntax. I run into this a lot working on an API server written in Rust. It’s part of a larger project that has a CLI and lib crates. So I’ve run into everything I mentioned from poor language support for expressing async traits, to mixing and matching async runtimes and calling conventions.

gardaani 4y ago

I have been hurt by issues similar to inc(&mut self) in the "Functional abstraction" section so many times. I have been considering using macros as a workaround, but so far I have just written the same code multiple times. DRY isn't easy in Rust.

edflsafoiewq 4y ago
Me too. You can make the references at the caller and pass them individually to the function, like inc(&mut self.x), but that's also not very nice. It's weird how this is at odds with the purpose of grouping variables into a logical struct in the first place.

netr0ute 4y ago

The part about the filesystem is completely ironic, because the C++ version of the same API uses no pointers at all.

ynik 4y ago
I think you misunderstood the article. That section is not talking about actual pointers. Filenames are like pointers, opening a file is like dereferencing the filename to get the actual file. Any other process could "free your pointer" (by deleting the file) at any point in time. That's true no matter which language you use.
- charcircuit 4y ago
  >Any other process could "free your pointer" (by deleting the file) at any point in time. Files are garbage collected. There is no use after free if someone deletes your file because the OS sees that you are still using it
  - hot_gril 4y ago
    Yeah, common confusion to be out of free space, rm some huge file, and still be out of space because something has the file open.
  - deleted 4y ago
    [deleted]
- mappu 4y ago
  There's a TOCTTOU going from the filename to the file handle, but once you have a handle, - on Windows, another process deleting the file is prevented (unless you explicitly opted-in to cooperative FILE_SHARE_DELETE); and - on Linux, another process deleting the file only unlinks its name, and your handle continues working with the nameless file and its content

Ciantic 4y ago

For intro and info about Software Transactional Memory which allows real fearless concurrency: "Beyond Locks: Software Transactional Memory" by Bartosz Milewski https://bartoszmilewski.com/2010/09/11/beyond-locks-software-transactional-memory/ https://bartoszmilewski.com/2010/09/11/beyond-locks-software... I wonder how far it can be implemented in Rust? There are people trying: https://github.com/aakoshh/async-stm-rs https://github.com/aakoshh/async-stm-rs (updated just days ago)

Georgelemental 4y ago

Nice article. - The "functional abstraction" issues could some day be adressed with view types: https://smallcultfollowing.com/babysteps//blog/2021/11/05/view-types/ https://smallcultfollowing.com/babysteps//blog/2021/11/05/vi... - "Newtype abstraction" can be made nicer with third-party crates, or maybe in the future with project safe transmute - "Views and bundles" I run into this one all the time, it's annoying - "Deref patterns" could allow pattern matching to see through boxes one day, but there are unresolved design issues

hkalbasi 4y ago

Among things mentioned in the article, pattern matching of strings is already available in the nightly.

pornel 4y ago

Haskell also solves data races and use-after-free (with immutability and GC), so it's not surprising the author isn't impressed by Rust's fearless concurrency. But debugging deadlocks in threaded code is a piece of cake compared to the hell of data races (attach debugger, and stack traces will show what is deadlocking vs races happening randomly, often impossible to reproduce in a debugger).

zamalek 4y ago

Some of these are more "When a linear type system hurts:" > Common expression elimination I feel as though there should be a warning when a parameter that could be a reference is not. I usually run into this when a function that I've written is incorrectly taking ownership of a parameter. > Newtype abstraction The Rust compiler could implicitly call `into` and `inner` (and then possibly optimize those away), but this (explicit or implicit) is required for linear types. What could work is: `Hex(&bytes)`.

alcover 4y ago

fn f<T: Into<MyType>>(t: T) -> MyType { t.into() } Picturing a group of nuclear-winter survivors trying to restart a coal plant. Found the control room. Inside, a working diesel generator, a computer and a Rust manual.

nequo 4y ago
The binary would most likely work correctly so they could restart the coal plant, and read the Rust manual later to understand the source code.
- badrequest 4y ago
  And who wouldn't want to run a binary that compiles to some unknown effect after a devastating apocalypse?
- alcover 4y ago
  I agree my fiction is leaky, but it sounds like a nice prompt.
Klonoar 4y ago
Are you just being snarky or are you actually assuming that is code you'd write?
- alcover 4y ago
  Being gently snarky. The code is from the article and frankly looks an irradiated still-born.
  - tialaramex 4y ago
    I find Rust way more readable than the similarly powerful C++. fn f<T: Into<MyType>>(t: T) -> MyType { t.into() } We've got an introducer keyword 'fn'. We're defining a fn function named f. Cool, in C++ I might have gotten a long way before I found out what the function's name was. Hopefully this is an example name, obviously don't really just name your functions "f" <T: Into<MyType>> means there's a type parameter here, our f function can work with different types, T, however, it also constrains the type parameter by saying it must implement a Trait, named Into<MyType>. In Rust we're allowed to also write these type constraints separately, and if they're complicated it's bad style to put them here, but this constraint was very simple. (t: T) means the f function takes a single parameter, named t (again, please give things better names in real life people) of type T, that T is the one we just met, it's some type which implements Into<MyType> -> MyType means the function returns a value of MyType. { t.into() } is the entire function definition. This function just calls the method into() on the parameter t. How does it know it can call into() ? If you're a C++ programmer you might assume it's just duck typed, C++ just does text replacement here which is how you get those horrible multi-page scrolling template errors, but nope, in Rust it knew because the thing we know about t is that its type is T, and the only thing we know about T is that it implements Into<MyType> which is a trait with exactly this into() method defined, where the return type of that into() method will be MyType. We actually didn't know how to do almost anything else with t except call into()

retrocat 4y ago

> We know that moving `Arc<Db>` will not change the location of the `Db` object, but there is no way to communicate this information to rustc. There is! `std::pin`[0] does just this (and `Arc<T>` even implements `Unpin`, so the overall struct can be `Unpin` if everything else is `Unpin`). The page even includes a self-referential struct example, though it does use `unsafe`. [0]: https://doc.rust-lang.org/std/pin/index.html https://doc.rust-lang.org/std/pin/index.html

mmastrac 4y ago

I think this article is missing a mention of #[repr(transparent)], which is supposed to help bridge the problem of orphan rules, etc. I'm not sure why there isn't a guarantee that Rust will treat a wrapped type with a single field in the same way as a naked type, but I suspect it's because they Just Haven't Gotten To It Yet. https://www.reddit.com/r/rust/comments/kbyb6z/what_is_the_purpose_of_reprtransparent/ https://www.reddit.com/r/rust/comments/kbyb6z/what_is_the_pu... Also, the "reference" issue described in the article later is painful, but can be solved w/the Yoke or Rental crates right now. I hope those make it into the stdlib, however.

tialaramex 4y ago
> I'm not sure why there isn't a guarantee that Rust will treat a wrapped type with a single field in the same way as a naked type, but I suspect it's because they Just Haven't Gotten To It Yet. What do you mean by "treat it in the same way" ? #[repr(transparent)] does promise you something with identical layout and alignment. If whatever is in there is actually the same kind of thing then the transmute will work, but the problem is - what if it's not? Take nook::BalancedI8 and std::num::NonZeroI8 these both have the same representation as Rust's i8 built-in type, they're one byte signed integers. But, wait a minute Option<BalancedI8> and Option<NonZeroI8> also have the same representation. Now, if that single byte is the value 9, we're fine, all five of these types agree that's just 9, no problem. Same for -106 or 35 or most other values. But, for 0 it's a problem. i8, BalancedI8 and Option<BalancedI8> all agree that's zero. However NonZeroI8 can't conceive of this value, we've induced Undefined Behaviour, and Option<NonZeroI8> says this is None. Similarly at -128, i8, NonZeroI8 and Option<NonZeroI8> all agree that's -128, their minimum possible value. However the minimum value of BalancedI8 is -127 so this representation is Undefined Behaviour, and in Option<BalanacedI8> that's None.

bornfreddy 4y ago

I only wrote a short "helloworld.rs" in Rust, but it looks way more complex than any other language I have tried. Which is a shame because I really like the idea of safe(r) C. Is there really no simpler way to achieve memory safety without GC? I'm not talking about the borrower, but the whole language... In other words, is the complexity of Rust a consequence of memory handling or of the design decisions?

mmastrac 4y ago
Are you doing more than this? pub fn main() { println!("Hello, world!"); } If you're hitting the borrow checker and you haven't found yourself at that point because a profiler or benchmark suggested it, you might just be trying to access too much of Rust's power for what you need. You can always just throw everything inside of Rc<RefCell<...>> if you're using a single thread, or Arc<Mutex<...>>/Arc<RwLock<...>> if you're doing multi-threading. It looks expensive, but what you end up with is approx. the same accounting you find in higher-level languages, but the same guarantees.
- mr_00ff00 4y ago
  Isn’t it kinda a catch-22 with Arc? That’s considered a form of garbage collection. It’s like, if you can use Arc than you can use garbage collection, in which case why battle the borrow checker and just use Elixir or something which better guarantees and an easier time.

wheelerof4te 4y ago

And this is why Python, Java and Javascript rule modern day programming landscape. Those languages let you just do the frickin' work TM.

vgatherps 4y ago
If Python or Javascript are even potential candidates for what you should build something in, it’s questionable whether you should use Rust in the first place.

topaz0 4y ago

I have mainly read about rust, not worked with it, but the section about: let x = compute_x(); f(x.clone()); g(x); was confusing/surprising to me. Isn't the whole point of default-immutable that you can use x without owning a copy? Forgive me if this is a stupid question.

hawski 4y ago
I'm no Rust expert, but it would be ok if you would do borrows there. let x = compute_x(); f(&x); g(&x); Otherwise you have moves there. Now it depends on the API, it may want the value and not the reference (a borrow).
- deleted 4y ago
  [deleted]
- topaz0 4y ago
  I see. Is it not common practice for functions to expect references when they can (i.e. when they don't need to modify the argument)?
  - tialaramex 4y ago
    Not just when they don't need to modify the argument, when they don't need to consume it. The way these parameters are written takes ownership of x, it's gone, not just modified. For a non-trivial object (which apparently x is), you would always prefer to write &x (immutable reference to x) or at least &mut x (mutable reference, not applicable here because x is not defined as a mutable thing). So apparently these functions intend to consume x. That's not common but there are a reasonable number of Rust APIs which consume larger things on purpose.
    - topaz0 4y ago
      > So apparently these functions intend to consume x. I guess in that case I don't really get the complaint. If there's a good reason for f or g to consume x, I would think I would always want the copy to be explicit. But I could still be missing some subtlety I guess.

pjmlp 4y ago

Found a soul mate regarding my point of view on fearless concurrency.

up2isomorphism 4y ago

The closest thing to fearless concurrency is the Unix os shell. Are you afraid of grep asdf 1.txt | gzip ? Probably not, you probably don’t even noticed this is a parallel execution.