8 ms·

Rust docs never claimed to prevent race conditions (which he doesn't call by name). https://doc.rust-lang.org/nomicon/races.html https://doc.rust-lang.org/nomi

Rust docs never claimed to prevent race conditions (which he doesn't call by name). https://doc.rust-lang.org/nomicon/races.html https://doc.rust-lang.org/nomicon/races.html

pjmlp 6y ago
True, but that is something that many miss a lot when discussing Rust virtues. Yes the type system is a great help when dealing with thread based concurrency/paralellism, but not so much for the scenarios where we are already using type safe managed languages in distributed computing across multiple processes, most likely not even written in the same language. Which by the way, given the recent security exploits, is much preferable to go back to, instead of relying on threads for parallelism.
- mratsim 6y ago
  That's my main grip with Rust "Fearless Concurrency", fearing concurrency and dipping into it with your toes is a good thing. Claiming "Fearless Concurrency" but ignoring synchronization bugs, at a low-level in concurrent data structures or at a high-level in concurrent systems is misleading. People will be fearlessly introducing livelocks.
  - jstrong 6y ago
    Have you written a lot of concurrent code in rust? Did your experience doing so convince you that it's something you should "fear" doing?
    - mratsim 6y ago
      I've written a multithreading runtime from the ground up in Nim.[1] And yes, writing that taught me that current programming languages are not sufficiently equipped to deal with concurrency bugs, including Rust. The bugs that I had where of the dining philosophers kind (deadlocks or livelocks when trying to put a thread to sleep for example or resolving data dependencies between multiple threads), some were due to a bug in glibc where only formal verification allowed me to ensure that I was doing the correct thing and it was the lower-level layers that was incorrect.[2] Similarly, state machine formal verification to prevent design bugs is something that hardware engineers are deeply aware of but would also be extremely useful to ensure "correct-by-construction" event driven code in software engineering. In comparison, we have many more tools to address memory bugs (Address Sanitizer, Valgrind and fuzzers in particular) but synchronization and communicating state machine bugs are still impractically addressed at the moment. This is something I'd really want Nim to explore and I'm really excited about the Z3 integration[3] to enable new formally verified use-cases[4]. Unfortunately, when I express concerns about those bugs to people from the Rust community, they tend to dismiss those as if the borrow checker was the panacea to all multithreading problems. [1]: https://github.com/mratsim/weave https://github.com/mratsim/weave [2]: https://github.com/mratsim/weave/issues/56 https://github.com/mratsim/weave/issues/56 [3]: https://nim-lang.org/docs/drnim.html https://nim-lang.org/docs/drnim.html [4]: https://github.com/nim-lang/RFCs/issues/222 https://github.com/nim-lang/RFCs/issues/222
      - staticassertion 6y ago
        Sorry that I see your post is downvoted, I'm generally in agreement, as an avid rust programmer. Rust can express a lot of safety guarantees, but there's a massive amount of room for improvements, and I'm sorry that the community hasn't been accepting of that in your interactions. > Similarly, state machine formal verification to prevent design bugs are something that hardware engineers are deeply aware of but would also be extremely useful to ensure "correct-by-construction" event driven code. This reminds me a lot of P. Rust has a crate that I wouldn't necessarily recommend, but it might be similar to what you want: https://github.com/fitzgen/state_machine_future https://github.com/fitzgen/state_machine_future Of course, in this case the code is the model, there is no verification here. The Z3 integration with Nim is super cool, would love to see something like that for Rust. I know of some existing research that reminds me of this but I don't have time to dig up the paper - it was implemented in macros, and looked a lot like what you've got there, with pre/post conditions, ensures, etc. If someone has that paper, please link it! I would love to reread. edit: Here's one rust crate leveraging Z3, but not the one I was thinking of https://github.com/Rust-Proof/rustproof https://github.com/Rust-Proof/rustproof
      - ibotty 6y ago
        Did you write a bug report to glibc? I'd love to read about how the bug is happening.
        mratsim 6y ago
        I didn't, however you can read my writeup and investigation attempts here: - https://github.com/mratsim/weave/issues/56 https://github.com/mratsim/weave/issues/56 And the fix was directly using Linux futexes: - https://github.com/mratsim/weave/pull/58/files https://github.com/mratsim/weave/pull/58/files
  - ragnese 6y ago
    I agree that "fearless concurrency" isn't the best tagline. While I'm definitely less fearful with concurrency in Rust, I'm still pretty fearful... I'm very good at writing deadlocks.
- bluejekyll 6y ago
  > but that is something that many miss a lot when discussing Rust virtues. When someone says “Rust is thread-safe”, Rustaceans in one common voice reply, “Rust is data-race-safe”. After that, folks get into a good conversation about what the difference is, and learn about deadlocks and other issue in threads. Generally speaking though, it does mean that multi-threaded programming in Rust is memory-safe, and won’t introduce a significant class of bugs that you see in C and C++, where so many exploits come from.
  - fluffything 6y ago
    Thread-safety is defined as "the absence of data-races", so technically speaking, saying that Rust is "thread-safe" and saying that it provides "data-race freedom", is just two different forms of saying the exact same thing. The problem is that some people don't know what these mean. I'm not sure why though. Like, I could understand that somebody coming to Rust from Java wouldn't know what "memory safety" means since it is essentially a solution to a problem that Java does not have. But most high-level languages do not protect you from data-races, so this is a problem that Rust does solve and one can run into in most widely-used programming languages (e.g. in Java they are also called data-races).
    - mratsim 6y ago
      This is not what thread safety means: > Thread safety is a computer programming concept applicable to multi-threaded code. Thread-safe code only manipulates shared data structures in a manner that ensures that all threads behave properly and fulfill their design specifications without unintended interaction. There are various strategies for making thread-safe data structures. (Wikipedia) Data races is only part of the problem. Thread-safe also means that your design is bug-free and you don't have livelocks for example because of how you state machines interact there is a way for them to loop forever in an unexpected way (example, dining philosophers). Similarly, data-race-freedom as provided by Rust is helpful for end users but not for library developers. Developers of concurrent data structures or threading primitives are on their own to sail in the complexities of the C++11 Memory Model (i.e. Acquire/Release) and the compiler cannot verify that you didn't introduce a synchronization bug in your new concurrent queue or event bus.
- fluffything 6y ago
  > True, but that is something that many miss a lot when discussing Rust virtues. The large majority of comments I read about Rust safety claims are very explicit that Rust provides memory safety and data-race freedom. You seem to be arguing that these comments are leaving out important information by not clarifying that "This does not mean that Rust prevents all bugs, e.g., you can still write programs with logic bugs, race conditions, dead locks, DDoS, ... and the list goes on and on... in Rust". If somebody does not know what that "memory safety" and "data-race freedom" is, they should just ask, but if they don't, and they make an incorrect assumption, then its kind of their fault, not the fault of whoever wrote such a comment for not mentioning all the things that these two properties do not guarantee. Also, it is actually trivial to build Rust programs that prevent some race-conditions, like deadlocks (just a cargo flag away), so if anything, I'd complain that people being "too correct" by restricting themselves to "data-race freedom" are leaving out some important advantages of Rust.
  - loopz 6y ago
    It's really hard to envision exactly all the problems parallellization might enable in a given system, as it'll all depend on exact implementation and execution details. The safe assumption is whenever you introduce concurrency or even event-handling, stuff might implicitly FUBAR, unless explicitly designed not to. This is a tough pill to swallow when people can't envision exactly all the problems parallellization might enable in a given system (circular argument).
  - ashtonkem 6y ago
    My experience is that Rust advocates often go well out of their way to specify the limits of Rust’s protection, and will even rush in to correct someone who over promises what Rust can give you.
    - htfy96 6y ago
      r/rust and users.rust-lang.org are informative and sensible when it comes to limitations. Twitter and, well, random programming discords are often full of cults
    - 6y ago
- skohan 6y ago
  Yes I can't tell you how many rust talks and articles I have seen which prominently state: "no data races". You never hear the caveat that general race conditions are still possible, and I think your average programming news consumer could be forgiven for not knowing the distinction.
JTenerife 6y ago
"Therefore, rather than making you spend lots of time trying to reproduce the exact circumstances under which a runtime concurrency bug occurs, incorrect code will refuse to compile and present an error explaining the problem. As a result, you can fix your code while you’re working on it rather than potentially after it has been shipped to production. We’ve nicknamed this aspect of Rust fearless concurrency. Fearless concurrency allows you to write code that is free of subtle bugs and is easy to refactor without introducing new bugs." https://doc.rust-lang.org/1.30.0/book/second-edition/ch16-00-concurrency.html https://doc.rust-lang.org/1.30.0/book/second-edition/ch16-00... These sound like quite bold claims to me.
- remram 6y ago
  Previous sentence in this paragraph: > By leveraging ownership and type checking, __many__ concurrency errors are compile-time errors in Rust rather than runtime errors.
  - mratsim 6y ago
    I'm pretty sure the OP meant that concurrency introduces many subtle bugs that are not related to memory safety and so cannot be prevented by the Rust compiler hence fearless is a bold claim.
lubesGordi 6y ago
Obviously no language can prevent race conditions. Race conditions are usually a flaw of protocol or api design. The whole point of 'fearless concurrency' is that you won't crash because of shared state.
- ajross 6y ago
  Yes, but there's a cost for that, which is something that's sort of at odds with Rust's design philosophy in other areas. Outlawing concurrent shared state means that every communication between two threads needs to be formatted as a "message" in whatever abstraction is in use. It needs to be copied, it needs to snapshot whatever data is needed to interpret it. You can't just set a flag and release the lock. And that's... a problem. Because a lot of code that actually wants concurrency in the same memory space is code that also wants high performance. And Rust serves these applications well in other areas, where "Just as fast as C" is true. But Rust isn't as fast as "C" here. It's demonstrably slower. Is that a good thing? I honestly don't know. It's just a hard problem. But you can't eliminate the idea of "shared locked memory" by fiat. It's a real requirement, and Rust has little to say about it in its standard library or (IIRC, though I'd have to go check) its memory model and locking primitives. It's sort of a no-mans land of "super unsafe outside the box" code. And that's a pity.
  - deleted 6y ago
    [deleted]
  - justinpombrio 6y ago
    > Outlawing concurrent shared state means that every communication between two threads needs to be formatted as a "message" in whatever abstraction is in use. > It's a real requirement, and Rust has little to say about it in its standard library So, I don't know much about concurrency, but you're looking for like, a Mutex, right? They're in the standard library and used all the time. https://doc.rust-lang.org/stable/std/sync/struct.Mutex.html https://doc.rust-lang.org/stable/std/sync/struct.Mutex.html And RwLock, a variant on Mutex that allows either one writer or multiple concurrent readers: https://doc.rust-lang.org/stable/std/sync/struct.RwLock.html https://doc.rust-lang.org/stable/std/sync/struct.RwLock.html
hinkley 6y ago
In particular, locks or ownership do not protect against re-entrancy errors. Recursive state change calls can end up applying multiple changes either in the wrong order or not at all.

Rust docs never claimed to prevent race conditions (which he doesn't call by name). https://doc.rust-lang.org/nomicon/races.html https://doc.rust-lang.org/nomicon/races.html

pjmlp 6y ago

True, but that is something that many miss a lot when discussing Rust virtues. Yes the type system is a great help when dealing with thread based concurrency/paralellism, but not so much for the scenarios where we are already using type safe managed languages in distributed computing across multiple processes, most likely not even written in the same language. Which by the way, given the recent security exploits, is much preferable to go back to, instead of relying on threads for parallelism.

mratsim 6y ago
That's my main grip with Rust "Fearless Concurrency", fearing concurrency and dipping into it with your toes is a good thing. Claiming "Fearless Concurrency" but ignoring synchronization bugs, at a low-level in concurrent data structures or at a high-level in concurrent systems is misleading. People will be fearlessly introducing livelocks.
- jstrong 6y ago
  Have you written a lot of concurrent code in rust? Did your experience doing so convince you that it's something you should "fear" doing?
  - mratsim 6y ago
    I've written a multithreading runtime from the ground up in Nim.[1] And yes, writing that taught me that current programming languages are not sufficiently equipped to deal with concurrency bugs, including Rust. The bugs that I had where of the dining philosophers kind (deadlocks or livelocks when trying to put a thread to sleep for example or resolving data dependencies between multiple threads), some were due to a bug in glibc where only formal verification allowed me to ensure that I was doing the correct thing and it was the lower-level layers that was incorrect.[2] Similarly, state machine formal verification to prevent design bugs is something that hardware engineers are deeply aware of but would also be extremely useful to ensure "correct-by-construction" event driven code in software engineering. In comparison, we have many more tools to address memory bugs (Address Sanitizer, Valgrind and fuzzers in particular) but synchronization and communicating state machine bugs are still impractically addressed at the moment. This is something I'd really want Nim to explore and I'm really excited about the Z3 integration[3] to enable new formally verified use-cases[4]. Unfortunately, when I express concerns about those bugs to people from the Rust community, they tend to dismiss those as if the borrow checker was the panacea to all multithreading problems. [1]: https://github.com/mratsim/weave https://github.com/mratsim/weave [2]: https://github.com/mratsim/weave/issues/56 https://github.com/mratsim/weave/issues/56 [3]: https://nim-lang.org/docs/drnim.html https://nim-lang.org/docs/drnim.html [4]: https://github.com/nim-lang/RFCs/issues/222 https://github.com/nim-lang/RFCs/issues/222
    - staticassertion 6y ago
      Sorry that I see your post is downvoted, I'm generally in agreement, as an avid rust programmer. Rust can express a lot of safety guarantees, but there's a massive amount of room for improvements, and I'm sorry that the community hasn't been accepting of that in your interactions. > Similarly, state machine formal verification to prevent design bugs are something that hardware engineers are deeply aware of but would also be extremely useful to ensure "correct-by-construction" event driven code. This reminds me a lot of P. Rust has a crate that I wouldn't necessarily recommend, but it might be similar to what you want: https://github.com/fitzgen/state_machine_future https://github.com/fitzgen/state_machine_future Of course, in this case the code is the model, there is no verification here. The Z3 integration with Nim is super cool, would love to see something like that for Rust. I know of some existing research that reminds me of this but I don't have time to dig up the paper - it was implemented in macros, and looked a lot like what you've got there, with pre/post conditions, ensures, etc. If someone has that paper, please link it! I would love to reread. edit: Here's one rust crate leveraging Z3, but not the one I was thinking of https://github.com/Rust-Proof/rustproof https://github.com/Rust-Proof/rustproof
    - ibotty 6y ago
      Did you write a bug report to glibc? I'd love to read about how the bug is happening.
      - mratsim 6y ago
        I didn't, however you can read my writeup and investigation attempts here: - https://github.com/mratsim/weave/issues/56 https://github.com/mratsim/weave/issues/56 And the fix was directly using Linux futexes: - https://github.com/mratsim/weave/pull/58/files https://github.com/mratsim/weave/pull/58/files
- ragnese 6y ago
  I agree that "fearless concurrency" isn't the best tagline. While I'm definitely less fearful with concurrency in Rust, I'm still pretty fearful... I'm very good at writing deadlocks.
bluejekyll 6y ago
> but that is something that many miss a lot when discussing Rust virtues. When someone says “Rust is thread-safe”, Rustaceans in one common voice reply, “Rust is data-race-safe”. After that, folks get into a good conversation about what the difference is, and learn about deadlocks and other issue in threads. Generally speaking though, it does mean that multi-threaded programming in Rust is memory-safe, and won’t introduce a significant class of bugs that you see in C and C++, where so many exploits come from.
- fluffything 6y ago
  Thread-safety is defined as "the absence of data-races", so technically speaking, saying that Rust is "thread-safe" and saying that it provides "data-race freedom", is just two different forms of saying the exact same thing. The problem is that some people don't know what these mean. I'm not sure why though. Like, I could understand that somebody coming to Rust from Java wouldn't know what "memory safety" means since it is essentially a solution to a problem that Java does not have. But most high-level languages do not protect you from data-races, so this is a problem that Rust does solve and one can run into in most widely-used programming languages (e.g. in Java they are also called data-races).
  - mratsim 6y ago
    This is not what thread safety means: > Thread safety is a computer programming concept applicable to multi-threaded code. Thread-safe code only manipulates shared data structures in a manner that ensures that all threads behave properly and fulfill their design specifications without unintended interaction. There are various strategies for making thread-safe data structures. (Wikipedia) Data races is only part of the problem. Thread-safe also means that your design is bug-free and you don't have livelocks for example because of how you state machines interact there is a way for them to loop forever in an unexpected way (example, dining philosophers). Similarly, data-race-freedom as provided by Rust is helpful for end users but not for library developers. Developers of concurrent data structures or threading primitives are on their own to sail in the complexities of the C++11 Memory Model (i.e. Acquire/Release) and the compiler cannot verify that you didn't introduce a synchronization bug in your new concurrent queue or event bus.
fluffything 6y ago
> True, but that is something that many miss a lot when discussing Rust virtues. The large majority of comments I read about Rust safety claims are very explicit that Rust provides memory safety and data-race freedom. You seem to be arguing that these comments are leaving out important information by not clarifying that "This does not mean that Rust prevents all bugs, e.g., you can still write programs with logic bugs, race conditions, dead locks, DDoS, ... and the list goes on and on... in Rust". If somebody does not know what that "memory safety" and "data-race freedom" is, they should just ask, but if they don't, and they make an incorrect assumption, then its kind of their fault, not the fault of whoever wrote such a comment for not mentioning all the things that these two properties do not guarantee. Also, it is actually trivial to build Rust programs that prevent some race-conditions, like deadlocks (just a cargo flag away), so if anything, I'd complain that people being "too correct" by restricting themselves to "data-race freedom" are leaving out some important advantages of Rust.
- loopz 6y ago
  It's really hard to envision exactly all the problems parallellization might enable in a given system, as it'll all depend on exact implementation and execution details. The safe assumption is whenever you introduce concurrency or even event-handling, stuff might implicitly FUBAR, unless explicitly designed not to. This is a tough pill to swallow when people can't envision exactly all the problems parallellization might enable in a given system (circular argument).
- ashtonkem 6y ago
  My experience is that Rust advocates often go well out of their way to specify the limits of Rust’s protection, and will even rush in to correct someone who over promises what Rust can give you.
  - htfy96 6y ago
    r/rust and users.rust-lang.org are informative and sensible when it comes to limitations. Twitter and, well, random programming discords are often full of cults
  - 6y ago
skohan 6y ago
Yes I can't tell you how many rust talks and articles I have seen which prominently state: "no data races". You never hear the caveat that general race conditions are still possible, and I think your average programming news consumer could be forgiven for not knowing the distinction.

JTenerife 6y ago

"Therefore, rather than making you spend lots of time trying to reproduce the exact circumstances under which a runtime concurrency bug occurs, incorrect code will refuse to compile and present an error explaining the problem. As a result, you can fix your code while you’re working on it rather than potentially after it has been shipped to production. We’ve nicknamed this aspect of Rust fearless concurrency. Fearless concurrency allows you to write code that is free of subtle bugs and is easy to refactor without introducing new bugs." https://doc.rust-lang.org/1.30.0/book/second-edition/ch16-00-concurrency.html https://doc.rust-lang.org/1.30.0/book/second-edition/ch16-00... These sound like quite bold claims to me.

remram 6y ago
Previous sentence in this paragraph: > By leveraging ownership and type checking, __many__ concurrency errors are compile-time errors in Rust rather than runtime errors.
- mratsim 6y ago
  I'm pretty sure the OP meant that concurrency introduces many subtle bugs that are not related to memory safety and so cannot be prevented by the Rust compiler hence fearless is a bold claim.

lubesGordi 6y ago

Obviously no language can prevent race conditions. Race conditions are usually a flaw of protocol or api design. The whole point of 'fearless concurrency' is that you won't crash because of shared state.

ajross 6y ago
Yes, but there's a cost for that, which is something that's sort of at odds with Rust's design philosophy in other areas. Outlawing concurrent shared state means that every communication between two threads needs to be formatted as a "message" in whatever abstraction is in use. It needs to be copied, it needs to snapshot whatever data is needed to interpret it. You can't just set a flag and release the lock. And that's... a problem. Because a lot of code that actually wants concurrency in the same memory space is code that also wants high performance. And Rust serves these applications well in other areas, where "Just as fast as C" is true. But Rust isn't as fast as "C" here. It's demonstrably slower. Is that a good thing? I honestly don't know. It's just a hard problem. But you can't eliminate the idea of "shared locked memory" by fiat. It's a real requirement, and Rust has little to say about it in its standard library or (IIRC, though I'd have to go check) its memory model and locking primitives. It's sort of a no-mans land of "super unsafe outside the box" code. And that's a pity.
- deleted 6y ago
  [deleted]
- justinpombrio 6y ago
  > Outlawing concurrent shared state means that every communication between two threads needs to be formatted as a "message" in whatever abstraction is in use. > It's a real requirement, and Rust has little to say about it in its standard library So, I don't know much about concurrency, but you're looking for like, a Mutex, right? They're in the standard library and used all the time. https://doc.rust-lang.org/stable/std/sync/struct.Mutex.html https://doc.rust-lang.org/stable/std/sync/struct.Mutex.html And RwLock, a variant on Mutex that allows either one writer or multiple concurrent readers: https://doc.rust-lang.org/stable/std/sync/struct.RwLock.html https://doc.rust-lang.org/stable/std/sync/struct.RwLock.html

hinkley 6y ago

In particular, locks or ownership do not protect against re-entrancy errors. Recursive state change calls can end up applying multiple changes either in the wrong order or not at all.