5 ms·

>It's incorrect to say that Rust hasn't prioritized higher-level concurrency tools I think it's a fair characterization given 1.0 shipped without them, and the

>It's incorrect to say that Rust hasn't prioritized higher-level concurrency tools I think it's a fair characterization given 1.0 shipped without them, and there doesn't seem to any timeline for standardization (please correct me if I'm wrong). I don't think I have to tell you that concurrency is one of the most important challenges in modern programming. But all Rust gives you today (and for the foreseeable future) is a pthreads wrapper. >I'm also confused about your perception that the rhetoric gets ahead of the language The parent comment says verbatim "Rust goes out of its way to make it easy and safe to write multithreaded code". This apparently doesn't include preventing deadlocks, a problem that is common, hard-to-avoid, and difficult to recover from. Does that not strike you as a bit of an overreach?

the_why_of_y 11y ago
> deadlocks, a problem that is common, hard-to-avoid, and difficult to recover from Deadlocks are by far the easiest concurrency problem to debug, since it's very obvious when your program is deadlocked, and in most cases a stack trace of the involved threads is sufficient to debug and fix a deadlock. Also, if your program is deadlocked it won't corrupt user data. Rust's std::sync::Mutex is fortunately non-recursive, which makes it easier to find deadlocks during testing. Data races are far worse since they may cause arbitrary effects at a later point in program execution, so they take a lot of developer time to track down and very likely lead to data corruption.
- acconsta 11y ago
  If deadlocks are so easy to fix why do they happen so often? Seriously, how many applications have you seen hang in your life? (not to discount the role of other race conditions in causing hangs, which the borrow checker also can't categorically prevent).
  - GFK_of_xmaspast 11y ago
    Null pointer derefs are easy to find and fix and : yet.
    - neandrake 11y ago
      This is not always the case. Null pointer derefs are just indicators that the program got into an invalid state. Is the fix to provide some alternative pathway when the program got in a bad state or is it to prevent the bad state from happening in the first place? If the fix is to prevent the bad state, tracking down how that state occurred in the first place can often be time consuming in the case of data races.
  - neandrake 11y ago
    I believe the comment reads that deadlocks are the "easiest concurrency problem" to debug -- but your question is moreso headed to the point that they are easy to introduce in a system. From the example, from a deadlock the locked threads and their stacktraces can be determined, which will help in pointing towards where the cause of the problem is. Compared to situations where the problem occurs due to a data-race causing an unexpected/invalid state but a problem doesn't manifest until later. Fixing this type of problem is more problematic as usually any exception/stacktrace that might crop up does not relate any information as to how the state became invalid in the first place. I tend to see these types of bugs more than deadlocks and in my experience they're always more involved in debugging compared to deadlocks.
steveklabnik 11y ago
> But all Rust gives you today (and for the foreseeable future) is a pthreads > wrapper. ... Plus significant static guarantees about the safety of using said wrapper.
- acconsta 11y ago
  And that's a great achievement! If there were a Nobel prize for practical use of a type system, the Rust team would get it. But overselling it ("Rust goes out of its way to make it easy and safe to write multithreaded code" except oh yeah it can deadlock at any time) is a mistake.
  - kibwen 11y ago
    You act as though deadlocks in Rust are trivial and pervasive, but they aren't. It just makes no guarantees that deadlocks will not occur. Show me a language that statically prevents deadlocks and I'll be quite curious to check it out. In any case, writing concurrent code in Rust is safe and easy. I suggest you try it. :)
    - acconsta 11y ago
      >You act as though deadlocks in Rust are trivial and pervasive, but they aren't Writing nontrivial multithreaded code using thread and mutex primitives is very hard to get right (not only due to data races, but also race conditions and deadlocks). Has your experience been different? >Show me a language that statically prevents deadlocks OK, idiomatic Go and Erlang will never deadlock. Sure, you can use mutexes in Go, but unlike in Rust they aren't the only means of achieving high levels of concurrency (in fact, they are explicitly discouraged). I find this attitude from the Rust community disheartening. Not everything needs to be statically verified to be useful.
      - dbaupp 11y ago
        Both of those languages will semantically deadlock: forget to send a message on a channel and a different thread will sit there waiting forever. It's not a mutex-deadlock, but it achieves the same thing. Neither of those languages protects against this. (And Rust has exactly that level of "deadlock" freedom: it has channels too.) In any case, you seem to be ignoring what everyone is saying: Rust doesn't guarantee deadlock freedom, but it still tries to help. Mutexes can be an important building for some things, but they're not the final story. There's atomics and channels in the standard library right now, and now that 1.0 is released, there'll be a growth of even better abstractions. One of the people working full time on Rust has a PhD in concurrent programming, and has it as a personal goal to make Rust great at it. You can see his initial work on his blog[1], and read his thesis which introduces "reagents" (something he has expressed interest in implementing in Rust)[2]. 1.0 is the start for Rust, not the end. [1]: http://aturon.github.io/blog/2015/08/27/epoch/ http://aturon.github.io/blog/2015/08/27/epoch/ [2]: https://www.mpi-sws.org/~turon/turon-thesis.pdf https://www.mpi-sws.org/~turon/turon-thesis.pdf
kibwen 11y ago
> I think it's a fair characterization given 1.0 shipped > without them, and there doesn't seem to any timeline for > standardization (please correct me if I'm wrong). The release of 1.0 was not an indication that the language was 100% complete, or that the stdlib was 100% comprehensive. The 1.0 release represented a stable foundation upon which to build an ecosystem. Going forward, the Rust developers absolutely do care about providing more concurrency primitives. Here's Aaron Turon's recent work on implementing epoch-based memory reclamation for implementing lock-free data structures: http://aturon.github.io/blog/2015/08/27/epoch/ http://aturon.github.io/blog/2015/08/27/epoch/ . Of the two interns the Rust project was granted this summer, one of them spent the entirety of their time working on supporting SIMD in the language (http://huonw.github.io/blog/2015/08/simd-in-rust/ http://huonw.github.io/blog/2015/08/simd-in-rust/) while the other spent their time specifying the behaviors that are allowed inside Rust's `unsafe` blocks, which includes taking a good long stare at Rust's memory model and all its concurrency primitives and making sure that they're sound. While we're on the topic of interns, you're also overlooking the `Arc` pointer in the stdlib, which is an intern project from more than three years ago which allows one to safely share memory between threads. Meanwhile, pcwalton (Rust core team member and full-time Servo developer) has been working on shmem support and multiprocess allocators for Servo. In addition, the Rust stdlib contained an implementation of fork/join prior to 1.0, but at the last minute the API was found unsound for a few edge cases and so it was deprecated and a working implementation was deferred until later (today there are at least two working reimplementations of this on crates.io with the unsoundness fixed). > This apparently doesn't include preventing deadlocks, a > problem that is common, hard-to-avoid, and difficult to > recover from. Does that not strike you as a bit of an > overreach? Not at all. Statically preventing data races is an enormous leap forward in the state of the art. Here's a quote from Matthias Felleisen of Northeastern University on teaching Rust to students without experience in concurrent programming: https://www.youtube.com/watch?v=JBmIQIZPaHY&feature=youtu.be&t=2663 https://www.youtube.com/watch?v=JBmIQIZPaHY&feature=youtu.be... "I had them program parallel programs in Rust, and as some of you know, Rust prevents race conditions with its type checking. [Note that he's incorrect here, Rust prevents data races, not race conditions in general.] I will admit, the first two weeks, because we used the beta release, was a mess. We couldn't understand the type error messages. But once we got over the hump, I was blown away, that these kids never had problems writing parallel programs in imperative style. There were no race conditions. The type system slapped their fingers. We taught them how to design, the type system enforced it, and lo and behold, I hate to admit this but I have to admit it, they just didn't have problems with this stuff."
openasocket 11y ago
FYI, detecting deadlocks in the general case is in-computable, whether at run time or compile time.
- heinrich5991 11y ago
  Stuff like the borrow-checker in the general case is also not computable, only for a small subset of programs, e.g. Rust programs.

the_why_of_y 11y ago

> deadlocks, a problem that is common, hard-to-avoid, and difficult to recover from Deadlocks are by far the easiest concurrency problem to debug, since it's very obvious when your program is deadlocked, and in most cases a stack trace of the involved threads is sufficient to debug and fix a deadlock. Also, if your program is deadlocked it won't corrupt user data. Rust's std::sync::Mutex is fortunately non-recursive, which makes it easier to find deadlocks during testing. Data races are far worse since they may cause arbitrary effects at a later point in program execution, so they take a lot of developer time to track down and very likely lead to data corruption.

acconsta 11y ago
If deadlocks are so easy to fix why do they happen so often? Seriously, how many applications have you seen hang in your life? (not to discount the role of other race conditions in causing hangs, which the borrow checker also can't categorically prevent).
- GFK_of_xmaspast 11y ago
  Null pointer derefs are easy to find and fix and : yet.
  - neandrake 11y ago
    This is not always the case. Null pointer derefs are just indicators that the program got into an invalid state. Is the fix to provide some alternative pathway when the program got in a bad state or is it to prevent the bad state from happening in the first place? If the fix is to prevent the bad state, tracking down how that state occurred in the first place can often be time consuming in the case of data races.
- neandrake 11y ago
  I believe the comment reads that deadlocks are the "easiest concurrency problem" to debug -- but your question is moreso headed to the point that they are easy to introduce in a system. From the example, from a deadlock the locked threads and their stacktraces can be determined, which will help in pointing towards where the cause of the problem is. Compared to situations where the problem occurs due to a data-race causing an unexpected/invalid state but a problem doesn't manifest until later. Fixing this type of problem is more problematic as usually any exception/stacktrace that might crop up does not relate any information as to how the state became invalid in the first place. I tend to see these types of bugs more than deadlocks and in my experience they're always more involved in debugging compared to deadlocks.

steveklabnik 11y ago

> But all Rust gives you today (and for the foreseeable future) is a pthreads > wrapper. ... Plus significant static guarantees about the safety of using said wrapper.

acconsta 11y ago
And that's a great achievement! If there were a Nobel prize for practical use of a type system, the Rust team would get it. But overselling it ("Rust goes out of its way to make it easy and safe to write multithreaded code" except oh yeah it can deadlock at any time) is a mistake.
- kibwen 11y ago
  You act as though deadlocks in Rust are trivial and pervasive, but they aren't. It just makes no guarantees that deadlocks will not occur. Show me a language that statically prevents deadlocks and I'll be quite curious to check it out. In any case, writing concurrent code in Rust is safe and easy. I suggest you try it. :)
  - acconsta 11y ago
    >You act as though deadlocks in Rust are trivial and pervasive, but they aren't Writing nontrivial multithreaded code using thread and mutex primitives is very hard to get right (not only due to data races, but also race conditions and deadlocks). Has your experience been different? >Show me a language that statically prevents deadlocks OK, idiomatic Go and Erlang will never deadlock. Sure, you can use mutexes in Go, but unlike in Rust they aren't the only means of achieving high levels of concurrency (in fact, they are explicitly discouraged). I find this attitude from the Rust community disheartening. Not everything needs to be statically verified to be useful.
    - dbaupp 11y ago
      Both of those languages will semantically deadlock: forget to send a message on a channel and a different thread will sit there waiting forever. It's not a mutex-deadlock, but it achieves the same thing. Neither of those languages protects against this. (And Rust has exactly that level of "deadlock" freedom: it has channels too.) In any case, you seem to be ignoring what everyone is saying: Rust doesn't guarantee deadlock freedom, but it still tries to help. Mutexes can be an important building for some things, but they're not the final story. There's atomics and channels in the standard library right now, and now that 1.0 is released, there'll be a growth of even better abstractions. One of the people working full time on Rust has a PhD in concurrent programming, and has it as a personal goal to make Rust great at it. You can see his initial work on his blog[1], and read his thesis which introduces "reagents" (something he has expressed interest in implementing in Rust)[2]. 1.0 is the start for Rust, not the end. [1]: http://aturon.github.io/blog/2015/08/27/epoch/ http://aturon.github.io/blog/2015/08/27/epoch/ [2]: https://www.mpi-sws.org/~turon/turon-thesis.pdf https://www.mpi-sws.org/~turon/turon-thesis.pdf

kibwen 11y ago

> I think it's a fair characterization given 1.0 shipped > without them, and there doesn't seem to any timeline for > standardization (please correct me if I'm wrong). The release of 1.0 was not an indication that the language was 100% complete, or that the stdlib was 100% comprehensive. The 1.0 release represented a stable foundation upon which to build an ecosystem. Going forward, the Rust developers absolutely do care about providing more concurrency primitives. Here's Aaron Turon's recent work on implementing epoch-based memory reclamation for implementing lock-free data structures: http://aturon.github.io/blog/2015/08/27/epoch/ http://aturon.github.io/blog/2015/08/27/epoch/ . Of the two interns the Rust project was granted this summer, one of them spent the entirety of their time working on supporting SIMD in the language (http://huonw.github.io/blog/2015/08/simd-in-rust/ http://huonw.github.io/blog/2015/08/simd-in-rust/) while the other spent their time specifying the behaviors that are allowed inside Rust's `unsafe` blocks, which includes taking a good long stare at Rust's memory model and all its concurrency primitives and making sure that they're sound. While we're on the topic of interns, you're also overlooking the `Arc` pointer in the stdlib, which is an intern project from more than three years ago which allows one to safely share memory between threads. Meanwhile, pcwalton (Rust core team member and full-time Servo developer) has been working on shmem support and multiprocess allocators for Servo. In addition, the Rust stdlib contained an implementation of fork/join prior to 1.0, but at the last minute the API was found unsound for a few edge cases and so it was deprecated and a working implementation was deferred until later (today there are at least two working reimplementations of this on crates.io with the unsoundness fixed). > This apparently doesn't include preventing deadlocks, a > problem that is common, hard-to-avoid, and difficult to > recover from. Does that not strike you as a bit of an > overreach? Not at all. Statically preventing data races is an enormous leap forward in the state of the art. Here's a quote from Matthias Felleisen of Northeastern University on teaching Rust to students without experience in concurrent programming: https://www.youtube.com/watch?v=JBmIQIZPaHY&feature=youtu.be&t=2663 https://www.youtube.com/watch?v=JBmIQIZPaHY&feature=youtu.be... "I had them program parallel programs in Rust, and as some of you know, Rust prevents race conditions with its type checking. [Note that he's incorrect here, Rust prevents data races, not race conditions in general.] I will admit, the first two weeks, because we used the beta release, was a mess. We couldn't understand the type error messages. But once we got over the hump, I was blown away, that these kids never had problems writing parallel programs in imperative style. There were no race conditions. The type system slapped their fingers. We taught them how to design, the type system enforced it, and lo and behold, I hate to admit this but I have to admit it, they just didn't have problems with this stuff."

openasocket 11y ago

FYI, detecting deadlocks in the general case is in-computable, whether at run time or compile time.

heinrich5991 11y ago
Stuff like the borrow-checker in the general case is also not computable, only for a small subset of programs, e.g. Rust programs.