5 ms·

This is why pthread_mutex_lock() and pthread_mutex_unlock() will always be kings - it is clear when things are locked and when they are unlocked. Nobody needs t

This is why pthread_mutex_lock() and pthread_mutex_unlock() will always be kings - it is clear when things are locked and when they are unlocked. Nobody needs to write articles warning you about pthread_mutex being taken randomly by syntactic sugar and not released till a magic later time due to the same sugar. Ditto for RWlock/XYZloc/ABClock/etc...

akira2501 2y ago
I see it as a consequence of Rust being too liberal with additions to the language. It's hard to maintain some of these guarantees in the presence of certain syntactic forms. This happens in all languages. It is a little hilarious to see a rustaceon write: > and you can sus out if a program is going to cause a deadlock by just making sure you aren't acquiring multiple simultaneous locks. Ah.. not a problem! You just have to not ever have the problem. Then, of course at the opposite end of the article: > I wrote this block because this has specifically bitten different Rust crates in the wasmCloud project multiple times :D
- deleted 2y ago
  [deleted]
- dathinab 2y ago
  The issue here isn't new features at all. `if let` is more or less syntax sugar for `match`. And match behaves like that, too and has been in rust since 1.0. That match did behaves like that is due to some old, you could say legacy, reasons and had been criticized even in the early rust 1.x days. But changing a behavior which subtle change when locks are released is not something you can easily fix with a rust edition so we are pretty much stuck with it.
  - Philpax 2y ago
    Funny you say that: https://github.com/rust-lang/rust/issues/131154 https://github.com/rust-lang/rust/issues/131154 Looks like this is getting fixed in Rust 2024 :)
    - dathinab 2y ago
      I haven't been supper up to date with changes to rust-lang recently (I had seen that they are stabilizing if_let_rescope but didn't look into what it is, to much to do at work). Through making if-let less syntax sugar for changing a implicit behavior people most likely didn't rely on but have problems with seems like a very good idea My comment about this being hard to change was mainly about match. Not considering the option of making if-let less syntax sugary. And in difference to if-let, for match people do (or at least did years ago in production code) rely on it.
  - akira2501 2y ago
    > `if let` is more or less syntax sugar for `match`. Which are new features for systems languages that otherwise rely on RAII for locking. So it's in the class of "original sin." > so we are pretty much stuck with it. You could refuse to compile it under some set of flags. Isn't that the basic value premise of the language here?
    - sunshowers 2y ago
      It is true that Rust is the first major language with both pervasive, deterministic RAII and pattern matching. Kind of wild to be honest! A real marriage of systems and functional code.
  - dathinab 2y ago
    To elaborate a bit more this is a consequence of a overlapping of two design decisions: 1) temporaries being added "alongside" the item they appear in. This makes a tone of thing much much simpler, but comes back to bites us here. 2) "alongside" for match statement meaning alongside the whole statement (but for `if <cond> {` it's alongside the condition) 3) things being always dropped at the very end of the scope if not moved out from it earlier, which again makes things easier to understand in most cases. both had been discussed a bunch around 1.0/early 1.x days and both are things which in most situations make it easier to write rust code (and for beginners potentially much easier) but both have also drawbacks like the not-that-common example in this blog or e.g. in async where rust has to keep any values which impl Drop around across async await calls as it can't know if there is a side effect in Drop In the past I personally had been a contender of allowing the compiler to drop value anywhere between the last time they have been referenced and the end of the scopes without any rules or stability about where exactly (i.e. if you need a guard to be kept around you need to be explicit about it). But working more together with people of very varying skill levels in the last 5/6 years made me change my mind and agree that that would have been a terrible idea. And having rules about guaranteed drops as early as possible seem initially easy but aren't due to things like conditional moves, partial moves etc. I.e. it would still be quite a bit more complicated to teach it. Furthermore in both alternatives to 3) you likely still wouldn't (guaranteed) drop the guard temporary in the other match branch before you requesting the new guard as guaranteeing compiler behavior like that means having a lot of additional edges in many partial move scenarios and potentially even a bunch of additional branch. In both cases it would likely increase code size and mess with the branch predictor and I-caches and be generally just not good (but it would help with async await boundaries).
deleted 2y ago
[deleted]
jvanderbot 2y ago
In rust, this is y=x.lock(), and later drop(y) or just end the scope block. Sugary syntax is always an issue when side effects matter. Having said that, I adore procedural code in general, but it does make ownership a fun mental exercise.
CJefferson 2y ago
There are however many PRs about people ignoring the return values of pthread_mutex_lock(), or missing an unlock on one route through a function.
- dathinab 2y ago
  and nothing prevents you from explicitly dropping lock guards in rust either actually it's a common pattern in complicated multi threaded code or similar to require that
  - freeone3000 2y ago
    Dropping a lock guard in rust requires you to have already dropped the borrow — the shared memory under lock is only accessible through the guard, and you can’t drop an object if you have a reference to it. This is not true in C, where you can reference the shared memory after the lock is released.
    - dathinab 2y ago
      > This is not true in C, where you can reference the shared memory after the lock is released. technically yes but practically this is nearly always programmer bug to bypass a lock
      - freeone3000 2y ago
        That is the point — this is (another) example of a programmer bug that C allows that Rust does not.
  - lodovic 2y ago
    The rust compiler (or the rust-analyzer) directs developers to use mutexes or other guards, even in single-threaded code when the use case wouldn't strictly require it. Where we would get undefined behavior in C, we get deadlocks in rust. This forces developers to handle these cases, but this can be quite painful to implement, especially in embedded or no_std environments.
sunshowers 2y ago
I think mutexes that own their data are genuinely much easier to reason about than mutexes that don't. In this case this is a Rust bug that will be fixed in the next edition: https://github.com/rust-lang/rust/issues/124085 https://github.com/rust-lang/rust/issues/124085
- ajross 2y ago
  That's arguably true, but only insofar as it makes sense to talk about a mutex "owning" "data". There are many kinds of data that can't be owned by the language runtime like this (think about async/shared/DMA buffers, register blocks on foreign hardware, memory-mapped database files), but that still clearly need synchronization. Even things like coarse-grained locks taken at the subsystem level (e.g. to avoid having a separate atomic in every little object) are a bad fit for this kind of API. There's value to being opinionated about API design, but there's also cost. And Rust is supposed to be playing in the same sandbox as the lower level tools.
  - nindalf 2y ago
    Here’s what I understand from your comment, correct me if I’m wrong. You’re saying there is a runtime cost associated with a mutex/rwlock owning the data. And secondly, you think it isn’t possible to implement this pattern in other areas that need locking. I don’t think either of those is true. Encoding the ownership in the type system makes things clearer, imposes a compile-time cost but not run-time cost. Also, there isn’t any magic in the stdlib implementation of Mutex and Rwlock other than implementing it natively for every OS. This means that it is possible to implement the pattern for the examples you gave.
  - zacmps 2y ago
    You can also use a mutex that doesn't own data in Rust if you need to.
  - sunshowers 2y ago
    That is true, but I don't believe Rust imposes a cost here. The worst case degrades to a `Mutex<()>` and then whatever's being guarded being managed separately.
vlovich123 2y ago
This really reads like satire but feels like the author is actually being sincere. I’ve seen so many deadlocks happening due to pthread mutexes. Certainly way more than this syntactic corner case. It’s so common in fact that I saw a codebase that made their mutexes recursive to avoid having to think about it.
- dmitrygr 2y ago
  [flagged]
  - vlovich123 2y ago
    Who hurt you?
  - techbrovanguard 2y ago
    this is pretty obvious bait
  - kelnos 2y ago
    Deadlocks are a sign of normal humans getting tripped up by crappy abstractions. If you're going to claim you've never written a deadlock in your life, well... I'm not sure I'd believe that.
echelon 2y ago
No way. The compiler doesn't help you. Plus you're working in a memory unsafe, concurrency unsafe language.
kelnos 2y ago
No thanks; it's so so so easy to forget to unlock a mutex; I'd rather the language and stdlib just make it impossible for me to do so. Ultimately this isn't really a problem with how Rust's Mutex/RwLock/etc. works, it's just a poor choice with how the lifetime of the lock guard is figured in the 'if let' case. This poor choice will be fixed in the 2024 Rust edition, and this problem will go away.
- nikki93 2y ago
  I think a 'linear types' ish model where the compiler flags an error if you didn't write the explicit unlock call, and only compiles if some acceptable unlock call (or set of calls) is added, would be a good design. It can / would also prevent use-after-consume. I do want the static checks, but I don't think that means that implicit function calls need to be generated on a } or a = (assigning to a variable causes a drop) etc. This is what Rust already does with `.clone()` -- needing the explicit call -- and I think it makes sense for a lot of cases of implicit drop. I have seen discussions about implementing this for Rust and I have it implemented in a C borrow checker experiment I'm trying. ATS is also an example of this, and going further is something like Frama-C or seL4 and so on. The main point being: the implicitness is not necessary for "compiler makes sure you don't forget". So the original comment about how usage of the explicitly named and paired APIs can clarify intent both for the writer and reader can still stand while not implying that forgetting is involved. I see this dichotomy often being drawn and I think it's important to consider the language design space more granularly. I think a reminder is a better fix for forgetting, rather than assuming what you wanted and doing it for you. (the explicit calls also let you / work better with "go to defintion" on the editor to see their code, learn what to look up in a manual, step in with a debugger, see reasonable function names in stack traces and profiles, pass / require more arguments to the drop call, let the drop call have return / error values you can do something about (consider `fclose`), let the drop call be async, ...)
- tucnak 2y ago
  https://gaultier.github.io/blog/perhaps_rust_needs_defer.html https://gaultier.github.io/blog/perhaps_rust_needs_defer.htm...

akira2501 2y ago

I see it as a consequence of Rust being too liberal with additions to the language. It's hard to maintain some of these guarantees in the presence of certain syntactic forms. This happens in all languages. It is a little hilarious to see a rustaceon write: > and you can sus out if a program is going to cause a deadlock by just making sure you aren't acquiring multiple simultaneous locks. Ah.. not a problem! You just have to not ever have the problem. Then, of course at the opposite end of the article: > I wrote this block because this has specifically bitten different Rust crates in the wasmCloud project multiple times :D

deleted 2y ago
[deleted]
dathinab 2y ago
The issue here isn't new features at all. `if let` is more or less syntax sugar for `match`. And match behaves like that, too and has been in rust since 1.0. That match did behaves like that is due to some old, you could say legacy, reasons and had been criticized even in the early rust 1.x days. But changing a behavior which subtle change when locks are released is not something you can easily fix with a rust edition so we are pretty much stuck with it.
- Philpax 2y ago
  Funny you say that: https://github.com/rust-lang/rust/issues/131154 https://github.com/rust-lang/rust/issues/131154 Looks like this is getting fixed in Rust 2024 :)
  - dathinab 2y ago
    I haven't been supper up to date with changes to rust-lang recently (I had seen that they are stabilizing if_let_rescope but didn't look into what it is, to much to do at work). Through making if-let less syntax sugar for changing a implicit behavior people most likely didn't rely on but have problems with seems like a very good idea My comment about this being hard to change was mainly about match. Not considering the option of making if-let less syntax sugary. And in difference to if-let, for match people do (or at least did years ago in production code) rely on it.
- akira2501 2y ago
  > `if let` is more or less syntax sugar for `match`. Which are new features for systems languages that otherwise rely on RAII for locking. So it's in the class of "original sin." > so we are pretty much stuck with it. You could refuse to compile it under some set of flags. Isn't that the basic value premise of the language here?
  - sunshowers 2y ago
    It is true that Rust is the first major language with both pervasive, deterministic RAII and pattern matching. Kind of wild to be honest! A real marriage of systems and functional code.
- dathinab 2y ago
  To elaborate a bit more this is a consequence of a overlapping of two design decisions: 1) temporaries being added "alongside" the item they appear in. This makes a tone of thing much much simpler, but comes back to bites us here. 2) "alongside" for match statement meaning alongside the whole statement (but for `if <cond> {` it's alongside the condition) 3) things being always dropped at the very end of the scope if not moved out from it earlier, which again makes things easier to understand in most cases. both had been discussed a bunch around 1.0/early 1.x days and both are things which in most situations make it easier to write rust code (and for beginners potentially much easier) but both have also drawbacks like the not-that-common example in this blog or e.g. in async where rust has to keep any values which impl Drop around across async await calls as it can't know if there is a side effect in Drop In the past I personally had been a contender of allowing the compiler to drop value anywhere between the last time they have been referenced and the end of the scopes without any rules or stability about where exactly (i.e. if you need a guard to be kept around you need to be explicit about it). But working more together with people of very varying skill levels in the last 5/6 years made me change my mind and agree that that would have been a terrible idea. And having rules about guaranteed drops as early as possible seem initially easy but aren't due to things like conditional moves, partial moves etc. I.e. it would still be quite a bit more complicated to teach it. Furthermore in both alternatives to 3) you likely still wouldn't (guaranteed) drop the guard temporary in the other match branch before you requesting the new guard as guaranteeing compiler behavior like that means having a lot of additional edges in many partial move scenarios and potentially even a bunch of additional branch. In both cases it would likely increase code size and mess with the branch predictor and I-caches and be generally just not good (but it would help with async await boundaries).

deleted 2y ago

[deleted]

jvanderbot 2y ago

In rust, this is y=x.lock(), and later drop(y) or just end the scope block. Sugary syntax is always an issue when side effects matter. Having said that, I adore procedural code in general, but it does make ownership a fun mental exercise.

CJefferson 2y ago

There are however many PRs about people ignoring the return values of pthread_mutex_lock(), or missing an unlock on one route through a function.

dathinab 2y ago
and nothing prevents you from explicitly dropping lock guards in rust either actually it's a common pattern in complicated multi threaded code or similar to require that
- freeone3000 2y ago
  Dropping a lock guard in rust requires you to have already dropped the borrow — the shared memory under lock is only accessible through the guard, and you can’t drop an object if you have a reference to it. This is not true in C, where you can reference the shared memory after the lock is released.
  - dathinab 2y ago
    > This is not true in C, where you can reference the shared memory after the lock is released. technically yes but practically this is nearly always programmer bug to bypass a lock
    - freeone3000 2y ago
      That is the point — this is (another) example of a programmer bug that C allows that Rust does not.
- lodovic 2y ago
  The rust compiler (or the rust-analyzer) directs developers to use mutexes or other guards, even in single-threaded code when the use case wouldn't strictly require it. Where we would get undefined behavior in C, we get deadlocks in rust. This forces developers to handle these cases, but this can be quite painful to implement, especially in embedded or no_std environments.

sunshowers 2y ago

I think mutexes that own their data are genuinely much easier to reason about than mutexes that don't. In this case this is a Rust bug that will be fixed in the next edition: https://github.com/rust-lang/rust/issues/124085 https://github.com/rust-lang/rust/issues/124085

ajross 2y ago
That's arguably true, but only insofar as it makes sense to talk about a mutex "owning" "data". There are many kinds of data that can't be owned by the language runtime like this (think about async/shared/DMA buffers, register blocks on foreign hardware, memory-mapped database files), but that still clearly need synchronization. Even things like coarse-grained locks taken at the subsystem level (e.g. to avoid having a separate atomic in every little object) are a bad fit for this kind of API. There's value to being opinionated about API design, but there's also cost. And Rust is supposed to be playing in the same sandbox as the lower level tools.
- nindalf 2y ago
  Here’s what I understand from your comment, correct me if I’m wrong. You’re saying there is a runtime cost associated with a mutex/rwlock owning the data. And secondly, you think it isn’t possible to implement this pattern in other areas that need locking. I don’t think either of those is true. Encoding the ownership in the type system makes things clearer, imposes a compile-time cost but not run-time cost. Also, there isn’t any magic in the stdlib implementation of Mutex and Rwlock other than implementing it natively for every OS. This means that it is possible to implement the pattern for the examples you gave.
- zacmps 2y ago
  You can also use a mutex that doesn't own data in Rust if you need to.
- sunshowers 2y ago
  That is true, but I don't believe Rust imposes a cost here. The worst case degrades to a `Mutex<()>` and then whatever's being guarded being managed separately.

vlovich123 2y ago

This really reads like satire but feels like the author is actually being sincere. I’ve seen so many deadlocks happening due to pthread mutexes. Certainly way more than this syntactic corner case. It’s so common in fact that I saw a codebase that made their mutexes recursive to avoid having to think about it.

dmitrygr 2y ago
[flagged]
- vlovich123 2y ago
  Who hurt you?
- techbrovanguard 2y ago
  this is pretty obvious bait
- kelnos 2y ago
  Deadlocks are a sign of normal humans getting tripped up by crappy abstractions. If you're going to claim you've never written a deadlock in your life, well... I'm not sure I'd believe that.

echelon 2y ago

No way. The compiler doesn't help you. Plus you're working in a memory unsafe, concurrency unsafe language.

kelnos 2y ago

No thanks; it's so so so easy to forget to unlock a mutex; I'd rather the language and stdlib just make it impossible for me to do so. Ultimately this isn't really a problem with how Rust's Mutex/RwLock/etc. works, it's just a poor choice with how the lifetime of the lock guard is figured in the 'if let' case. This poor choice will be fixed in the 2024 Rust edition, and this problem will go away.

nikki93 2y ago
I think a 'linear types' ish model where the compiler flags an error if you didn't write the explicit unlock call, and only compiles if some acceptable unlock call (or set of calls) is added, would be a good design. It can / would also prevent use-after-consume. I do want the static checks, but I don't think that means that implicit function calls need to be generated on a } or a = (assigning to a variable causes a drop) etc. This is what Rust already does with `.clone()` -- needing the explicit call -- and I think it makes sense for a lot of cases of implicit drop. I have seen discussions about implementing this for Rust and I have it implemented in a C borrow checker experiment I'm trying. ATS is also an example of this, and going further is something like Frama-C or seL4 and so on. The main point being: the implicitness is not necessary for "compiler makes sure you don't forget". So the original comment about how usage of the explicitly named and paired APIs can clarify intent both for the writer and reader can still stand while not implying that forgetting is involved. I see this dichotomy often being drawn and I think it's important to consider the language design space more granularly. I think a reminder is a better fix for forgetting, rather than assuming what you wanted and doing it for you. (the explicit calls also let you / work better with "go to defintion" on the editor to see their code, learn what to look up in a manual, step in with a debugger, see reasonable function names in stack traces and profiles, pass / require more arguments to the drop call, let the drop call have return / error values you can do something about (consider `fclose`), let the drop call be async, ...)
tucnak 2y ago
https://gaultier.github.io/blog/perhaps_rust_needs_defer.html https://gaultier.github.io/blog/perhaps_rust_needs_defer.htm...