6 ms·

Intentionally or not, this post demonstrates one of the things that makes safer abstractions in C less desirable: the shared pointer implementation uses a POSIX

Intentionally or not, this post demonstrates one of the things that makes safer abstractions in C less desirable: the shared pointer implementation uses a POSIX mutex, which means it’s (1) not cross platform, and (2) pays the mutex overhead even in provably single-threaded contexts. In other words, it’s not a zero-cost abstraction. C++’s shared pointer has the same problem; Rust avoids it by having two types (Rc and Arc) that the developer can select from (and which the compiler will prevent you from using unsafely).

kouteiheika 1y ago
> the shared pointer implementation uses a POSIX mutex [...] C++’s shared pointer has the same problem It doesn't. C++'s shared pointers use atomics, just like Rust's Arc does. There's no good reason (unless you have some very exotic requirements, into which I won't get into here) to implement shared pointers with mutexes. The implementation in the blog post here is just suboptimal. (But it's true that C++ doesn't have Rust's equivalent of Rc, which means that if you just need a reference counted pointer then using std::shared_ptr is not a zero cost abstraction.)
- woodruffw 1y ago
  To be clear, the “same problem” is that it’s not a zero-cost abstraction, not that it uses the same specific suboptimal approach as this blog post.
  - kouteiheika 1y ago
    I think that's an orthogonal issue. It's not that C++'s shared pointer is not a zero cost abstraction (it's as much a zero cost abstraction as in Rust), but that it only provides one type of a shared pointer. But I suppose we're wasting time on useless nitpicking. So, fair enough.
    - woodruffw 1y ago
      I think they’re one and the same: C++ doesn’t have program-level thread safety by construction, so primitives like shared pointers need to be defensive by default instead of letting the user pick the right properties for their use case. Edit: in other words C++ could provide an equivalent of Rc, but we’d see no end of people complaining when they shoot themselves in the foot with it. (This is what “zero cost abstraction” means: it doesn’t mean no cost, just that the abstraction’s cost is no greater than the semantically equivalent version written by the user. So both Arc and shared_ptr are zero-cost in a MT setting, but only Rust has a zero-cost abstraction in a single-threaded setting.)
      - kouteiheika 1y ago
        I can't say I agree with this? If C++ had an Rc equivalent (or if you'd write one yourself) it would be just as zero cost as it is in Rust, both in a single-threaded setting and in a multithreaded-setting. "Zero cost abstraction" doesn't mean that it cannot be misused or that it doesn't have any cognitive overhead to use correctly, just that it matches whatever you'd write without the abstraction in place. Plenty of "zero cost" features in C++ still need to you pay attention to not accidentally blow you leg off. Simply put, just as a `unique_ptr` (`Box`) is an entirely different abstraction than `shared_ptr` (`Arc`), an `Rc` is also an entirely different abstraction than `Arc`, and C++ simply happens to completely lack `Rc` (at least in the standard; Boost of course has one). But if it had one you could use it with exactly the same cost as in Rust, you'd just have to manually make sure to not use it across threads (which indeed is easier said than done, which is why it's not in the standard), exactly the same as if you'd manually maintain the reference count without the nice(er) abstraction. Hence "zero cost abstraction".
        woodruffw 1y ago
        Sorry, I realized I’m mixing two things in a confusing way: you’re right that C++ could easily have a standard zero-cost Rc equivalent; I’m saying that it can’t have a safe one. I think this is relevant given the weight OP gives to both performance and safety.
      - SR2Z 1y ago
- cogman10 1y ago
  > very exotic requirements I'd be interested to know what you are thinking. The primary exotic thing I can imagine is an architecture lacking the ability to do atomic operations. But even in that case, C11 has atomic operations [1] built in. So worst case, the C library for the target architecture would likely boil down to mutex operations. [1] https://en.cppreference.com/w/c/atomic.html https://en.cppreference.com/w/c/atomic.html
  - kouteiheika 1y ago
    Well, basically, yeah, if your platform lacks support for atomics, or if you'd need some extra functionality around the shared pointer like e.g. logging the shared pointer refcounts while enforcing consistent ordering of logs (which can be useful if you're unfortunate enough to have to debug a race condition where you need to pay attention to refcounts, assuming the extra mutex won't make your heisenbug disappear), or synchronizing something else along with the refcount (basically a "fat", custom shared pointer that does more than just shared-pointering).
    - colonwqbang 1y ago
      Does there exist any platform which has multithreading but not atomics? Such a platform would be quite impractical as you can't really implement locks or any other threading primitive without atomics.
      - cogman10 1y ago
        Certainly such systems can pretty readily exist. You merely need atomic reads/writes in order to implement locks. You can't create userspace locks which is a bummer, but the OS has the capability of enforcing locks. That's basically how early locking worked. The main thing needed to make a correct lock is interrupt protection. Something every OS has. To go fast, you need atomic operations. It especially becomes important if you are dealing with multiple cores. However, for a single core system atomics aren't needed for the OS to create locks.
        1y ago
spacedcowboy 1y ago
The number of times I might want to write something in C and have it less likely to crash absolutely dwarfs the number of times I care about that code being cross-platform. Sure, cross-platform is desirable, if there's no cost involved, and mandatory if you actually need it, but it's a "nice to have" most of the time, not a "needs this". As for mutex overheads, yep, that's annoying, but really, how annoying ? Modern CPUs are fast. Very very fast. Personally I'm far more likely to use an os_unfair_lock_t than a pthread_mutex_t (see the previous point) which minimizes the locking to a memory barrier, but even if locking were slow, I think I'd prefer safe. Rust is, I'm sure, great. It's not something I'm personally interested in getting involved with, but it's not necessary for C (or even this extra header) to do everything that Rust can do, for it to be an improvement on what is available. There's simply too much out there written in C to say "just use Rust, or Swift, or ..." - too many libraries, too many resources, too many tutorials, etc. You pays your money and takes your choice.
- woodruffw 1y ago
  That’s all reasonable, but here’s one of the primary motivations from the post: > We love its raw speed, its direct connection to the metal If this is a strong motivating factor (versus, say, refactoring risk), then C’s lack of safe zero-cost abstractions is a valid concern.
- lelanthran 1y ago
  > As for mutex overheads, yep, that's annoying, but really, how annoying ? For this use-case, you might not notice. ISTR, when examing the pthreads source code for some platform, that mutexes only do a context switch as a fallback, if the lock cannot be acquired. So, for most use-cases of this header, you should not see any performance impact. You'll see some bloat, to be sure.
- lmm 1y ago
  > There's simply too much out there written in C to say "just use Rust, or Swift, or ..." - too many libraries, too many resources, too many tutorials, etc. There really isn't. Speaking as someone who works in JVM-land, you really can avoid C all the time if you're willing to actually try.
saurik 1y ago
I'd think a POSIX mutex--a standard API that I not only could implement anywhere, but which has already been implemented all over the place--is way more "cross platform" than use of atomics.
- woodruffw 1y ago
  To lift things up a level: I think a language’s abstractions have failed if we even need to have a conversation around what “cross platform” really means :-)
  - deleted 1y ago
    [deleted]
  - jhatemyjob 1y ago
    If that's the bar, what language's abstractions haven't failed?
    - ruined 1y ago
      wasm and lambda calculus
      - deleted 1y ago
        [deleted]
- wat10000 1y ago
  If you're targeting a vaguely modern C standard, atomics win by being part of the language. C11 has atomics and it's straightforward to use them to implement thread-safe reference counting.
lelanthran 1y ago
> Intentionally or not, this post demonstrates one of the things that makes safer abstractions in C less desirable: the shared pointer implementation uses a POSIX mutex, which means it’s (1) not cross platform, and (2) pays the mutex overhead even in provably single-threaded contexts. In other words, it’s not a zero-cost abstraction. It's an implementation detail. They could have used atomic load/store (since c11) to implement the increment/decrement. TBH I'm not sure what a mutex buys you in this situation (reference counting)
aidenn0 1y ago
> the shared pointer implementation uses a POSIX mutex Do you have a source for this? I couldn't find the implementation in TFA nor a link to safe_c.h
- deleted 1y ago
  [deleted]
accelbred 1y ago
Unfortunately, for C++, thats not true. At least with glibc and libstdc++, if you do not link with pthreads, then shared pointers are not thread-safe. At runtime it will do a symbol lookup for a pthreads symbol, and based off the result, the shared pointer code will either take the atomic or non-atomic path. I'd much rather it didnt try to be zero-cost and it always used atomics...
- woodruffw 1y ago
  This is, impressively, significantly worse than I realized!
- TuxSH 1y ago
  True, but that's a fault of the implementation, which assumes POSIX is the only thing in town & makes questionable optimization choices, rather that of the language itself (for reference, the person above is referring to what's described here: https://snf.github.io/2019/02/13/shared-ptr-optimization/ https://snf.github.io/2019/02/13/shared-ptr-optimization/)
  - wyldfire 1y ago
    > the language itself The "language" is conventionally thought of as the sum of the effects given by the { compiler + runtime libraries }. The "language" often specifies features that are implemented exclusively in target libraries, for example. You're correct to say that they're not "language features" but the two domains share a single label like "C++20" / "C11" - so unless you're designing the toolchain it's not as significant a difference. We're down to ~three compilers: gcc, clang, MSVC and three corresponding C++ libraries.
    - TuxSH 11mo ago
      I agree with what you said, however neither libc++ nor MS-STL have this "optimization" to my knowledge
- eddd-ddde 1y ago
  Why use atomics if you don't need them? There really should just be two different shared pointer types.
kev009 1y ago
C11 has a mutex API (threads.h), so why would it rely on POSIX? Are you sure it's not an runtime detail on one platform? https://devblogs.microsoft.com/cppblog/c11-threads-in-visual-studio-2022-version-17-8-preview-2/ https://devblogs.microsoft.com/cppblog/c11-threads-in-visual...
- loeg 1y ago
  The article has an excerpt using posix mutexes specifically. But you're right that C11 code can just portably use standard mutexes. // The old way of manual reference counting typedef struct { MatchStore* store; int ref_count; pthread_mutex_t mutex; } SharedStore;
layer8 1y ago
The shared-pointer implementation isn’t actually shown (i.e. shared_ptr_copy), and the SharedPtr type doesn’t use a pthread_mutex_t.
loeg 1y ago
Tecnhically the mutex refcounting example is shown as an example of the before the header the author is talking about. We don't know what they've chosen to implement shared_ptr with.
cryptonector 1y ago
Meh, it could easily use atomics instead, no lock needed.
kazinator 1y ago
ISO C has had mutexes since C11 I think. In any case, you could use the provided primitives to wrap the C11 mutex, or any other mutex. With some clever #ifdef, you can probably have a single or multithreaded build switch at compile time which makes all the mutex stuff do nothing.
up2isomorphism 1y ago
it is quite obvious which one is easier: type bunch of ifdefs vs learn a new language. BTW don’t fight C for portability, it is unlikely you will win.
nurettin 11mo ago
Rust pays the cumbersome lifetime syntax tax even in provably single threaded contexts. When will Rust develop ergonomics with better defaults and less boilerplate in such contexts?

kouteiheika 1y ago

> the shared pointer implementation uses a POSIX mutex [...] C++’s shared pointer has the same problem It doesn't. C++'s shared pointers use atomics, just like Rust's Arc does. There's no good reason (unless you have some very exotic requirements, into which I won't get into here) to implement shared pointers with mutexes. The implementation in the blog post here is just suboptimal. (But it's true that C++ doesn't have Rust's equivalent of Rc, which means that if you just need a reference counted pointer then using std::shared_ptr is not a zero cost abstraction.)

woodruffw 1y ago
To be clear, the “same problem” is that it’s not a zero-cost abstraction, not that it uses the same specific suboptimal approach as this blog post.
- kouteiheika 1y ago
  I think that's an orthogonal issue. It's not that C++'s shared pointer is not a zero cost abstraction (it's as much a zero cost abstraction as in Rust), but that it only provides one type of a shared pointer. But I suppose we're wasting time on useless nitpicking. So, fair enough.
  - woodruffw 1y ago
    I think they’re one and the same: C++ doesn’t have program-level thread safety by construction, so primitives like shared pointers need to be defensive by default instead of letting the user pick the right properties for their use case. Edit: in other words C++ could provide an equivalent of Rc, but we’d see no end of people complaining when they shoot themselves in the foot with it. (This is what “zero cost abstraction” means: it doesn’t mean no cost, just that the abstraction’s cost is no greater than the semantically equivalent version written by the user. So both Arc and shared_ptr are zero-cost in a MT setting, but only Rust has a zero-cost abstraction in a single-threaded setting.)
    - kouteiheika 1y ago
      I can't say I agree with this? If C++ had an Rc equivalent (or if you'd write one yourself) it would be just as zero cost as it is in Rust, both in a single-threaded setting and in a multithreaded-setting. "Zero cost abstraction" doesn't mean that it cannot be misused or that it doesn't have any cognitive overhead to use correctly, just that it matches whatever you'd write without the abstraction in place. Plenty of "zero cost" features in C++ still need to you pay attention to not accidentally blow you leg off. Simply put, just as a `unique_ptr` (`Box`) is an entirely different abstraction than `shared_ptr` (`Arc`), an `Rc` is also an entirely different abstraction than `Arc`, and C++ simply happens to completely lack `Rc` (at least in the standard; Boost of course has one). But if it had one you could use it with exactly the same cost as in Rust, you'd just have to manually make sure to not use it across threads (which indeed is easier said than done, which is why it's not in the standard), exactly the same as if you'd manually maintain the reference count without the nice(er) abstraction. Hence "zero cost abstraction".
      - woodruffw 1y ago
        Sorry, I realized I’m mixing two things in a confusing way: you’re right that C++ could easily have a standard zero-cost Rc equivalent; I’m saying that it can’t have a safe one. I think this is relevant given the weight OP gives to both performance and safety.
    - SR2Z 1y ago
cogman10 1y ago
> very exotic requirements I'd be interested to know what you are thinking. The primary exotic thing I can imagine is an architecture lacking the ability to do atomic operations. But even in that case, C11 has atomic operations [1] built in. So worst case, the C library for the target architecture would likely boil down to mutex operations. [1] https://en.cppreference.com/w/c/atomic.html https://en.cppreference.com/w/c/atomic.html
- kouteiheika 1y ago
  Well, basically, yeah, if your platform lacks support for atomics, or if you'd need some extra functionality around the shared pointer like e.g. logging the shared pointer refcounts while enforcing consistent ordering of logs (which can be useful if you're unfortunate enough to have to debug a race condition where you need to pay attention to refcounts, assuming the extra mutex won't make your heisenbug disappear), or synchronizing something else along with the refcount (basically a "fat", custom shared pointer that does more than just shared-pointering).
  - colonwqbang 1y ago
    Does there exist any platform which has multithreading but not atomics? Such a platform would be quite impractical as you can't really implement locks or any other threading primitive without atomics.
    - cogman10 1y ago
      Certainly such systems can pretty readily exist. You merely need atomic reads/writes in order to implement locks. You can't create userspace locks which is a bummer, but the OS has the capability of enforcing locks. That's basically how early locking worked. The main thing needed to make a correct lock is interrupt protection. Something every OS has. To go fast, you need atomic operations. It especially becomes important if you are dealing with multiple cores. However, for a single core system atomics aren't needed for the OS to create locks.
      - 1y ago

spacedcowboy 1y ago

The number of times I might want to write something in C and have it less likely to crash absolutely dwarfs the number of times I care about that code being cross-platform. Sure, cross-platform is desirable, if there's no cost involved, and mandatory if you actually need it, but it's a "nice to have" most of the time, not a "needs this". As for mutex overheads, yep, that's annoying, but really, how annoying ? Modern CPUs are fast. Very very fast. Personally I'm far more likely to use an os_unfair_lock_t than a pthread_mutex_t (see the previous point) which minimizes the locking to a memory barrier, but even if locking were slow, I think I'd prefer safe. Rust is, I'm sure, great. It's not something I'm personally interested in getting involved with, but it's not necessary for C (or even this extra header) to do everything that Rust can do, for it to be an improvement on what is available. There's simply too much out there written in C to say "just use Rust, or Swift, or ..." - too many libraries, too many resources, too many tutorials, etc. You pays your money and takes your choice.

woodruffw 1y ago
That’s all reasonable, but here’s one of the primary motivations from the post: > We love its raw speed, its direct connection to the metal If this is a strong motivating factor (versus, say, refactoring risk), then C’s lack of safe zero-cost abstractions is a valid concern.
lelanthran 1y ago
> As for mutex overheads, yep, that's annoying, but really, how annoying ? For this use-case, you might not notice. ISTR, when examing the pthreads source code for some platform, that mutexes only do a context switch as a fallback, if the lock cannot be acquired. So, for most use-cases of this header, you should not see any performance impact. You'll see some bloat, to be sure.
lmm 1y ago
> There's simply too much out there written in C to say "just use Rust, or Swift, or ..." - too many libraries, too many resources, too many tutorials, etc. There really isn't. Speaking as someone who works in JVM-land, you really can avoid C all the time if you're willing to actually try.

saurik 1y ago

I'd think a POSIX mutex--a standard API that I not only could implement anywhere, but which has already been implemented all over the place--is way more "cross platform" than use of atomics.

woodruffw 1y ago
To lift things up a level: I think a language’s abstractions have failed if we even need to have a conversation around what “cross platform” really means :-)
- deleted 1y ago
  [deleted]
- jhatemyjob 1y ago
  If that's the bar, what language's abstractions haven't failed?
  - ruined 1y ago
    wasm and lambda calculus
    - deleted 1y ago
      [deleted]
wat10000 1y ago
If you're targeting a vaguely modern C standard, atomics win by being part of the language. C11 has atomics and it's straightforward to use them to implement thread-safe reference counting.

lelanthran 1y ago

> Intentionally or not, this post demonstrates one of the things that makes safer abstractions in C less desirable: the shared pointer implementation uses a POSIX mutex, which means it’s (1) not cross platform, and (2) pays the mutex overhead even in provably single-threaded contexts. In other words, it’s not a zero-cost abstraction. It's an implementation detail. They could have used atomic load/store (since c11) to implement the increment/decrement. TBH I'm not sure what a mutex buys you in this situation (reference counting)

aidenn0 1y ago

> the shared pointer implementation uses a POSIX mutex Do you have a source for this? I couldn't find the implementation in TFA nor a link to safe_c.h

deleted 1y ago
[deleted]

accelbred 1y ago

Unfortunately, for C++, thats not true. At least with glibc and libstdc++, if you do not link with pthreads, then shared pointers are not thread-safe. At runtime it will do a symbol lookup for a pthreads symbol, and based off the result, the shared pointer code will either take the atomic or non-atomic path. I'd much rather it didnt try to be zero-cost and it always used atomics...

woodruffw 1y ago
This is, impressively, significantly worse than I realized!
TuxSH 1y ago
True, but that's a fault of the implementation, which assumes POSIX is the only thing in town & makes questionable optimization choices, rather that of the language itself (for reference, the person above is referring to what's described here: https://snf.github.io/2019/02/13/shared-ptr-optimization/ https://snf.github.io/2019/02/13/shared-ptr-optimization/)
- wyldfire 1y ago
  > the language itself The "language" is conventionally thought of as the sum of the effects given by the { compiler + runtime libraries }. The "language" often specifies features that are implemented exclusively in target libraries, for example. You're correct to say that they're not "language features" but the two domains share a single label like "C++20" / "C11" - so unless you're designing the toolchain it's not as significant a difference. We're down to ~three compilers: gcc, clang, MSVC and three corresponding C++ libraries.
  - TuxSH 11mo ago
    I agree with what you said, however neither libc++ nor MS-STL have this "optimization" to my knowledge
eddd-ddde 1y ago
Why use atomics if you don't need them? There really should just be two different shared pointer types.

kev009 1y ago

C11 has a mutex API (threads.h), so why would it rely on POSIX? Are you sure it's not an runtime detail on one platform? https://devblogs.microsoft.com/cppblog/c11-threads-in-visual-studio-2022-version-17-8-preview-2/ https://devblogs.microsoft.com/cppblog/c11-threads-in-visual...

loeg 1y ago
The article has an excerpt using posix mutexes specifically. But you're right that C11 code can just portably use standard mutexes. // The old way of manual reference counting typedef struct { MatchStore* store; int ref_count; pthread_mutex_t mutex; } SharedStore;

layer8 1y ago

The shared-pointer implementation isn’t actually shown (i.e. shared_ptr_copy), and the SharedPtr type doesn’t use a pthread_mutex_t.

loeg 1y ago

Tecnhically the mutex refcounting example is shown as an example of the before the header the author is talking about. We don't know what they've chosen to implement shared_ptr with.

cryptonector 1y ago

Meh, it could easily use atomics instead, no lock needed.

kazinator 1y ago

ISO C has had mutexes since C11 I think. In any case, you could use the provided primitives to wrap the C11 mutex, or any other mutex. With some clever #ifdef, you can probably have a single or multithreaded build switch at compile time which makes all the mutex stuff do nothing.

up2isomorphism 1y ago

it is quite obvious which one is easier: type bunch of ifdefs vs learn a new language. BTW don’t fight C for portability, it is unlikely you will win.

nurettin 11mo ago

Rust pays the cumbersome lifetime syntax tax even in provably single threaded contexts. When will Rust develop ergonomics with better defaults and less boilerplate in such contexts?