6 ms·

Shared_ptr<T>: the (not always) atomic reference counted smart pointer (2019)

51 points by klaussilveira 1y ago

TillE 1y ago
> VisualC++ doesn’t have its source code available Got all the way here and had to look back up to see this post was from 2019. The MSVC standard library has been open source for several years now. https://github.com/microsoft/STL https://github.com/microsoft/STL Though to be perfectly honest, setting a breakpoint and looking at the disassembly is probably easier than reading standard library code.
- tialaramex 1y ago
  As STL (nominative determinism at work) points out in the r/cpp thread about this, even when that git repo didn't exist you could have gone to see how this template works because C++ has to monomorphize generics somehow and that means when you write shared_ptr<goose> your C++ compiler needs to compile the source code for shared_ptr with the T replaced by goose. But you're correct, while I can read https://doc.rust-lang.org/src/alloc/sync.rs.html https://doc.rust-lang.org/src/alloc/sync.rs.html (where Rust's Arc is defined) ... ... good luck to me in https://github.com/microsoft/STL/blob/main/stl/inc/memory https://github.com/microsoft/STL/blob/main/stl/inc/memory There are tricks to cope with C++ macros not being hygienic, layered on top of tricks to cope with the fact C++ doesn't have ZSTs, tricks to reduce redundancy in writing all this out for related types, and hacks to improve compiler diagnostics when you do something especially stupid. Do its maintainers learn to read like this? I guess so, as it's Open Source.
  - chuckadams 1y ago
    It also helps that the Rust version is lavishly documented with examples, and the C++ version has barely any comments at all.
    - tialaramex 1y ago
      Fair, although slightly cheating because some of the examples in the Rust are literally documentation & that C++ isn't doing the same thing /// This is inline markdown documentation in Rust source,
- snfernandez 1y ago
  I was working at MS at the time and actually had access to the source code (my project involved devdiv). I don't remember the exact details, but I opted for not adding any of my "private" knowledge to the post. I agree with you that I prefer looking at optimized assembly with symbols rather than following code through files (which are usually filled with #ifdefs and macros).
mackman 1y ago
> It is possible to create threads by using the OS syscalls bypassing completely the requirement of pthead. (Un)fortunately, I couldn’t find any popular libraries that implement the functionality by using the syscall interface instead of relying on pthread. I have tried and failed to do this for a C++ program because the amount of C++ runtime static init/shutdown stuff you would need to deal with isn't practical to implement yourself.
ptspts 1y ago
The correct spelling is in lowercase: shared_ptr<T> . The title of the article is correct, the title of the HN post is incorrect.
- tialaramex 1y ago
  In hindsight this convention seems weird to me by the way. I didn't question it for the decades I was paid money to write C, but after adopting Rust it jumped out more that it's weird how monocase the C and C++ standard libraries are. Maybe there's a reason I'd never run into, but this seems like a missed opportunity. Even if I have no idea what Goose is, I can see it's a type, that seems like a win.
  - recursivecaveat 1y ago
    Yeah I don't know why so many C programmers ended up on a convention where case is entirely unused. I wonder if it's some ancient compatability thing that they have just been keeping self-consistent with forever. To me not using case is like designing a road map in black-and-white just because you didn't have any ideas for what colors should represent.
- Waterluvian 1y ago
  I believe HN does that automatically.
sesuximo 1y ago
Why is the atomic version slower? Is it slower on modern x86?
- eptcyka 1y ago
  Atomic write operations force a cache line flush and can wait until the memory is updated. Atomic reads have to be read from memory or a shared cache. Atomics are slow because memory is slow.
  - Krssst 1y ago
    I don't think an atomic operation necessarily demands a cache flush. L1 cache lines can move across cores as needed in my understanding (maybe not on multi-socket machines?). Barriers are required if further memory ordering guarantees are needed.
    - ot 1y ago
      Not a L1/L2/... cache flush, but a store buffer flush, at least on x86. This is true for LOCK instructions. Loads/stores (again on x86) are always acquire/release, so they don't need additional fences if you don't need seq-cst. However, seq-cst atomics in C++ lower stores to LOCK XCHG, so you get a fence.
      - tialaramex 1y ago
        There is no way the shared_ptr<T> is using the expensive sequentially consistent atomic operations. Even if you're one of the crazy people who thinks that's the sane default, the value from analysing and choosing a better ordering rule for this key type is enormous and when you do that analysis your answer is going to be acquire-release and only for some edge cases, in many places the relaxed atomic ordering is fine.
        loeg 1y ago
        > when you do that analysis your answer is going to be acquire-release and only for some edge cases, in many places the relaxed atomic ordering is fine. Why would shared_ptr refcounting need anything other than relaxed? Acq/rel are for implementing multi-variable atomic protocols, and shared_ptr refcounting simply doesn't have other variables.
TuxSH 1y ago
In particular, std::thread constructor's (the non-default one) has a workaround against LTO optimizing the pthread call away: https://github.com/gcc-mirror/gcc/blob/master/libstdc%2B%2B-v3/include/bits/std_thread.h#L155 https://github.com/gcc-mirror/gcc/blob/master/libstdc%2B%2B-... > Parallelism without pthread To get __atomic_add_dispatch to work, looks like one is expected to ensure pthread_create is referenced. One way to do it without creating a pthread or std::thread, is to do it outside LTO'd files, or like they did above. > > It is possible to create threads by using the OS syscalls bypassing completely the requirement of pthead As the other person said, it is impratical to do so, and it's easier to just reimplement gthread and pthread functions to be hooks (some toolchains do this).
manwe150 1y ago
> In conclusion, I’ll assume this is not a typical scenario and it is mostly safe. Ughh, this brings bad memories of the days I spent trying to diagnose why glibc often would give wrong answers for some users and not other users (they’ve since mitigated this problem slightly by combining pthreads and libdl into the same library). I wish they would get rid of this, since even the comment on it notes that the optimization is unsound (the ability to make syscalls directly, as used by go and others, makes this optimization potentially dangerous). It also upsets static analysis tools, since they see that glibc doesn’t appear to have the synchronization the library promises.
marthacamila 1y ago
My spouse has a million passwords on his phone which makes it highly out of bounds, I knew he was cheating on me and all I needed was some evidence. Medialord really made it a piece of cake catching him in the act, he helped me install some monitoring spyware that was so easy for me to use (as I am a computer dummy) all I had to do was login to see the info. He helped me through the whole process and now I have enough evidence for my divorce case. Am sure he can help you if you have similar issue, its as easy as sending him a mail HACKSECRETE@ GMAILCOM , you can also reach him if you could not withdraw your funds from online trading platform such as expert-option ,cal financial, Analyst , coinspot, Ctxprime and many more. he’s reliable and affordable.

51 points by klaussilveira 1y ago

> VisualC++ doesn’t have its source code available Got all the way here and had to look back up to see this post was from 2019. The MSVC standard library has been open source for several years now. https://github.com/microsoft/STL https://github.com/microsoft/STL Though to be perfectly honest, setting a breakpoint and looking at the disassembly is probably easier than reading standard library code.

tialaramex 1y ago
As STL (nominative determinism at work) points out in the r/cpp thread about this, even when that git repo didn't exist you could have gone to see how this template works because C++ has to monomorphize generics somehow and that means when you write shared_ptr<goose> your C++ compiler needs to compile the source code for shared_ptr with the T replaced by goose. But you're correct, while I can read https://doc.rust-lang.org/src/alloc/sync.rs.html https://doc.rust-lang.org/src/alloc/sync.rs.html (where Rust's Arc is defined) ... ... good luck to me in https://github.com/microsoft/STL/blob/main/stl/inc/memory https://github.com/microsoft/STL/blob/main/stl/inc/memory There are tricks to cope with C++ macros not being hygienic, layered on top of tricks to cope with the fact C++ doesn't have ZSTs, tricks to reduce redundancy in writing all this out for related types, and hacks to improve compiler diagnostics when you do something especially stupid. Do its maintainers learn to read like this? I guess so, as it's Open Source.
- chuckadams 1y ago
  It also helps that the Rust version is lavishly documented with examples, and the C++ version has barely any comments at all.
  - tialaramex 1y ago
    Fair, although slightly cheating because some of the examples in the Rust are literally documentation & that C++ isn't doing the same thing /// This is inline markdown documentation in Rust source,
snfernandez 1y ago
I was working at MS at the time and actually had access to the source code (my project involved devdiv). I don't remember the exact details, but I opted for not adding any of my "private" knowledge to the post. I agree with you that I prefer looking at optimized assembly with symbols rather than following code through files (which are usually filled with #ifdefs and macros).

mackman 1y ago

> It is possible to create threads by using the OS syscalls bypassing completely the requirement of pthead. (Un)fortunately, I couldn’t find any popular libraries that implement the functionality by using the syscall interface instead of relying on pthread. I have tried and failed to do this for a C++ program because the amount of C++ runtime static init/shutdown stuff you would need to deal with isn't practical to implement yourself.

ptspts 1y ago

The correct spelling is in lowercase: shared_ptr<T> . The title of the article is correct, the title of the HN post is incorrect.

tialaramex 1y ago
In hindsight this convention seems weird to me by the way. I didn't question it for the decades I was paid money to write C, but after adopting Rust it jumped out more that it's weird how monocase the C and C++ standard libraries are. Maybe there's a reason I'd never run into, but this seems like a missed opportunity. Even if I have no idea what Goose is, I can see it's a type, that seems like a win.
- recursivecaveat 1y ago
  Yeah I don't know why so many C programmers ended up on a convention where case is entirely unused. I wonder if it's some ancient compatability thing that they have just been keeping self-consistent with forever. To me not using case is like designing a road map in black-and-white just because you didn't have any ideas for what colors should represent.
Waterluvian 1y ago
I believe HN does that automatically.

sesuximo 1y ago

Why is the atomic version slower? Is it slower on modern x86?

eptcyka 1y ago
Atomic write operations force a cache line flush and can wait until the memory is updated. Atomic reads have to be read from memory or a shared cache. Atomics are slow because memory is slow.
- Krssst 1y ago
  I don't think an atomic operation necessarily demands a cache flush. L1 cache lines can move across cores as needed in my understanding (maybe not on multi-socket machines?). Barriers are required if further memory ordering guarantees are needed.
  - ot 1y ago
    Not a L1/L2/... cache flush, but a store buffer flush, at least on x86. This is true for LOCK instructions. Loads/stores (again on x86) are always acquire/release, so they don't need additional fences if you don't need seq-cst. However, seq-cst atomics in C++ lower stores to LOCK XCHG, so you get a fence.
    - tialaramex 1y ago
      There is no way the shared_ptr<T> is using the expensive sequentially consistent atomic operations. Even if you're one of the crazy people who thinks that's the sane default, the value from analysing and choosing a better ordering rule for this key type is enormous and when you do that analysis your answer is going to be acquire-release and only for some edge cases, in many places the relaxed atomic ordering is fine.
      - loeg 1y ago
        > when you do that analysis your answer is going to be acquire-release and only for some edge cases, in many places the relaxed atomic ordering is fine. Why would shared_ptr refcounting need anything other than relaxed? Acq/rel are for implementing multi-variable atomic protocols, and shared_ptr refcounting simply doesn't have other variables.

TuxSH 1y ago

In particular, std::thread constructor's (the non-default one) has a workaround against LTO optimizing the pthread call away: https://github.com/gcc-mirror/gcc/blob/master/libstdc%2B%2B-v3/include/bits/std_thread.h#L155 https://github.com/gcc-mirror/gcc/blob/master/libstdc%2B%2B-... > Parallelism without pthread To get __atomic_add_dispatch to work, looks like one is expected to ensure pthread_create is referenced. One way to do it without creating a pthread or std::thread, is to do it outside LTO'd files, or like they did above. > > It is possible to create threads by using the OS syscalls bypassing completely the requirement of pthead As the other person said, it is impratical to do so, and it's easier to just reimplement gthread and pthread functions to be hooks (some toolchains do this).

manwe150 1y ago

> In conclusion, I’ll assume this is not a typical scenario and it is mostly safe. Ughh, this brings bad memories of the days I spent trying to diagnose why glibc often would give wrong answers for some users and not other users (they’ve since mitigated this problem slightly by combining pthreads and libdl into the same library). I wish they would get rid of this, since even the comment on it notes that the optimization is unsound (the ability to make syscalls directly, as used by go and others, makes this optimization potentially dangerous). It also upsets static analysis tools, since they see that glibc doesn’t appear to have the synchronization the library promises.

marthacamila 1y ago

My spouse has a million passwords on his phone which makes it highly out of bounds, I knew he was cheating on me and all I needed was some evidence. Medialord really made it a piece of cake catching him in the act, he helped me install some monitoring spyware that was so easy for me to use (as I am a computer dummy) all I had to do was login to see the info. He helped me through the whole process and now I have enough evidence for my divorce case. Am sure he can help you if you have similar issue, its as easy as sending him a mail HACKSECRETE@ GMAILCOM , you can also reach him if you could not withdraw your funds from online trading platform such as expert-option ,cal financial, Analyst , coinspot, Ctxprime and many more. he’s reliable and affordable.