8 ms·

Implementing a class with void*

62 points by azhenley 5y ago

sillycross 5y ago
This is a well-known way to achieve separation of definition and implementation. The disadvantage of the method proposed in the article is that, now everything needs a pointer indirection. A solution that fixes the drawback is used by lz4's library implementation: instead of storing a void star, store a char[] array of same size as the real struct. (of course, now you have to manually make sure the struct size are in sync. It's more error prone, but still not that bad).
- guerrilla 5y ago
  Why not `typedef struct state state_t` in the header file, then have a `state_t state` in the header file with the `struct state { ... }` definition in the source file? This is similar to what I do in C, I don't see why it wouldn't work in C++.
  - kyralis 5y ago
    The module that's importing the header needs to know the state size. To do that, it either needs to see the struct declaration or be given the explicit size with the array approach.
  - sgerenser 5y ago
    That would fix the type safety issue of using a void* but not the extra pointer indirection issue, since your state struct would still have to be a pointer if you want the declaration outside of the header.
- deleted 5y ago
  [deleted]
- temac 5y ago
  Or don't do that (the char array trick) in C++ because implementers and standardizers are not clear about when/if you are allowed to store other objects in an array of chars, and even if you are it is tricky because you need to manually manage the alignment, or they are attempting to replace char with std::byte in the long term but don't really have a comprehensive and detailed plan to do so, etc. The implementation should probably provide you with std::aligned_storage which may involve some magic to handle some of those concerns, although merely the easiest ones, so I would say probably still don't use that either, unless you are already quite a C++ expert and/or are prepared to dig into the standard with no clear response about what you are attempting to do is even formally possible (the implementers/standardizers do not even know some things they make impossible for quite a long time, see for example the insanity of std::launder, or if you want to loose your mind forever the semantic of pointer provenance analysis that compilers are maybe already using to "optimize" but that what the semantic should even be is still being debated.)
  - sillycross 5y ago
    Yes, strict aliasing (or type-based alias analysis?) is quite crazy, and there are some murky dark corners where the specification is different between C and C++.. I think they have a std::launder thing exactly for this purpose of "safely casting an array of bytes into an object". However, in this particular case (of using char array to hide real implementation), the implementation resides in another translation unit, so I don't think anything is going to break if LTO is not enabled. With LTO I have no idea..
    - temac 5y ago
      For projects of mixed quality without basically an unbounded workforce maintaining them (who could investigate rare/arcane bugs "introduced" by the "optimizers" in some builds), and/or using "tricks", I too am fond of not using LTO. But then I force myself to find a second reason for why the program will run correctly, and unfortunately nowadays it is more and more being strictly-conforming. Relearning std::launder, TBAA, pointer provenance, etc. every time is way too consuming. I'm forced to give-up on programmer optimization and hope for the compiler to be really up to its mythical promises (and this yet: without LTO; too dangerous...)
- josefx 5y ago
  > of course, now you have to manually make sure the struct size are in sync. Can't you just use a static assert in the implementation file?
- hoseja 5y ago
  That breaks the ABI-compatibility part of this trick.
  - sillycross 5y ago
    Yes, it's a tradeoff. The upside is now you can put the object on the stack without invoking memory allocators.
prewett 5y ago
I'm surprised he doesn't refer to it by name (Pimpl). In C++ it has the advantage that you can change your implementation and retain binary compatibility because the size of your class doesn't change as you add/remove member variables (and maybe more surprisingly, add/removing virtual functions). If you're going to use C++, seems like it would be better to avoid the void* by class C { protected: struct Impl; std::unique_ptr<Impl> mImpl; }; And then define C::Impl in the .cpp file.
- vlovich123 5y ago
  The downside of unique_ptr used this way is that you have to define ~C in the same translation unit that the declaration of C::Impl lives.
  - gerdesj 5y ago
    No it isn't. What is a downside is you could not be arsed to explain why (blah means blah) to a reasonably intelligent audience.
  - ridiculous_fish 5y ago
    You can use a custom deleter, e.g. struct deleter_t { void operator()(impl_t *); }; using impl_unique_ptr_t = std::unique_ptr<impl_t, deleter_t>; You can place the implementation of the deleter alongside the impl.
    - deleted 5y ago
      [deleted]
    - cma 5y ago
      That breaks make_unique. Maybe unique_ptr inside an std::any? Error-prone casting, but managed lifetime. Though make_unique is maybe obsolete now that parameter evaluation order is better defined.
  - leni536 5y ago
    You have to do that anyway, unique_ptr only makes forgetting that a compile error.
zokier 5y ago
Qt uses pimpl extensively to maintain binary compatibility: https://wiki.qt.io/D-Pointer https://wiki.qt.io/D-Pointer
ttul 5y ago
Back in the 1990s, the video game company I worked at would hang a couple of extra void pointers in every class just to, you know, store extra things that are needed as time goes on…
- dnautics 5y ago
  This was common in the 90s, BeOS had these everywhere in its API classes
  - nielsbot 5y ago
    This is because of the fragile ABI problem in C++--it's not good for APIs for this reason. I used to program for the macOS kernel and they (used to) do exactly the same thing. See `OSMetaClassDefineReservedUnused` in `OSObject.cpp`, here: https://github.com/apple/darwin-xnu/blob/main/libkern/c++/OSObject.cpp https://github.com/apple/darwin-xnu/blob/main/libkern/c++/OS...
kazinator 5y ago
> This is not an "industry-standard" way to program in C++. It absolutely is a de facto industry standard way to program C++, and has a name: PIMPL (Pointer to IMPLementation). It has that name, because it's famous. It's probably less fashionable in newer code bases; probably someone whose head is up in C++20 will probably scoff at this, and certainly at any version where the secret is hidden by void *. It provides a good way to wrap C API's in C++. And, speaking of that, the technique is basically the spiritual equivalent of what happens in many a FFI module in languages other than C++ too, where some C handle is represented as an opaque foreign pointer, which is wrapped in some object native to the language.
- cjensen 5y ago
  It is not an industry standard way. It is not a sensible way. It is not even a C way. As others have said, there's no reason whatsoever to use void* here. Just declare the struct without defining it in the header. Then only define it in the C file. C is fine with pointers to structs which are merely declared but not defined.
  - kazinator 5y ago
    It is a de facto standard in that it is fairly widely documented and actually done. It's not in any ISO or IEEE standard or anything of the sort. There is almost no reason whatsoever to use void * anywhere other than to write a declaration that is compatible with another one which uses it. (A pointer to any object is better implemented as a typedef for unsigned char *. This requires a cast in both directions, thus it is safer. At the same time, it is more convenient when you actually want to work with the memory as such: you have bytewise arithmetic and dereferencing.) > C is fine with pointers to structs which are merely declared but not defined. Particularly if the secret object is a struct/class then this certainly improves the code (and when the secret isn't such a thing, it can probably be made into one: e.g. a secret array of integers can probably just be a a struct containing an array, possibly a flexible one). Numerous unsafe casts are thereby eliminated. But it makes no material difference to its organization or semantics. It's still the same PIMPL pattern.
srwest 5y ago
I haven't thought about him in years, but James Plank is the reason I'm a programmer today. I was a terrible student and he likely doesn't remember me, but his classes were too much fun. He just assigned projects and a timeline and live coded similar exercises in class mostly along with the timeline. I probably would have switched away from Computer Engineering (a degree I barely use) without his classes.
deleted 5y ago
[deleted]
not2b 5y ago
Yes, as others have said, using an opaque class is preferable to using a void pointer. class Impl; This isn't just for hiding, it also gives you faster compilation.
- hvdijk 5y ago
  > This isn't just for hiding, it also gives you faster compilation. That seems unlikely. What are you basing this on?
  - int_19h 5y ago
    It means that the header that declares class A doesn't have to #include the header that declares class B, unless B is a part of its public interface. It doesn't sound like much, but the implementation dependency chain can be much longer in practice; and more importantly, all those savings apply to every translation unit that includes the header.
    - hvdijk 5y ago
      Ah, that's what you meant. Your previous comment reads as if an opaque class results in faster compilation than a void pointer, but those are the same as far as that goes, and you were comparing both to a non-pimpl version.
      - chippiewill 5y ago
        There might be a few hundred CPU cycles saved in compilation from not having to static cast the void* pointer to the implementation class.
        hvdijk 5y ago
        Haha, sure, and the other way around, time is spent constructing the compile-time type info for the pimpl class. Let's ignore both since the impact would be so small it would almost certainly be drowned out by all the random noise that happens all the time.
pjmlp 5y ago
Besides what everyone else was written regarding pimpl, with c++ modules the only reason to use PIMPL is to ensure ABI stability.
ncmncm 5y ago
It seems kind of stupid to use "-std=c++98" when building this stuff. Nothing in later Standards interferes with it. Maybe it is meant to serve notice that the method is archaic, and that better ways to achieve the same thing will be presented later. But then one might just as well code the throwaway examples in C and use a damn C compiler. It's archaic even for C. There is just no merit in storing a void* that will then be cast to some arbitrary other known type every time it is used. The only really valid use for a void pointer is when it is being passed through a subsystem and comes back to somebody who knows what it is, such as when an abstract handler function is being registered along with a pointer to some context that will be passed to the handler; this is extremely common in C code. In C++, you can just take a pointer to a type T with virtual member f() you promise to call. The caller supplies a pointer to T2 derived from T and with its own f(). This, too, feels a bit archaic, but is at least not actively silly.
mlang23 5y ago
Consider the rule of zero.
- pyinstallwoes 5y ago
  Interesting thing indeed. Would you care to expand? Edit: This is interesting from the case of "beginning of the calendar" - coincidentally just read this from a reddit thread (linked): > Let me start with a quote you may hear in a lot of history classrooms: "Jesus Christ has been born 7 years Before Christ", sounds a bit weird doesn't it. Yep, historically the idea was to mark his birth as a dividing point, however, there are a bit of problem of determining precisely what year we're trying to set as first one. To keep the meaning intact, we would have to move dates if we find better information about exact year of his birth, not convenient at all. > When we say that we are in year 2012 CE, then we don't care that year 1 CE should be some specific event (there isn't year 0 in common notation BTW), we just need all agree on the same starting point. We leave the date intact because it's widespread, so it's more convenient not to change the date, the same way we still use non-decimal hours, minutes, seconds. If f.e. Anno Mundi (from Creation of the World) system remained in use in Europe with the agreement on the same date (Latin and Greek scholars disagreed on Biblical age of Earth), we would use it as Common Era. https://www.reddit.com/r/TrueAtheism/comments/14nuuw/i_think_using_ce_and_bce_instead_of_ad_and_bc_is/c7es912/ https://www.reddit.com/r/TrueAtheism/comments/14nuuw/i_think... This is interesting to me from a wild out there idea that feels similar to 'coordinate space for storing data.' Time as a coordinate space for simulations in a way.
  - mlang23 5y ago
    I was refering to this: https://en.cppreference.com/w/cpp/language/rule_of_three#Rule_of_zero https://en.cppreference.com/w/cpp/language/rule_of_three#Rul...

62 points by azhenley 5y ago

This is a well-known way to achieve separation of definition and implementation. The disadvantage of the method proposed in the article is that, now everything needs a pointer indirection. A solution that fixes the drawback is used by lz4's library implementation: instead of storing a void star, store a char[] array of same size as the real struct. (of course, now you have to manually make sure the struct size are in sync. It's more error prone, but still not that bad).

guerrilla 5y ago
Why not `typedef struct state state_t` in the header file, then have a `state_t state` in the header file with the `struct state { ... }` definition in the source file? This is similar to what I do in C, I don't see why it wouldn't work in C++.
- kyralis 5y ago
  The module that's importing the header needs to know the state size. To do that, it either needs to see the struct declaration or be given the explicit size with the array approach.
- sgerenser 5y ago
  That would fix the type safety issue of using a void* but not the extra pointer indirection issue, since your state struct would still have to be a pointer if you want the declaration outside of the header.
deleted 5y ago
[deleted]
temac 5y ago
Or don't do that (the char array trick) in C++ because implementers and standardizers are not clear about when/if you are allowed to store other objects in an array of chars, and even if you are it is tricky because you need to manually manage the alignment, or they are attempting to replace char with std::byte in the long term but don't really have a comprehensive and detailed plan to do so, etc. The implementation should probably provide you with std::aligned_storage which may involve some magic to handle some of those concerns, although merely the easiest ones, so I would say probably still don't use that either, unless you are already quite a C++ expert and/or are prepared to dig into the standard with no clear response about what you are attempting to do is even formally possible (the implementers/standardizers do not even know some things they make impossible for quite a long time, see for example the insanity of std::launder, or if you want to loose your mind forever the semantic of pointer provenance analysis that compilers are maybe already using to "optimize" but that what the semantic should even be is still being debated.)
- sillycross 5y ago
  Yes, strict aliasing (or type-based alias analysis?) is quite crazy, and there are some murky dark corners where the specification is different between C and C++.. I think they have a std::launder thing exactly for this purpose of "safely casting an array of bytes into an object". However, in this particular case (of using char array to hide real implementation), the implementation resides in another translation unit, so I don't think anything is going to break if LTO is not enabled. With LTO I have no idea..
  - temac 5y ago
    For projects of mixed quality without basically an unbounded workforce maintaining them (who could investigate rare/arcane bugs "introduced" by the "optimizers" in some builds), and/or using "tricks", I too am fond of not using LTO. But then I force myself to find a second reason for why the program will run correctly, and unfortunately nowadays it is more and more being strictly-conforming. Relearning std::launder, TBAA, pointer provenance, etc. every time is way too consuming. I'm forced to give-up on programmer optimization and hope for the compiler to be really up to its mythical promises (and this yet: without LTO; too dangerous...)
josefx 5y ago
> of course, now you have to manually make sure the struct size are in sync. Can't you just use a static assert in the implementation file?
hoseja 5y ago
That breaks the ABI-compatibility part of this trick.
- sillycross 5y ago
  Yes, it's a tradeoff. The upside is now you can put the object on the stack without invoking memory allocators.

prewett 5y ago

I'm surprised he doesn't refer to it by name (Pimpl). In C++ it has the advantage that you can change your implementation and retain binary compatibility because the size of your class doesn't change as you add/remove member variables (and maybe more surprisingly, add/removing virtual functions). If you're going to use C++, seems like it would be better to avoid the void* by class C { protected: struct Impl; std::unique_ptr<Impl> mImpl; }; And then define C::Impl in the .cpp file.

vlovich123 5y ago
The downside of unique_ptr used this way is that you have to define ~C in the same translation unit that the declaration of C::Impl lives.
- gerdesj 5y ago
  No it isn't. What is a downside is you could not be arsed to explain why (blah means blah) to a reasonably intelligent audience.
- ridiculous_fish 5y ago
  You can use a custom deleter, e.g. struct deleter_t { void operator()(impl_t *); }; using impl_unique_ptr_t = std::unique_ptr<impl_t, deleter_t>; You can place the implementation of the deleter alongside the impl.
  - deleted 5y ago
    [deleted]
  - cma 5y ago
    That breaks make_unique. Maybe unique_ptr inside an std::any? Error-prone casting, but managed lifetime. Though make_unique is maybe obsolete now that parameter evaluation order is better defined.
- leni536 5y ago
  You have to do that anyway, unique_ptr only makes forgetting that a compile error.

zokier 5y ago

Qt uses pimpl extensively to maintain binary compatibility: https://wiki.qt.io/D-Pointer https://wiki.qt.io/D-Pointer

ttul 5y ago

Back in the 1990s, the video game company I worked at would hang a couple of extra void pointers in every class just to, you know, store extra things that are needed as time goes on…

dnautics 5y ago
This was common in the 90s, BeOS had these everywhere in its API classes
- nielsbot 5y ago
  This is because of the fragile ABI problem in C++--it's not good for APIs for this reason. I used to program for the macOS kernel and they (used to) do exactly the same thing. See `OSMetaClassDefineReservedUnused` in `OSObject.cpp`, here: https://github.com/apple/darwin-xnu/blob/main/libkern/c++/OSObject.cpp https://github.com/apple/darwin-xnu/blob/main/libkern/c++/OS...

kazinator 5y ago

> This is not an "industry-standard" way to program in C++. It absolutely is a de facto industry standard way to program C++, and has a name: PIMPL (Pointer to IMPLementation). It has that name, because it's famous. It's probably less fashionable in newer code bases; probably someone whose head is up in C++20 will probably scoff at this, and certainly at any version where the secret is hidden by void *. It provides a good way to wrap C API's in C++. And, speaking of that, the technique is basically the spiritual equivalent of what happens in many a FFI module in languages other than C++ too, where some C handle is represented as an opaque foreign pointer, which is wrapped in some object native to the language.

cjensen 5y ago
It is not an industry standard way. It is not a sensible way. It is not even a C way. As others have said, there's no reason whatsoever to use void* here. Just declare the struct without defining it in the header. Then only define it in the C file. C is fine with pointers to structs which are merely declared but not defined.
- kazinator 5y ago
  It is a de facto standard in that it is fairly widely documented and actually done. It's not in any ISO or IEEE standard or anything of the sort. There is almost no reason whatsoever to use void * anywhere other than to write a declaration that is compatible with another one which uses it. (A pointer to any object is better implemented as a typedef for unsigned char *. This requires a cast in both directions, thus it is safer. At the same time, it is more convenient when you actually want to work with the memory as such: you have bytewise arithmetic and dereferencing.) > C is fine with pointers to structs which are merely declared but not defined. Particularly if the secret object is a struct/class then this certainly improves the code (and when the secret isn't such a thing, it can probably be made into one: e.g. a secret array of integers can probably just be a a struct containing an array, possibly a flexible one). Numerous unsafe casts are thereby eliminated. But it makes no material difference to its organization or semantics. It's still the same PIMPL pattern.

srwest 5y ago

I haven't thought about him in years, but James Plank is the reason I'm a programmer today. I was a terrible student and he likely doesn't remember me, but his classes were too much fun. He just assigned projects and a timeline and live coded similar exercises in class mostly along with the timeline. I probably would have switched away from Computer Engineering (a degree I barely use) without his classes.

deleted 5y ago

[deleted]

not2b 5y ago

Yes, as others have said, using an opaque class is preferable to using a void pointer. class Impl; This isn't just for hiding, it also gives you faster compilation.

hvdijk 5y ago
> This isn't just for hiding, it also gives you faster compilation. That seems unlikely. What are you basing this on?
- int_19h 5y ago
  It means that the header that declares class A doesn't have to #include the header that declares class B, unless B is a part of its public interface. It doesn't sound like much, but the implementation dependency chain can be much longer in practice; and more importantly, all those savings apply to every translation unit that includes the header.
  - hvdijk 5y ago
    Ah, that's what you meant. Your previous comment reads as if an opaque class results in faster compilation than a void pointer, but those are the same as far as that goes, and you were comparing both to a non-pimpl version.
    - chippiewill 5y ago
      There might be a few hundred CPU cycles saved in compilation from not having to static cast the void* pointer to the implementation class.
      - hvdijk 5y ago
        Haha, sure, and the other way around, time is spent constructing the compile-time type info for the pimpl class. Let's ignore both since the impact would be so small it would almost certainly be drowned out by all the random noise that happens all the time.

pjmlp 5y ago

Besides what everyone else was written regarding pimpl, with c++ modules the only reason to use PIMPL is to ensure ABI stability.

ncmncm 5y ago

It seems kind of stupid to use "-std=c++98" when building this stuff. Nothing in later Standards interferes with it. Maybe it is meant to serve notice that the method is archaic, and that better ways to achieve the same thing will be presented later. But then one might just as well code the throwaway examples in C and use a damn C compiler. It's archaic even for C. There is just no merit in storing a void* that will then be cast to some arbitrary other known type every time it is used. The only really valid use for a void pointer is when it is being passed through a subsystem and comes back to somebody who knows what it is, such as when an abstract handler function is being registered along with a pointer to some context that will be passed to the handler; this is extremely common in C code. In C++, you can just take a pointer to a type T with virtual member f() you promise to call. The caller supplies a pointer to T2 derived from T and with its own f(). This, too, feels a bit archaic, but is at least not actively silly.

mlang23 5y ago

Consider the rule of zero.

pyinstallwoes 5y ago
Interesting thing indeed. Would you care to expand? Edit: This is interesting from the case of "beginning of the calendar" - coincidentally just read this from a reddit thread (linked): > Let me start with a quote you may hear in a lot of history classrooms: "Jesus Christ has been born 7 years Before Christ", sounds a bit weird doesn't it. Yep, historically the idea was to mark his birth as a dividing point, however, there are a bit of problem of determining precisely what year we're trying to set as first one. To keep the meaning intact, we would have to move dates if we find better information about exact year of his birth, not convenient at all. > When we say that we are in year 2012 CE, then we don't care that year 1 CE should be some specific event (there isn't year 0 in common notation BTW), we just need all agree on the same starting point. We leave the date intact because it's widespread, so it's more convenient not to change the date, the same way we still use non-decimal hours, minutes, seconds. If f.e. Anno Mundi (from Creation of the World) system remained in use in Europe with the agreement on the same date (Latin and Greek scholars disagreed on Biblical age of Earth), we would use it as Common Era. https://www.reddit.com/r/TrueAtheism/comments/14nuuw/i_think_using_ce_and_bce_instead_of_ad_and_bc_is/c7es912/ https://www.reddit.com/r/TrueAtheism/comments/14nuuw/i_think... This is interesting to me from a wild out there idea that feels similar to 'coordinate space for storing data.' Time as a coordinate space for simulations in a way.
- mlang23 5y ago
  I was refering to this: https://en.cppreference.com/w/cpp/language/rule_of_three#Rule_of_zero https://en.cppreference.com/w/cpp/language/rule_of_three#Rul...