30 ms·

This is a well-known way to achieve separation of definition and implementation. The disadvantage of the method proposed in the article is that, now everything

This is a well-known way to achieve separation of definition and implementation. The disadvantage of the method proposed in the article is that, now everything needs a pointer indirection. A solution that fixes the drawback is used by lz4's library implementation: instead of storing a void star, store a char[] array of same size as the real struct. (of course, now you have to manually make sure the struct size are in sync. It's more error prone, but still not that bad).

guerrilla 5y ago
Why not `typedef struct state state_t` in the header file, then have a `state_t state` in the header file with the `struct state { ... }` definition in the source file? This is similar to what I do in C, I don't see why it wouldn't work in C++.
- kyralis 5y ago
  The module that's importing the header needs to know the state size. To do that, it either needs to see the struct declaration or be given the explicit size with the array approach.
- sgerenser 5y ago
  That would fix the type safety issue of using a void* but not the extra pointer indirection issue, since your state struct would still have to be a pointer if you want the declaration outside of the header.
deleted 5y ago
[deleted]
temac 5y ago
Or don't do that (the char array trick) in C++ because implementers and standardizers are not clear about when/if you are allowed to store other objects in an array of chars, and even if you are it is tricky because you need to manually manage the alignment, or they are attempting to replace char with std::byte in the long term but don't really have a comprehensive and detailed plan to do so, etc. The implementation should probably provide you with std::aligned_storage which may involve some magic to handle some of those concerns, although merely the easiest ones, so I would say probably still don't use that either, unless you are already quite a C++ expert and/or are prepared to dig into the standard with no clear response about what you are attempting to do is even formally possible (the implementers/standardizers do not even know some things they make impossible for quite a long time, see for example the insanity of std::launder, or if you want to loose your mind forever the semantic of pointer provenance analysis that compilers are maybe already using to "optimize" but that what the semantic should even be is still being debated.)
- sillycross 5y ago
  Yes, strict aliasing (or type-based alias analysis?) is quite crazy, and there are some murky dark corners where the specification is different between C and C++.. I think they have a std::launder thing exactly for this purpose of "safely casting an array of bytes into an object". However, in this particular case (of using char array to hide real implementation), the implementation resides in another translation unit, so I don't think anything is going to break if LTO is not enabled. With LTO I have no idea..
  - temac 5y ago
    For projects of mixed quality without basically an unbounded workforce maintaining them (who could investigate rare/arcane bugs "introduced" by the "optimizers" in some builds), and/or using "tricks", I too am fond of not using LTO. But then I force myself to find a second reason for why the program will run correctly, and unfortunately nowadays it is more and more being strictly-conforming. Relearning std::launder, TBAA, pointer provenance, etc. every time is way too consuming. I'm forced to give-up on programmer optimization and hope for the compiler to be really up to its mythical promises (and this yet: without LTO; too dangerous...)
  - int_19h 5y ago
    You don't need to cast anything - just placement-new it inside the array. So long as the array is properly aligned, this is fine. After that, it would be UB to peek at the bytes of the object via the array because of aliasing issues, but I don't see why it would be improper to use the pointer returned by new.
    - sillycross 5y ago
      After the object is constructed by placement-new, the class methods still needs to reinterpret_cast the char array to an object pointer to access the object. I don't think in this specific case there is an UB involved, but I'm not language standard lawyer so I'm not sure. I feel the standard's specification on what is allowed to reinterpret_cast and what isn't is arcane (or at least far from straightforward to understand).
josefx 5y ago
> of course, now you have to manually make sure the struct size are in sync. Can't you just use a static assert in the implementation file?
hoseja 5y ago
That breaks the ABI-compatibility part of this trick.
- sillycross 5y ago
  Yes, it's a tradeoff. The upside is now you can put the object on the stack without invoking memory allocators.

guerrilla 5y ago

Why not `typedef struct state state_t` in the header file, then have a `state_t state` in the header file with the `struct state { ... }` definition in the source file? This is similar to what I do in C, I don't see why it wouldn't work in C++.

kyralis 5y ago
The module that's importing the header needs to know the state size. To do that, it either needs to see the struct declaration or be given the explicit size with the array approach.
sgerenser 5y ago
That would fix the type safety issue of using a void* but not the extra pointer indirection issue, since your state struct would still have to be a pointer if you want the declaration outside of the header.

deleted 5y ago

[deleted]

temac 5y ago

Or don't do that (the char array trick) in C++ because implementers and standardizers are not clear about when/if you are allowed to store other objects in an array of chars, and even if you are it is tricky because you need to manually manage the alignment, or they are attempting to replace char with std::byte in the long term but don't really have a comprehensive and detailed plan to do so, etc. The implementation should probably provide you with std::aligned_storage which may involve some magic to handle some of those concerns, although merely the easiest ones, so I would say probably still don't use that either, unless you are already quite a C++ expert and/or are prepared to dig into the standard with no clear response about what you are attempting to do is even formally possible (the implementers/standardizers do not even know some things they make impossible for quite a long time, see for example the insanity of std::launder, or if you want to loose your mind forever the semantic of pointer provenance analysis that compilers are maybe already using to "optimize" but that what the semantic should even be is still being debated.)

sillycross 5y ago
Yes, strict aliasing (or type-based alias analysis?) is quite crazy, and there are some murky dark corners where the specification is different between C and C++.. I think they have a std::launder thing exactly for this purpose of "safely casting an array of bytes into an object". However, in this particular case (of using char array to hide real implementation), the implementation resides in another translation unit, so I don't think anything is going to break if LTO is not enabled. With LTO I have no idea..
- temac 5y ago
  For projects of mixed quality without basically an unbounded workforce maintaining them (who could investigate rare/arcane bugs "introduced" by the "optimizers" in some builds), and/or using "tricks", I too am fond of not using LTO. But then I force myself to find a second reason for why the program will run correctly, and unfortunately nowadays it is more and more being strictly-conforming. Relearning std::launder, TBAA, pointer provenance, etc. every time is way too consuming. I'm forced to give-up on programmer optimization and hope for the compiler to be really up to its mythical promises (and this yet: without LTO; too dangerous...)
- int_19h 5y ago
  You don't need to cast anything - just placement-new it inside the array. So long as the array is properly aligned, this is fine. After that, it would be UB to peek at the bytes of the object via the array because of aliasing issues, but I don't see why it would be improper to use the pointer returned by new.
  - sillycross 5y ago
    After the object is constructed by placement-new, the class methods still needs to reinterpret_cast the char array to an object pointer to access the object. I don't think in this specific case there is an UB involved, but I'm not language standard lawyer so I'm not sure. I feel the standard's specification on what is allowed to reinterpret_cast and what isn't is arcane (or at least far from straightforward to understand).

josefx 5y ago

> of course, now you have to manually make sure the struct size are in sync. Can't you just use a static assert in the implementation file?

hoseja 5y ago

That breaks the ABI-compatibility part of this trick.

sillycross 5y ago
Yes, it's a tradeoff. The upside is now you can put the object on the stack without invoking memory allocators.