6 ms·
Struct Iteration Through Abuse of the C Preprocessor
- kevingadd 11y agoYou can do some interesting things using __COUNTER__ to make code using this less ugly. And if you're willing to use a small amount of C++ templates, you can make it possible to resolve all the iteration/member info at compile time to eliminate runtime overhead. I did both of these things for Grim Fandango for vertex/uniform buffers, so that the game could use buffers without caring about whether it was using OpenGL or console APIs - but it's not open source :-( Even if you don't need something like this under normal circumstances, consider whether it could let you identify mistakes in debug builds, or avoid errors when writing your serialization code.
- Kristine1975 11y agoI'm doing something similar in C++ for vertex attributes, integration with Lua et.al. First I define the struct: INTROSPECTION_DEFINE_STRUCT(S, (int, anInt), (bool, aBool), (float, aFloat), (double, aDouble), (std::string, aString) ); Then I create an instance of it: S s{ 23, true, 37.5f, 42.25, "String" }; And finally I can e.g. iterate over all fields and output their value using a generic lambda: forEachField(s, [](auto v) { std::cout << v; }); There are also definitions for the field's type, offset, name etc. generated by the macro, so it's easy to create a binding for vertex attributes, map the struct to a table in Lua, serialize/deserialize it and so on. I'm really hoping C++ gets introspection soon. There are some proposals like https://isocpp.org/files/papers/n3996.pdf https://isocpp.org/files/papers/n3996.pdf and a working group at https://groups.google.com/a/isocpp.org/forum/#!forum/reflection https://groups.google.com/a/isocpp.org/forum/#!forum/reflect... but I'm not holding my breath for 2017.
- dman 11y agoIs the code for this available somewhere?
- Kristine1975 11y agoNot at the moment, no. But it's not rocket science. The macro essentially creates a struct like this one: struct S { // How many fields are in this struct? static constexpr unsigned int FIELD_COUNT = 5; // Full specializations are not allowed inside classes, so add a dummy parameter <T> that is equivalent to S template <unsigned int Index, typename T = S> struct FieldTraits; // Partial specialization for the first field template <typename T> struct FieldTraits<0, T> { using type = int; static constexpr std::size_t OFFSET = offsetof(S, anInt); static constexpr const char* name() { return "anInt"; } static constexpr type& value(S& self) { return self.anInt; } }; // The first field itself int anInt; // Partial specialization for the second field template <typename T> struct FieldTraits<1, T> { using type = bool; static constexpr std::size_t OFFSET = offsetof(S, aBool); static constexpr const char* name() { return "aBool"; } static constexpr type& value(S& self) { return self.aBool; } }; // The second field itself bool aBool; ... }; Now that we have the meta-information about the struct's fields, we can process them: // Meta-function to process a struct's fields template <typename T, unsigned int FieldIndex, unsigned int FieldCount> struct ForEach { template <typename Functor> static void apply(T& object, Functor f) { // Get the field's value and call the functor with it auto value = T::FieldTraits<FieldIndex>::value(object); f(value); // Next field ForEach<T, FieldIndex + 1, FieldCount>::apply(object, f); }; }; // Partial specialization to end iteration template <typename T, unsigned int FieldCount> struct ForEach<T, FieldCount, FieldCount> { template <typename Functor> static void apply(T& object, Functor f) { }; }; And finally a bit of syntactic sugar: template <typename T, typename Functor> void forEachField(T& object, Functor f) { ForEach<T, 0, T::FIELD_COUNT>::apply(object, f); } *(omitted for the sake of clarity: dealing with const objects, using rvalue references and std::forward for the functor to make the code work with non-copyable functors)
- dman 11y agoThanks for taking the time to write this!
- helmut_hed 11y agoHave you used Boost.Fusion? This looks very much like BOOST_FUSION_ADAPT_STRUCT and seems to have similar design goals
- cperciva 11y ago__COUNTER__ isn't portable (yet).
- Kristine1975 11y agoThat (define macros, include file that uses them) is fairly vanilla usage of the C preprocessor. Abuse is something like this: https://github.com/rofl0r/chaos-pp https://github.com/rofl0r/chaos-pp and https://github.com/rofl0r/order-pp https://github.com/rofl0r/order-pp
- evmar 11y agoTaking a step back: what you're trying to accomplish is code generation -- a program that generates code. From that perspective, why would you choose to write that program using the C preprocessor, when there are so many other nice languages around?
- anon4 11y agoThis is what I was going to suggest. C is even pretty easy to parse - you can just take the ready ANTLR grammar and use it to generate a parser in whatever language you want. Of course, he might go even one step further and specify the structs in some other language, then generate C source files from them, which would also allow you to easily generate consumers for that data format in other languages and now you're reinventing Cap'n Proto.
- DannyBee 11y agoErrr, he doesn't even need to use ANTLR, he can use clang, as a library. It can be linked into an app, has a nice AST, and matching abilities: http://clang.llvm.org/docs/LibASTMatchersReference.html http://clang.llvm.org/docs/LibASTMatchersReference.html Now he has the actual compiler doing the work for him.
- comex 11y agoSure, and now everyone who wants to compile your program has to install clang, which might take forever to compile for someone on a source-based distro, or require PITA manual work such as on Windows. Not to mention that you have to figure out how to configure your build system to (a) depend on clang and (b) build a helper program (including overriding the normal target architecture choice) - the difficulty of which depends on your setup, but if you follow the common practice of maintaining separate Visual Studio and CMake/etc. project files, in order to cater to IDE users while still being cross-platform, you have to do it twice! I hate build systems.
- ncraun 11y agoIn actual production use, I think it could be better to write a separate program that will generate the code for you instead of using the C preprocessor (or use a language with this kind of introspection built in). However, using this hack lets you depend only on the facilities of the language, and that can be useful in simplifying the build process. Plus it was fun to create, too!
- luckydude 11y agoWhile it is true that compilers pad for alignment it is pretty easy to avoid that, at least in my experience. So an easier approach, but more fragile, is just to define the struct, count up how big you think it should be and then do an assert(sizeof(struct foo) == FOO_SIZE); If that assert doesn't pop then you can just do write(fd, &foo, FOO_SIZE); I know that is a lot less elegant but it works and isn't doing the translation back and forth. If you don't like that you might consider using Sun's RPC marshalling code that is (used to be?) part of Linux.
- luckydude 11y agoYeah, in poking around it looks like RPC is still a thing and it's solved this problem in an automated way. http://docs.freebsd.org/44doc/psd/23.rpc/paper.pdf http://docs.freebsd.org/44doc/psd/23.rpc/paper.pdf http://docs.freebsd.org/44doc/psd/22.rpcgen/paper.pdf http://docs.freebsd.org/44doc/psd/22.rpcgen/paper.pdf
- asteadman 11y agoWow, I just finished doing something similar. The use of x-macros is definitely a little cleaner / canonical than my approach, which instead "recurses" through a VA_MACRO. For an excellent guide on recursion in macros, see this article on how to abuse the C pre-processor: https://github.com/pfultz2/Cloak/wiki/C-Preprocessor-tricks,-tips,-and-idioms https://github.com/pfultz2/Cloak/wiki/C-Preprocessor-tricks,.... In the end I end up defining structs like this: SERIALIZER_STRUCT( structname, int8_t, field1, int16_t, field2, ... ); Initializing them like this: structname instance = {.field1 = 1, .field2=2}; and serializing them like this: SERIALIZER_SERIALIZE( structname, &instance, bigbuff_ptr ); As for just using a packed structure, there are definitely caveats with some compilers (ie: compilers for embedded systems may not handle various edge-cases for misaligned access, particularly when dereferencing pointers to structs.)
- Mellowcandle 11y agowhy not using packed structure compiler feature?
- WesternStar 11y agoI had the same question #pragma pack works just fine.
- blt 11y agoMaybe wants the default alignment for performance. Plus that still leaves endian issues
- comex 11y agoA bit cleaner setup, in my opinion, makes the "x macro" an argument to the user-defined macro, rather than depending on #include. For example, to define an enum along with a string representation function for it: #define DEFINE_ENUM(name, variants) \ enum name { \ variants(DEFINE_ENUM_VARIANT_ITSELF) \ }; \ const char *repr_##name(enum name e) { \ switch (e) { \ variants(DEFINE_ENUM_VARIANT_REPR_CASE) \ default: return "?"; \ } \ } #define DEFINE_ENUM_VARIANT_ITSELF(variant) variant, #define DEFINE_ENUM_VARIANT_REPR_CASE(variant) \ case variant: return #variant; // client code: #define foo_variants(x) \ x(FOO_1) \ x(FOO_2) DEFINE_ENUM(foo, foo_variants)
- Kristine1975 11y agoC preprocessor "abuse" to create reflective enums: http://www.codeproject.com/Articles/1002895/Clean-Reflective-Enums-Enum-to-String-with-Nice-Sy http://www.codeproject.com/Articles/1002895/Clean-Reflective...