8 ms·

> Runtime reflection is, in my opinion, the biggest missing feature in standard C++. Opinions?

by BruceEel 6y ago

jfkebwjsbx 6y ago
What is definitely missing is compile time reflection. I don’t think many devs care about the runtime one.
- pjmlp 6y ago
  We can anyway generate the runtime one at compile time, that is how many Java and .NET compiler plugins and annotation processors work.
  - jfkebwjsbx 6y ago
    Exactly, with a compile-time one it would be relatively easy to build portable runtime reflection frameworks/libs, for those that really need it. Otherwise it would be a feature like RTTI that everyone "hates" if enabled by default.
  - preshing 6y ago
    This might come true some day, but it seems not in C++20 and in recent static reflection proposals there was no way to distinguish reflected from non-reflected members in the same class.
    - pjmlp 6y ago
      Yep that is post C++20. I am hardly a C++ user nowadays, but such stuff interests me, because the Windows Development team managed to push C++/WinRT as replacement for C++/CX, but are declining any improvement to Visual Studio support (at least comparable to C++/CX) until ISO C++ gets similar capabilities to C++/CX. So given that some C++ usage is required depending on which APIs you want to access from .NET, you can imagine there are many WinDevs not very happy with the downgrade in tooling support. Back to your point, in what cases might such distinction be relevant?
      - preshing 6y ago
        One example is in: https://github.com/arc80/plywood/blob/master/repos/plywood/src/web/cook-docs/ply-web-cook-docs/WebCookerIndex.h#L56 https://github.com/arc80/plywood/blob/master/repos/plywood/s... Here, only one data member is meant to be serialized. The others members are there to accelerate lookups into the first data member. (Full disclosure: that structure isn't actually serialized yet, but the Arc80 Engine which uses Plywood has similar examples.)
        BruceEel 6y ago
        Interesting! Could sufficiently enhanced compile-time reflection (say we do get supercharged CT reflection in C++2b*) be used to implement fully transparent&automatic versioning in serialization frameworks? I know very little on this subject but I think boost serialization, though capable of handling multiple versions of a given class, requires manually 'tagging' source files as being say, version 1,2,etc.
        pjmlp 6y ago
        Thanks!
    - gpderetta 6y ago
      I haven't looked at the C++ reflection proposal in a bit, but I think that attributes are reflected so you should be able to use your own attributes to mark non serializable objects
- preshing 6y ago
  Perhaps, but it's easily overlooked because every project that needs runtime reflection badly enough has already rolled their own. (Some game engines have multiple reflection systems, eg. shader parameters vs. serialization.) This framework takes a particular approach and makes it an (almost) standalone component.
  - jfkebwjsbx 6y ago
    Why do/did you go for runtime reflection for all that? I thought you would use it for external scripting or mods or things like that, but not for eg serialization. I guess I am missing something. Everyone misses compile time reflection because it solves many use cases easily (like serialization) without having to go for a complex solution or incur in performance penalties of runtime reflection. In contrast, I doubt many people care about general runtime reflection which I’d expect to be a mess in C++ and most likely not as fast as people wanted.
    - preshing 6y ago
      The biggest advantage I find with runtime reflection it that it's possible to load and manipulate data that has no C++ definition at all. This is actually handy when working with 3D geometry consumed by a GPU.
- jcranmer 6y ago
  For a statically-compiled language, runtime reflection is probably a violation of the zero-cost overhead principle. There is a limited amount of reflection that can be done cheaply that has some value: getting the address of the vtable of an object [assuming it has one] for uniqueness, or getting function pointer identities for generic invokable objects. Given compile-time reflection, providing standard library functionality for certain runtime reflection tasks (such as providing a list of field information for a struct/class/union, or listing function arguments) could be useful. On the other hand, it could end up being a locale-like scenario where it's either overkill or too weak for your use case, never in the middle.
  - BruceEel 6y ago
    > a locale-like scenario where it's either overkill or too weak for your use case, never in the middle. well put.
jcelerier 6y ago
Yes (with large preference for compile-time reflection / metaclasses - runtime reflection can easily be derived from that) : http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p0707r4.pdf http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p070...
pjmlp 6y ago
It is a big pain when you need to implement stuff like serialization or data type mappers for stuff like RPC or database access. However with template metaprogramming and constexpr if there is already quite something when can do into supporting it, today, specially with C++20. Post C++20, there are ongoing plans to have support for static reflection at compile time, and metaclasses. Microsoft demoed the current state of their VC++ prototype at Virtual C++2020 days. "Dynamic Polymorphism with Metaclasses and Code Injection" https://www.youtube.com/watch?v=drt3yXI-fqk https://www.youtube.com/watch?v=drt3yXI-fqk
- badsectoracula 6y ago
  Strictly speaking, you can do it without that stuff, here are some snippets of a test i was doing in Borland C++ Builder 1 (from 1996, that is pre-C++98) a few years ago [0] (not using any of BCB's C++ extensions) which end up with this [1] (it is a bunch of controls thrown in a form mainly to test stuff, but the interesting bit is the property editor at the right). If you do not mind relying a bit too hard on the preprocessor you can also do it in plain C [2] (i mean, the C++ approach still relies on it a lot, but the C one goes all the way :-P). A game engine (for a big published game, not my own stuff) i worked on at the past used this approach (the C++ one) heavily and i think it is also quite common in some frameworks. Though personally outside of experimentation i just use Free Pascal which provides the functionality as a core feature of the language (and next version will allow attaching arbitrary attributes to properties, which addresses my "i'd like to have a description for this property for the editor without writing a GetPropertyDescription method" wish :-P). [0] https://pastebin.com/eCYs1tv8 https://pastebin.com/eCYs1tv8 [1] https://i.imgur.com/Ry5114y.png https://i.imgur.com/Ry5114y.png [2] https://pastebin.com/8udWrNcv https://pastebin.com/8udWrNcv
  - pjmlp 6y ago
    Agreed, but that is why it has been an ongoing discussion at ISO, many devs are fed up with such approaches, because it is the kind of stuff that prevents C++ to have nice IDE tooling like Java and .NET.
- flumpcakes 6y ago
jcranmer 6y ago
I've recently been toying with #[derive] macros in Rust. I have to say that framing reflection in terms of deriving an implementation of an interface is pretty powerful. But C++ makes it harder to do this than Rust, because you need to mutate the class definition to insert these methods in C++, but the Rust definition happens outside of the class (as vtable pointers are not contained within the class but passed around separately).
- kccqzy 6y ago
  Not necessarily so. Consider std::hash, which is a template heavily specialized by many types. Then containers like std::unordered_set use these specializations. You can totally turn a Rust trait into a C++ templated struct, and then turn Rust impls into template specializations of that struct. > I have to say that framing reflection in terms of deriving an implementation of an interface is pretty powerful. Just wait till you see Haskell's Generic. (Not to be confused with generics in other languages.) It turns out for most applications you don't even need to derive an implementation of an interface.
- gpderetta 6y ago
  you do not need the vtable to be inside the object itself. For example c++ std::function act similarly to an rust impl trait. But the lack of static reflection (or equivalent builtin functionality [1]) means you have to build a dedicated class for each C++ concept (the equivalent of rust trait), although libraries can help. [1] the currently non existing equivalent in C++ has been sometimes refererred as virtual concept. Once upon a time, in the 2.x era, g++ had an extension known as 'signature' which would do exactly that.
pierrebai 6y ago
And somewhat unfortunately, C++ has plan for a kind of reflection in the future [1]. But it's not here today. I'm wary of relying on a parser though: it could prove a real drag as C++ evolve and you need to keep the parser up to date. [1] http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p0592r4.html http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p059...

> Runtime reflection is, in my opinion, the biggest missing feature in standard C++. Opinions?

jfkebwjsbx 6y ago

What is definitely missing is compile time reflection. I don’t think many devs care about the runtime one.

pjmlp 6y ago
We can anyway generate the runtime one at compile time, that is how many Java and .NET compiler plugins and annotation processors work.
- jfkebwjsbx 6y ago
  Exactly, with a compile-time one it would be relatively easy to build portable runtime reflection frameworks/libs, for those that really need it. Otherwise it would be a feature like RTTI that everyone "hates" if enabled by default.
- preshing 6y ago
  This might come true some day, but it seems not in C++20 and in recent static reflection proposals there was no way to distinguish reflected from non-reflected members in the same class.
  - pjmlp 6y ago
    Yep that is post C++20. I am hardly a C++ user nowadays, but such stuff interests me, because the Windows Development team managed to push C++/WinRT as replacement for C++/CX, but are declining any improvement to Visual Studio support (at least comparable to C++/CX) until ISO C++ gets similar capabilities to C++/CX. So given that some C++ usage is required depending on which APIs you want to access from .NET, you can imagine there are many WinDevs not very happy with the downgrade in tooling support. Back to your point, in what cases might such distinction be relevant?
    - preshing 6y ago
      One example is in: https://github.com/arc80/plywood/blob/master/repos/plywood/src/web/cook-docs/ply-web-cook-docs/WebCookerIndex.h#L56 https://github.com/arc80/plywood/blob/master/repos/plywood/s... Here, only one data member is meant to be serialized. The others members are there to accelerate lookups into the first data member. (Full disclosure: that structure isn't actually serialized yet, but the Arc80 Engine which uses Plywood has similar examples.)
      - BruceEel 6y ago
        Interesting! Could sufficiently enhanced compile-time reflection (say we do get supercharged CT reflection in C++2b*) be used to implement fully transparent&automatic versioning in serialization frameworks? I know very little on this subject but I think boost serialization, though capable of handling multiple versions of a given class, requires manually 'tagging' source files as being say, version 1,2,etc.
      - pjmlp 6y ago
        Thanks!
  - gpderetta 6y ago
    I haven't looked at the C++ reflection proposal in a bit, but I think that attributes are reflected so you should be able to use your own attributes to mark non serializable objects
preshing 6y ago
Perhaps, but it's easily overlooked because every project that needs runtime reflection badly enough has already rolled their own. (Some game engines have multiple reflection systems, eg. shader parameters vs. serialization.) This framework takes a particular approach and makes it an (almost) standalone component.
- jfkebwjsbx 6y ago
  Why do/did you go for runtime reflection for all that? I thought you would use it for external scripting or mods or things like that, but not for eg serialization. I guess I am missing something. Everyone misses compile time reflection because it solves many use cases easily (like serialization) without having to go for a complex solution or incur in performance penalties of runtime reflection. In contrast, I doubt many people care about general runtime reflection which I’d expect to be a mess in C++ and most likely not as fast as people wanted.
  - preshing 6y ago
    The biggest advantage I find with runtime reflection it that it's possible to load and manipulate data that has no C++ definition at all. This is actually handy when working with 3D geometry consumed by a GPU.
jcranmer 6y ago
For a statically-compiled language, runtime reflection is probably a violation of the zero-cost overhead principle. There is a limited amount of reflection that can be done cheaply that has some value: getting the address of the vtable of an object [assuming it has one] for uniqueness, or getting function pointer identities for generic invokable objects. Given compile-time reflection, providing standard library functionality for certain runtime reflection tasks (such as providing a list of field information for a struct/class/union, or listing function arguments) could be useful. On the other hand, it could end up being a locale-like scenario where it's either overkill or too weak for your use case, never in the middle.
- BruceEel 6y ago
  > a locale-like scenario where it's either overkill or too weak for your use case, never in the middle. well put.

jcelerier 6y ago

Yes (with large preference for compile-time reflection / metaclasses - runtime reflection can easily be derived from that) : http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p0707r4.pdf http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p070...

pjmlp 6y ago

It is a big pain when you need to implement stuff like serialization or data type mappers for stuff like RPC or database access. However with template metaprogramming and constexpr if there is already quite something when can do into supporting it, today, specially with C++20. Post C++20, there are ongoing plans to have support for static reflection at compile time, and metaclasses. Microsoft demoed the current state of their VC++ prototype at Virtual C++2020 days. "Dynamic Polymorphism with Metaclasses and Code Injection" https://www.youtube.com/watch?v=drt3yXI-fqk https://www.youtube.com/watch?v=drt3yXI-fqk

badsectoracula 6y ago
Strictly speaking, you can do it without that stuff, here are some snippets of a test i was doing in Borland C++ Builder 1 (from 1996, that is pre-C++98) a few years ago [0] (not using any of BCB's C++ extensions) which end up with this [1] (it is a bunch of controls thrown in a form mainly to test stuff, but the interesting bit is the property editor at the right). If you do not mind relying a bit too hard on the preprocessor you can also do it in plain C [2] (i mean, the C++ approach still relies on it a lot, but the C one goes all the way :-P). A game engine (for a big published game, not my own stuff) i worked on at the past used this approach (the C++ one) heavily and i think it is also quite common in some frameworks. Though personally outside of experimentation i just use Free Pascal which provides the functionality as a core feature of the language (and next version will allow attaching arbitrary attributes to properties, which addresses my "i'd like to have a description for this property for the editor without writing a GetPropertyDescription method" wish :-P). [0] https://pastebin.com/eCYs1tv8 https://pastebin.com/eCYs1tv8 [1] https://i.imgur.com/Ry5114y.png https://i.imgur.com/Ry5114y.png [2] https://pastebin.com/8udWrNcv https://pastebin.com/8udWrNcv
- pjmlp 6y ago
  Agreed, but that is why it has been an ongoing discussion at ISO, many devs are fed up with such approaches, because it is the kind of stuff that prevents C++ to have nice IDE tooling like Java and .NET.
flumpcakes 6y ago

jcranmer 6y ago

I've recently been toying with #[derive] macros in Rust. I have to say that framing reflection in terms of deriving an implementation of an interface is pretty powerful. But C++ makes it harder to do this than Rust, because you need to mutate the class definition to insert these methods in C++, but the Rust definition happens outside of the class (as vtable pointers are not contained within the class but passed around separately).

kccqzy 6y ago
Not necessarily so. Consider std::hash, which is a template heavily specialized by many types. Then containers like std::unordered_set use these specializations. You can totally turn a Rust trait into a C++ templated struct, and then turn Rust impls into template specializations of that struct. > I have to say that framing reflection in terms of deriving an implementation of an interface is pretty powerful. Just wait till you see Haskell's Generic. (Not to be confused with generics in other languages.) It turns out for most applications you don't even need to derive an implementation of an interface.
gpderetta 6y ago
you do not need the vtable to be inside the object itself. For example c++ std::function act similarly to an rust impl trait. But the lack of static reflection (or equivalent builtin functionality [1]) means you have to build a dedicated class for each C++ concept (the equivalent of rust trait), although libraries can help. [1] the currently non existing equivalent in C++ has been sometimes refererred as virtual concept. Once upon a time, in the 2.x era, g++ had an extension known as 'signature' which would do exactly that.

pierrebai 6y ago

And somewhat unfortunately, C++ has plan for a kind of reflection in the future [1]. But it's not here today. I'm wary of relying on a parser though: it could prove a real drag as C++ evolve and you need to keep the parser up to date. [1] http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p0592r4.html http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p059...