5 ms·

Recently had a discussion with a colleague about JSON in C++. It's actually a big issue, because reflection isn't really supported in C++. It's not just a bloc

Recently had a discussion with a colleague about JSON in C++. It's actually a big issue, because reflection isn't really supported in C++. It's not just a blocker for parsing objects as JSON, it's a fundamental limitation for being able to map objects to any kind of format in a general fashion without creating initializers for every object. I actually have no idea what kind of black magic they're doing to achieve this, but it sounds pretty admirable to me.

cocoa19 7y ago
Why do you need reflection for a message format that requires objects (use hash map) and lists (array or linked list).
- chrisseaton 7y ago
  I think because looking up in a hash map is going to be bad for performance. You want it in an object so you can use fixed offsets. And using individual hash maps for each object is wasteful if they all have the same layout.
  - stefan_ 7y ago
    If performance is a concern, why on earth are you using JSON. Just the absurd amount of string-integer conversions kill any hope of that.
    - chrisseaton 7y ago
      > If performance is a concern, why on earth are you using JSON. Maybe JSON is what you get sent and you've got no control over that? Or JSON is what you want to expose and you've got no control over that either? You can want to achieve reasonable performance while still meeting external requirements.
      - jstimpfle 7y ago
        You shouldn't take major compromises to improve a 90 to a 100, if you could get a 1000 by simply choosing the right approach (like using binary when performance really matters).
      - dodobirdlord 7y ago
        If you're receiving JSON from outside of your system then the performance talk about converting to objects with fixed offsets and hashmaps with the same layout goes out the window, as you don't control the format.
        chrisseaton 7y ago
        You can meet an expected format in a fast path, and have a slow path fallback where it doesn’t meet and expected format.
    - jcranmer 7y ago
      JSON is effectively a data interchange format. And for data interchange formats, there are reasons to emit property names (for example, because you want humans to be able to read, and maybe even write it). The fact that you want to use JSON as your serialization format shouldn't require you to make your internal data formats use hash tables instead of structs.
jstimpfle 7y ago
Parsing is simple, you just need to be explicit about what you expect to parse. Like, write 10 lines of straightforward parse function (or maybe better, data description + your own code that you use for all your types) that fills your object's fields. Alternatively, if you have a super structured approach about your data (like you're required to have with template metaprogramming) then you can easily generate this stuff. Like, using a usable scripting language or such. The idea that types can replace code is just broken. The type system is not a programming language. Or at least, as proven by C++/Haskell/..., not a remotely usable one.
- chrisseaton 7y ago
  > Parsing is simple It's obviously not simple - no two parsers have the same behaviour! http://seriot.ch/parsing_json.php http://seriot.ch/parsing_json.php
  - jstimpfle 7y ago
    Tip, if you want to defeat an argument by quoting only three words, make sure you're talking about the same thing. To be clear, I was not talking about lexing JSON, but about how to configure a "meta-parser" like this library to actually convert some JSON to concrete application data objects. The point of contention was, "should we really use such a massive library or mustn't it be sufficient to have a simple bag of parsing primitives (which we can use to actually build the right thing)". And I'm not a fan of JSON either...
    - chrisseaton 7y ago
      Parsing JSON is not simple - not even when you know the structure you're parsing against. For example that reference shows even parsing numbers is not the same everywhere, so that applies even when the structure is as expected. > Tip, if you want to defeat an argument by quoting only three words, make sure you're talking about the same thing. Please don't be snarky - if you disagree say so and why.
      - 7y ago
omouse 7y ago
Reflection isn't required; keys in JSON are strings, and there are basic data types that are supported (strings, numbers, booleans, arrays, and dictionaries which are more of the same). What's wrong with writing "initializers" which are serializers/deserializers? If you're looking for automatic file format to C++ class object, why settle for JSON (whether it's this library or JSONCpp) why not use Thrift or Protocol Buffers? - Thrift: https://thrift.apache.org/ https://thrift.apache.org/ - Protocol Buffers: https://developers.google.com/protocol-buffers/ https://developers.google.com/protocol-buffers/
- mristin 7y ago
  See also Mapry [0], a code generator we particularly made for JSON. [0] https://github.com/Parquery/mapry https://github.com/Parquery/mapry
- biggestdecision 7y ago
  >why settle for JSON Because I'm not writing code in a vacuum, and JSON is what everyone else is using. JSON is chosen for simplicity & interoperability.
  - Ephiiz 7y ago
    This exactly, sometimes you have to use something not because it is the best option, but because it is what your team is using, or because the ease of use is worth the performance drawbacks.
CoolGuySteve 7y ago
I think worries about reflection are overthinking the problem for the most part. This has carried me for 99% of all use cases along with a try/catch that barfs and an array specialization that makes a vector: template<typename T> T get(const char* key) { return boost::lexical_cast<T>( json.GetObject(key) ); } The reality is that if I see some value of a type I wasn't expecting, something probably went wrong or changed in the protocol anyways.

cocoa19 7y ago

Why do you need reflection for a message format that requires objects (use hash map) and lists (array or linked list).

chrisseaton 7y ago
I think because looking up in a hash map is going to be bad for performance. You want it in an object so you can use fixed offsets. And using individual hash maps for each object is wasteful if they all have the same layout.
- stefan_ 7y ago
  If performance is a concern, why on earth are you using JSON. Just the absurd amount of string-integer conversions kill any hope of that.
  - chrisseaton 7y ago
    > If performance is a concern, why on earth are you using JSON. Maybe JSON is what you get sent and you've got no control over that? Or JSON is what you want to expose and you've got no control over that either? You can want to achieve reasonable performance while still meeting external requirements.
    - jstimpfle 7y ago
      You shouldn't take major compromises to improve a 90 to a 100, if you could get a 1000 by simply choosing the right approach (like using binary when performance really matters).
    - dodobirdlord 7y ago
      If you're receiving JSON from outside of your system then the performance talk about converting to objects with fixed offsets and hashmaps with the same layout goes out the window, as you don't control the format.
      - chrisseaton 7y ago
        You can meet an expected format in a fast path, and have a slow path fallback where it doesn’t meet and expected format.
  - jcranmer 7y ago
    JSON is effectively a data interchange format. And for data interchange formats, there are reasons to emit property names (for example, because you want humans to be able to read, and maybe even write it). The fact that you want to use JSON as your serialization format shouldn't require you to make your internal data formats use hash tables instead of structs.

jstimpfle 7y ago

Parsing is simple, you just need to be explicit about what you expect to parse. Like, write 10 lines of straightforward parse function (or maybe better, data description + your own code that you use for all your types) that fills your object's fields. Alternatively, if you have a super structured approach about your data (like you're required to have with template metaprogramming) then you can easily generate this stuff. Like, using a usable scripting language or such. The idea that types can replace code is just broken. The type system is not a programming language. Or at least, as proven by C++/Haskell/..., not a remotely usable one.

chrisseaton 7y ago
> Parsing is simple It's obviously not simple - no two parsers have the same behaviour! http://seriot.ch/parsing_json.php http://seriot.ch/parsing_json.php
- jstimpfle 7y ago
  Tip, if you want to defeat an argument by quoting only three words, make sure you're talking about the same thing. To be clear, I was not talking about lexing JSON, but about how to configure a "meta-parser" like this library to actually convert some JSON to concrete application data objects. The point of contention was, "should we really use such a massive library or mustn't it be sufficient to have a simple bag of parsing primitives (which we can use to actually build the right thing)". And I'm not a fan of JSON either...
  - chrisseaton 7y ago
    Parsing JSON is not simple - not even when you know the structure you're parsing against. For example that reference shows even parsing numbers is not the same everywhere, so that applies even when the structure is as expected. > Tip, if you want to defeat an argument by quoting only three words, make sure you're talking about the same thing. Please don't be snarky - if you disagree say so and why.
    - 7y ago

omouse 7y ago

Reflection isn't required; keys in JSON are strings, and there are basic data types that are supported (strings, numbers, booleans, arrays, and dictionaries which are more of the same). What's wrong with writing "initializers" which are serializers/deserializers? If you're looking for automatic file format to C++ class object, why settle for JSON (whether it's this library or JSONCpp) why not use Thrift or Protocol Buffers? - Thrift: https://thrift.apache.org/ https://thrift.apache.org/ - Protocol Buffers: https://developers.google.com/protocol-buffers/ https://developers.google.com/protocol-buffers/

mristin 7y ago
See also Mapry [0], a code generator we particularly made for JSON. [0] https://github.com/Parquery/mapry https://github.com/Parquery/mapry
biggestdecision 7y ago
>why settle for JSON Because I'm not writing code in a vacuum, and JSON is what everyone else is using. JSON is chosen for simplicity & interoperability.
- Ephiiz 7y ago
  This exactly, sometimes you have to use something not because it is the best option, but because it is what your team is using, or because the ease of use is worth the performance drawbacks.

CoolGuySteve 7y ago

I think worries about reflection are overthinking the problem for the most part. This has carried me for 99% of all use cases along with a try/catch that barfs and an array specialization that makes a vector: template<typename T> T get(const char* key) { return boost::lexical_cast<T>( json.GetObject(key) ); } The reality is that if I see some value of a type I wasn't expecting, something probably went wrong or changed in the protocol anyways.