10 ms·

I don't understand the JSON obsession. JSON as any other file format should be a small detail in any application and require very little plumbing code. In ever

I don't understand the JSON obsession. JSON as any other file format should be a small detail in any application and require very little plumbing code. In every application, any dependency to JSON should be minimized, contained and preferably eliminated.

jstimpfle 7y ago
I tried to suppress the urge to release my frustration, but seeing that the only slightly critical comment in this thread is the downvoted one let me forget my good intentions. WTF. 10s of thousands of line of code, > 10K lines in include files (yay compile times!) for a task that should be only a side concern and should be straightforward to implement. JSON has how many? 2? data types, and writing super efficient parser primitives for it must be easily possible to do in <1K lines of implementation and <50 lines of headers. If there are special requirements in performance or type safety / structural integrity, JSON is far from the ideal choice. And even if one is required to use it under such circumstances due to political issues (in which case, sorry), that doesn't mean that all the other >99% of applications call for a massively oversized library like this. (Update, upon closer inspection I'm not even sure that speed is the main concern of this library. It really looks like it tries hard to be "Modern C++" by implementing all sorts of compile time insanity, bells and whistles (which will never fit 100% with clients and will be impossible to adapt)).
- Betelgeuse90 7y ago
  Recently had a discussion with a colleague about JSON in C++. It's actually a big issue, because reflection isn't really supported in C++. It's not just a blocker for parsing objects as JSON, it's a fundamental limitation for being able to map objects to any kind of format in a general fashion without creating initializers for every object. I actually have no idea what kind of black magic they're doing to achieve this, but it sounds pretty admirable to me.
  - cocoa19 7y ago
    Why do you need reflection for a message format that requires objects (use hash map) and lists (array or linked list).
    - chrisseaton 7y ago
      I think because looking up in a hash map is going to be bad for performance. You want it in an object so you can use fixed offsets. And using individual hash maps for each object is wasteful if they all have the same layout.
      - stefan_ 7y ago
        If performance is a concern, why on earth are you using JSON. Just the absurd amount of string-integer conversions kill any hope of that.
        chrisseaton 7y ago
        > If performance is a concern, why on earth are you using JSON. Maybe JSON is what you get sent and you've got no control over that? Or JSON is what you want to expose and you've got no control over that either? You can want to achieve reasonable performance while still meeting external requirements.
        jstimpfle 7y ago
        You shouldn't take major compromises to improve a 90 to a 100, if you could get a 1000 by simply choosing the right approach (like using binary when performance really matters).
        dodobirdlord 7y ago
        If you're receiving JSON from outside of your system then the performance talk about converting to objects with fixed offsets and hashmaps with the same layout goes out the window, as you don't control the format.
- jbverschoor 7y ago
  WTF indeed. Comment made me check out the source code.. I could only find some headerfiles which a) have the complete implementation in it and b) of course include all dependencies. This is the exact nonsense that is happening all over the place, and why we need 512GB ram in workstations to compile something. I can't help but think that some basic knowledge about libraries is missing.
jcelerier 7y ago
> In every application, any dependency to JSON should be minimized, contained and preferably eliminated. how do you suggest interaction with $standardised_protocol which uses JSON in that case ?
- kyberias 7y ago
  When implementing that protocol, dependency to JSON should be minimized, contained and preferably eliminated.
- jstimpfle 7y ago
  JSON is just a transport. Get the payload off from library structures to your own data structures ASAP, to remove dependencies, and to profit from your own setup. And how to make the transformation? Using your own setup. No point in expressing (duplicating!) your own data structure definitions in a random library's DDL (which, apart from the inflicted duplication, will not fit very well since it doesn't know your project).
  - nepeckman 7y ago
    But this advice only really applies to projects where you create every service and layer. If you want to incorporate a 3rd party service, or want to allow your service to be used by others, you need some sort of common format. JSON isn't the most efficient, but it gets a lot of exposure from being a first class citizen of the web. Msgpack is good if you want efficiency. But you need something if you want your program to communicate with others.
  - jcelerier 7y ago
    > Get the payload off from library structures to your own data structures ASAP, I don't understand, doing this step is exactly why you need to have a library such as the one which is being discussed here. There's no in-language way in C++ to go from a string that looks like `{ "foo": [1,2,3] }` to `struct { int foo[3]; }`.
    - jstimpfle 7y ago
omouse 7y ago
Yep this is why I like JSON Cpp, there's a single-file build and I've had success using it with Repl.it: https://neverfriday.com/2013/07/26/learning-jsoncpp/ https://neverfriday.com/2013/07/26/learning-jsoncpp/ Since JSON only has basic data types, it's easy to just parse and retrieve objects, similar to using XML tree parsing libraries back in the day.
StreamBright 7y ago
Yes but JSON what everybody knows. I have seen a presentation about a not too small company where the engineer explained how much hurdle is to have JSON everywhere in their big data platform. I asked him why they do not use something like Avro instead. He just gave me this weird look like I am asking something extremely stupid and could not come up with any reason. Some engineers just have no idea of the alternatives or the pros and cons of chosing different file/message formats.
rurban 7y ago
Then you should be enlightened: Of the myriads of existing transfer or serialization formats, * JSON is still the only secure by default one (if you ignore the two later updates, which made it insecure), * JSON is by far the easiest to parse (small and secure, no references), * is natively supported by Javascript. It has some minor design mistakes, and is not binary (such as msgpack, which is therefore faster), but still is a sane and proper default. Esp. it does not support objects and other extensions, with its initializer problems on MITM attacks, and is properly ended. Unlike XML, YAML, BJSON, BSON, ... which do have major problems, or msgpack, protobuf, ... which do have minor problems. In every application, any dependency to XML should be minimized, contained and preferably eliminated.
- jstimpfle 7y ago
  > JSON is still the only secure by default one (if you ignore the two later updates, which made it How can a data format be insecure? > JSON is by far the easiest to parse (small and secure, no references), It's relatively easy to parse (while not as easy as a sane well-specified version of CSV would be). But it also offers no structural integrity, which means that you need to augment the parsing code with terribly ugly validation code. (I don't really use JSON frequently, but the attempts at providing a principled validation framework on top of JSON that I've seen were... unsatisfying). > In every application, any dependency to XML should be minimized, contained and preferably eliminated. That's missing the point. It's not about JSON vs XML vs whatever. It's that application logic shouldn't be operating on transport data formats. And even if JSON has a simple representation as runtime data objects (in most scripting languages), that representation is pretty far from ideal compared to a representation tailored towards your specific application logic.
  - legulere 7y ago
    > How can a data format be insecure? XML external entities allow for arbitrary file inclusion: https://en.wikipedia.org/wiki/XML_external_entity_attack https://en.wikipedia.org/wiki/XML_external_entity_attack You can make a badly configured XML parser allocate memory until it crashes: https://en.wikipedia.org/wiki/Billion_laughs_attack https://en.wikipedia.org/wiki/Billion_laughs_attack You cannot host user-generated xmls on a domain without making yourself vulnerable to cross site scripting attacks. Browsers will happily execute any javascript you include: https://stackoverflow.com/questions/384639/how-to-include-javascript-in-xml-document https://stackoverflow.com/questions/384639/how-to-include-ja...
deleted 7y ago
[deleted]
paulddraper 7y ago
> In every application, any dependency to JSON should be minimized, contained and preferably eliminated. It depends on the extant to which your program needs to communicate/interoperate with other programs...in particular ones without the same programming language (no Java RMI). Under a "Unix" philosophy, this happens quite a bit.
devwastaken 7y ago
Json is easier for humans to read and modify, and it's more straightforward to work with in languages like python where you don't have to declare types. Json c++ implimentation is large probably because of trying to provide one library that does all the dynamic things json can do, but in a static language. It also has to be safe to parse, so there's that.
- CoolGuySteve 7y ago
  For config files at least, I've moved entirely to ini format. Python's configparser is handy for machine generating complicated configs and the nesting/list support in JSON adds way more complexity than I need compared to just using comma separated lists as values. This is my entire ini parser for making a std::map<std::string, std::string>. I cast/parse the values at the use site: std::regex section_test("\\[(.*?)\\]"); std::regex value_test("([\\w\\.]+)\\s*=\\s*([^\\+]+(?!\\+{3}))"); while(-1 != getline((char**)&buf, &len, file)) { std::string line(buf); trim(line); if(!line.size() || line[0] == '#') continue; std::smatch match; if(std::regex_search(line, match, section_test)) current_section = match[1].str(); else if(std::regex_search(line, match, value_test)) { std::string key(match[1].str()); std::string value(match[2].str()); info("INIReader: %s::%s = %s\n", current_section.c_str(), key.c_str(), value.c_str()); config[current_section][key] = value; } }
mrlala 7y ago
I can tell this is a C++ thread, heaven forbid you touch anything new within the last 20 years... The JSON obsession is that everything under the moon supports it- and it's a nice easy structure which can be converted to a dynamic or statically typed object with no issues. Plus it's very human readable, so it's great for debugging. If you sit there and ONLY use C++, ok why bother? But many systems need to talk to different systems these days. I can setup a webapi that can communicate with my angular typescript, C#, matlab all through nice easy JSON. Each language just converts it to the object and I'm good to go. The question is why are you so against such a widely accepted format of communication?
Kenji 7y ago
If you don't understand why your application requires marshalling/serialisation, you never wrote a real application. You question goes away as soon as you do.
umvi 7y ago
Well I work on embedded systems that have embedded webservers (think your router's webpage). And what data format is easiest to work with when dealing with webpages and browsers? JSON. Hence, our embedded C++ backend can now easily accommodate and return JSON to the front end using this library. Plus, what data format do you propose for things like configuration files, etc? No matter what format you choose, you are going to need a parser of that format for C/C++.
- snagglegaggle 7y ago
  I think TOML is better, maybe with some modifications. I dislike that it has a person's name in it though; maybe we retcon it to Text Object Markup Language. TOML has inline and multiline forms for most objects.
  - afiori 7y ago
    but in this context TOML and JSON are almost isomorphic (ok, parsing JSON is more unreliable than many realize apparently). As far as I can see the problem is that it is difficult to convert arbitrary JSON object to C++ objects. TOML (I also like it, for different reasons) isn't gonna help with that.
    - snagglegaggle 7y ago
      Wait, everyone is complaining you can't do `object.member.submember`? I don't see the issue with object['member']['sub']. It is slightly more typing, but such is life.
      - umvi 7y ago
        It's better to do object.at("member").at("sub") so that it throws an exception instead of seg faulting if one of the keys doesn't exist
        snagglegaggle 7y ago
jarjoura 7y ago
Developers obsess about JSON for a simple reason, JSON works natively with Javascript. If you have a backend service written in C++, then communicating over JSON is easily consumed by nodejs and the browser. Of course there are other data transport protocols, XML and ProtoBuf to name a couple off the top of my head, but those are extremely heavyweight. If you want to use JSON for config files, again, they are easy to make human readable and parse. Grab an off the shelf parser and support whatever scenario users may need. Sure we have yaml and ini, or whatever key=value config format your project creates, but you'll be much more on your own with how deep you want to support various value types. As far as your complaint or desire for JSON to be minimized or eliminated as a philosophy, I'm not sure why this is the top comment about a very well made and loved C++ library to parse JSON. Are you hoping your comment will change people's minds about what library to use or whether their system needs JSON at all? I don't really understand your goal.

jstimpfle 7y ago

I tried to suppress the urge to release my frustration, but seeing that the only slightly critical comment in this thread is the downvoted one let me forget my good intentions. WTF. 10s of thousands of line of code, > 10K lines in include files (yay compile times!) for a task that should be only a side concern and should be straightforward to implement. JSON has how many? 2? data types, and writing super efficient parser primitives for it must be easily possible to do in <1K lines of implementation and <50 lines of headers. If there are special requirements in performance or type safety / structural integrity, JSON is far from the ideal choice. And even if one is required to use it under such circumstances due to political issues (in which case, sorry), that doesn't mean that all the other >99% of applications call for a massively oversized library like this. (Update, upon closer inspection I'm not even sure that speed is the main concern of this library. It really looks like it tries hard to be "Modern C++" by implementing all sorts of compile time insanity, bells and whistles (which will never fit 100% with clients and will be impossible to adapt)).

Betelgeuse90 7y ago
Recently had a discussion with a colleague about JSON in C++. It's actually a big issue, because reflection isn't really supported in C++. It's not just a blocker for parsing objects as JSON, it's a fundamental limitation for being able to map objects to any kind of format in a general fashion without creating initializers for every object. I actually have no idea what kind of black magic they're doing to achieve this, but it sounds pretty admirable to me.
- cocoa19 7y ago
  Why do you need reflection for a message format that requires objects (use hash map) and lists (array or linked list).
  - chrisseaton 7y ago
    I think because looking up in a hash map is going to be bad for performance. You want it in an object so you can use fixed offsets. And using individual hash maps for each object is wasteful if they all have the same layout.
    - stefan_ 7y ago
      If performance is a concern, why on earth are you using JSON. Just the absurd amount of string-integer conversions kill any hope of that.
      - chrisseaton 7y ago
        > If performance is a concern, why on earth are you using JSON. Maybe JSON is what you get sent and you've got no control over that? Or JSON is what you want to expose and you've got no control over that either? You can want to achieve reasonable performance while still meeting external requirements.
        jstimpfle 7y ago
        You shouldn't take major compromises to improve a 90 to a 100, if you could get a 1000 by simply choosing the right approach (like using binary when performance really matters).
        dodobirdlord 7y ago
        If you're receiving JSON from outside of your system then the performance talk about converting to objects with fixed offsets and hashmaps with the same layout goes out the window, as you don't control the format.
jbverschoor 7y ago
WTF indeed. Comment made me check out the source code.. I could only find some headerfiles which a) have the complete implementation in it and b) of course include all dependencies. This is the exact nonsense that is happening all over the place, and why we need 512GB ram in workstations to compile something. I can't help but think that some basic knowledge about libraries is missing.

jcelerier 7y ago

> In every application, any dependency to JSON should be minimized, contained and preferably eliminated. how do you suggest interaction with $standardised_protocol which uses JSON in that case ?

kyberias 7y ago
When implementing that protocol, dependency to JSON should be minimized, contained and preferably eliminated.
jstimpfle 7y ago
JSON is just a transport. Get the payload off from library structures to your own data structures ASAP, to remove dependencies, and to profit from your own setup. And how to make the transformation? Using your own setup. No point in expressing (duplicating!) your own data structure definitions in a random library's DDL (which, apart from the inflicted duplication, will not fit very well since it doesn't know your project).
- nepeckman 7y ago
  But this advice only really applies to projects where you create every service and layer. If you want to incorporate a 3rd party service, or want to allow your service to be used by others, you need some sort of common format. JSON isn't the most efficient, but it gets a lot of exposure from being a first class citizen of the web. Msgpack is good if you want efficiency. But you need something if you want your program to communicate with others.
- jcelerier 7y ago
  > Get the payload off from library structures to your own data structures ASAP, I don't understand, doing this step is exactly why you need to have a library such as the one which is being discussed here. There's no in-language way in C++ to go from a string that looks like `{ "foo": [1,2,3] }` to `struct { int foo[3]; }`.
  - jstimpfle 7y ago

omouse 7y ago

Yep this is why I like JSON Cpp, there's a single-file build and I've had success using it with Repl.it: https://neverfriday.com/2013/07/26/learning-jsoncpp/ https://neverfriday.com/2013/07/26/learning-jsoncpp/ Since JSON only has basic data types, it's easy to just parse and retrieve objects, similar to using XML tree parsing libraries back in the day.

StreamBright 7y ago

Yes but JSON what everybody knows. I have seen a presentation about a not too small company where the engineer explained how much hurdle is to have JSON everywhere in their big data platform. I asked him why they do not use something like Avro instead. He just gave me this weird look like I am asking something extremely stupid and could not come up with any reason. Some engineers just have no idea of the alternatives or the pros and cons of chosing different file/message formats.

rurban 7y ago

Then you should be enlightened: Of the myriads of existing transfer or serialization formats, * JSON is still the only secure by default one (if you ignore the two later updates, which made it insecure), * JSON is by far the easiest to parse (small and secure, no references), * is natively supported by Javascript. It has some minor design mistakes, and is not binary (such as msgpack, which is therefore faster), but still is a sane and proper default. Esp. it does not support objects and other extensions, with its initializer problems on MITM attacks, and is properly ended. Unlike XML, YAML, BJSON, BSON, ... which do have major problems, or msgpack, protobuf, ... which do have minor problems. In every application, any dependency to XML should be minimized, contained and preferably eliminated.

jstimpfle 7y ago
> JSON is still the only secure by default one (if you ignore the two later updates, which made it How can a data format be insecure? > JSON is by far the easiest to parse (small and secure, no references), It's relatively easy to parse (while not as easy as a sane well-specified version of CSV would be). But it also offers no structural integrity, which means that you need to augment the parsing code with terribly ugly validation code. (I don't really use JSON frequently, but the attempts at providing a principled validation framework on top of JSON that I've seen were... unsatisfying). > In every application, any dependency to XML should be minimized, contained and preferably eliminated. That's missing the point. It's not about JSON vs XML vs whatever. It's that application logic shouldn't be operating on transport data formats. And even if JSON has a simple representation as runtime data objects (in most scripting languages), that representation is pretty far from ideal compared to a representation tailored towards your specific application logic.
- legulere 7y ago
  > How can a data format be insecure? XML external entities allow for arbitrary file inclusion: https://en.wikipedia.org/wiki/XML_external_entity_attack https://en.wikipedia.org/wiki/XML_external_entity_attack You can make a badly configured XML parser allocate memory until it crashes: https://en.wikipedia.org/wiki/Billion_laughs_attack https://en.wikipedia.org/wiki/Billion_laughs_attack You cannot host user-generated xmls on a domain without making yourself vulnerable to cross site scripting attacks. Browsers will happily execute any javascript you include: https://stackoverflow.com/questions/384639/how-to-include-javascript-in-xml-document https://stackoverflow.com/questions/384639/how-to-include-ja...

deleted 7y ago

[deleted]

paulddraper 7y ago

> In every application, any dependency to JSON should be minimized, contained and preferably eliminated. It depends on the extant to which your program needs to communicate/interoperate with other programs...in particular ones without the same programming language (no Java RMI). Under a "Unix" philosophy, this happens quite a bit.

devwastaken 7y ago

Json is easier for humans to read and modify, and it's more straightforward to work with in languages like python where you don't have to declare types. Json c++ implimentation is large probably because of trying to provide one library that does all the dynamic things json can do, but in a static language. It also has to be safe to parse, so there's that.

CoolGuySteve 7y ago
For config files at least, I've moved entirely to ini format. Python's configparser is handy for machine generating complicated configs and the nesting/list support in JSON adds way more complexity than I need compared to just using comma separated lists as values. This is my entire ini parser for making a std::map<std::string, std::string>. I cast/parse the values at the use site: std::regex section_test("\\[(.*?)\\]"); std::regex value_test("([\\w\\.]+)\\s*=\\s*([^\\+]+(?!\\+{3}))"); while(-1 != getline((char**)&buf, &len, file)) { std::string line(buf); trim(line); if(!line.size() || line[0] == '#') continue; std::smatch match; if(std::regex_search(line, match, section_test)) current_section = match[1].str(); else if(std::regex_search(line, match, value_test)) { std::string key(match[1].str()); std::string value(match[2].str()); info("INIReader: %s::%s = %s\n", current_section.c_str(), key.c_str(), value.c_str()); config[current_section][key] = value; } }

mrlala 7y ago

I can tell this is a C++ thread, heaven forbid you touch anything new within the last 20 years... The JSON obsession is that everything under the moon supports it- and it's a nice easy structure which can be converted to a dynamic or statically typed object with no issues. Plus it's very human readable, so it's great for debugging. If you sit there and ONLY use C++, ok why bother? But many systems need to talk to different systems these days. I can setup a webapi that can communicate with my angular typescript, C#, matlab all through nice easy JSON. Each language just converts it to the object and I'm good to go. The question is why are you so against such a widely accepted format of communication?

Kenji 7y ago

If you don't understand why your application requires marshalling/serialisation, you never wrote a real application. You question goes away as soon as you do.

umvi 7y ago

Well I work on embedded systems that have embedded webservers (think your router's webpage). And what data format is easiest to work with when dealing with webpages and browsers? JSON. Hence, our embedded C++ backend can now easily accommodate and return JSON to the front end using this library. Plus, what data format do you propose for things like configuration files, etc? No matter what format you choose, you are going to need a parser of that format for C/C++.

snagglegaggle 7y ago
I think TOML is better, maybe with some modifications. I dislike that it has a person's name in it though; maybe we retcon it to Text Object Markup Language. TOML has inline and multiline forms for most objects.
- afiori 7y ago
  but in this context TOML and JSON are almost isomorphic (ok, parsing JSON is more unreliable than many realize apparently). As far as I can see the problem is that it is difficult to convert arbitrary JSON object to C++ objects. TOML (I also like it, for different reasons) isn't gonna help with that.
  - snagglegaggle 7y ago
    Wait, everyone is complaining you can't do `object.member.submember`? I don't see the issue with object['member']['sub']. It is slightly more typing, but such is life.
    - umvi 7y ago
      It's better to do object.at("member").at("sub") so that it throws an exception instead of seg faulting if one of the keys doesn't exist
      - snagglegaggle 7y ago

jarjoura 7y ago

Developers obsess about JSON for a simple reason, JSON works natively with Javascript. If you have a backend service written in C++, then communicating over JSON is easily consumed by nodejs and the browser. Of course there are other data transport protocols, XML and ProtoBuf to name a couple off the top of my head, but those are extremely heavyweight. If you want to use JSON for config files, again, they are easy to make human readable and parse. Grab an off the shelf parser and support whatever scenario users may need. Sure we have yaml and ini, or whatever key=value config format your project creates, but you'll be much more on your own with how deep you want to support various value types. As far as your complaint or desire for JSON to be minimized or eliminated as a philosophy, I'm not sure why this is the top comment about a very well made and loved C++ library to parse JSON. Are you hoping your comment will change people's minds about what library to use or whether their system needs JSON at all? I don't really understand your goal.