11 ms·
JSON for Modern C++ version 3.10.0
- esprehn 5y agoThis doesn't appear to be all that "modern": it doesn't support string_view for lookup without extra allocations, and it doesn't support unordered_map (or even use it as the default) for objects. It seems they're targeting C++11?
- howolduis 5y agopeople love to complain
- q-rews 5y agoPeople love correct statements. C++11 was release 10 years ago and there are 3 newer versions.
- rualca 5y ago> people love to complain Do you find it unreasonable to point out that what's been advertised doesn't match what's being offered?
- nlohmann 5y agoAs I tried to describe earlier: "modern C++" is not necessarily "using the latest standard", but rather "C++ since it was updated with C++11 (and later)".
- rualca 5y agoAiming at C++11 is not a reasonable definition of "modern", as it was C++'s second ISO standard that was published a decade ago and which has since seen between two and three major updates (depending on the opinions on c++14)
- nlohmann 5y agoIt's not the "aiming at C++11", but rather "Write code that does not look odd in a code base that is using constructs from C++11, C++14, C++17, etc." - The library uses C++11 to implement an API that should not feel weird when used together with other containers from the STL.
- bobnamob 5y agoIn C++ land "modern" has become synonymous with post 11. Effectively a domain specific definition. Reasonable considering the difference between pre and post 11. Pre and post 20 will probably be treated similarly in a decade
- maleldil 5y agoC++11 was a huge shift in how C++ is written, and term coined for "code written using the new techniques" was "Modern C++". Whether you think that term should instead mean "the latest C++ version" is a different matter altogether.
- beached_whale 5y agoThe C++11 is still common enough that if you want a large usage group it's the target. However, it's parsing to a DOM and doesn't use features like PMR, so the language features needed won't be more than std containers and stuff.
- pjmlp 5y agoSpecially among the C+ community groups.
- beached_whale 5y agoIt is changing and I saw some survey’s showing 17 around 1/2 the shops. I know for my JSON library it wouldn’t be possible without C++ 17 so it’s a loss of eyes
- pjmlp 5y agoYeah, thing is how much C++17 is actually used beyond the language version. For example, C++/WinRT is a C++17 library, yet plenty of samples use C style strings and vectors.
- beached_whale 5y agoI use a lot of core C++17 isms. They make some things doable and others clearer. But I think C++17 really helped a lot in generic programming in addition to the containers it introduced.
- quietbritishjim 5y agoI think it makes sense to target C++11. Just that you shouldn't then call the library "modern".
- nlohmann 5y agoThere is also a SAX parser. But to be honest, I have not yet played around with PMR.
- pjmlp 5y agoFor many even using RAII, which even Turbo Vision for MS-DOS made use of, is already "modern".
- MauranKilom 5y ago> and it doesn't support unordered_map (or even use it as the default) for objects. std::unordered_map has massive overhead in space and time. Maybe I misunderstand your idea, but outside of extremely niche use cases, std::unordered_map is basically never the right tool (unless you don't care in the slightest about performance or memory overhead).
- agent327 5y agoSource? A hash map should in principle be faster than a tree, with far fewer comparisons, and against a computed hash value instead of the whole object, in principle.
- einpoklum 5y agoWell, std::unordered_map is faster than than std::map, but it is still slow, since it uses lists for its buckets, with lots of dynamic allocation. See also: https://stackoverflow.com/a/42588384/1593077 https://stackoverflow.com/a/42588384/1593077 and the link there. Not sure that's what GP meant though.
- agent327 5y agoI don't believe it has to, some kind of bucket-optimisation (multiple elements per allocation, instead of one element per allocation) should also meet all constraints the standard sets for this type. And I don't like it when people use ridiculous hyperbole to describe what is in reality barely perceptible overhead, so it would be good if the person I originally responded to came out and defended his POV. Hopefully using actual numbers instead of agitated handwaving and screaming...
- beached_whale 5y agoAbseil has a btree based hash map and it performs much better. I think it maintains the same constrains(iterator invalidation/exception safety...) as std::unordered_map. The issue is that the implementors won't change their implementation of unordered_map as it's an ABI break, to say it simply. It could be better. But also, there are other tools like flat maps/open addressing that are not done in the std library.
- mike_hock 5y agoIt uses the ordering of std::map for its own comparison operators and hash function.
- nlohmann 5y agoThe development started in 2013 when C++11 was still modern. Since then, the term "modern C++" is, to my understanding, a synonym for "C++11 and later". Of course, some code may look dated compared to newer C++ constructs, but the main goal is to integrate JSON into any C++ code bases without making it look odd. The string_view lookup is nearly done, but I did not want to wait for it, because it would have delayed the release even more. I'm also working on supporting unordered_map - using it as container for objects would be easy if we would just break the existing API - the hard part is to support it with the current (probably bad designed) template API.
- adzm 5y agoYou can provide alternative collections if needed by templating basic_json; std::map is just a default. Personally I like using a flat_map which is an ordered map built on top of a vector, which provides nice cache locality and is generally even faster than a hash map / unordered_map except for extremely large objects.
- versatran01 5y agoI prefer boost json
- beached_whale 5y agoIt does seem to have a very similar interface to this and has the performance of RapidJSON. So a nice balance in that regard.
- darknavi 5y agoHow do people go about integrating boost these days? For any cross platform projects I use CMake and Boost seems like a _very_ scary library to try and build/ship on multiple platforms.
- chrisjericho 5y agoI use CMake almost exclusively (even for Windows) and using "find_package(Boost REQUIRED)" works most of the time. A lot of common boost libraries (algorithm, geometry) are header only and are not much of a hassle to include. IIRC, you would need to modify the statement to "find_package(Boost REQUIRED COMPONENTS filesystem)" if you want to use boost dependencies that need a separate .dll/.so (in this case boost::filesystem). However I rarely need to use these, so I may be wrong.
- nly 5y agoCMake supports Boost out of the box, you just need to set BOOST_ROOT to your Boost install path. With proper CI it's not a big deal
- rualca 5y agoWhat exactly makes Boost scary to ship? I understand that Boost requires some attention to build right across platforms (see zlib support in Windows) but other than this it's pretty straight forward to add a FindBoost statement to your code, and move onto other things.
- Johnnynator 5y agoYou should be able to build and include boost json as a standalone subproject in CMake if you are using C++17. (Or also possible to use as header only lib) It gets far more complicated with C++11, since you also need a ton of other boost modules there. For more Details you can read the Readme of it. https://github.com/boostorg/json https://github.com/boostorg/json
- _3u10 5y agoCool project. Others may also be interested in simdjson which parses at about 4GB/sec. https://github.com/simdjson/simdjson/blob/master/doc/basics.md#the-basics-loading-and-parsing-json-documents https://github.com/simdjson/simdjson/blob/master/doc/basics....
- joshxyz 5y agoyes another great one these projects are just commendable!
- adzm 5y agosimdjson is an awesome project. However, it's analogous to a SAX parser for XML vs a DOM model. If that is all you need (or if you are building something like a DOM yourself) then it is pretty much impossible to beat. Having an entire JSON document parsed and in memory at once though has a bunch of advantages for a friendlier API.
- Narishma 5y ago> which parses at about 4GB/sec. That's a weird thing to say. Doesn't that depend on what hardware it's running on?
- deleted 5y ago[deleted]
- TeeMassive 5y ago> The CSV format itself is notoriously inconsistent So what? The format is so simple that the inconsistencies doesn't even matter and hell the format can be inferred from the data and that can even be automated. Unlike JSON/XML, most software can automatically make sense of CSV; JSON needs a developer to understand it to transform it into workable data. > CSVs often begin life as exported spreadsheets or table dumps from legacy databases, and often end life as a pile of undifferentiated files in a data lake, awaiting the restoration of their precious metadata so they can be organized and mined for insights. CSVs are the de facto way to export spreadsheets and if you have to do this then the problem are that you are still using spreadsheets in your data flow, not CSVs. Same goes for legacy systems. Also CSVs comes from places where having more sophisticated data serializers is near impossible and the time to code, compute, parse and analyze CSVs is trivial compared to any other format out there. > I’ve spent many years battling CSVs in various capacities. As an engineer at Trifacta, a leading data preparation tool vendor, I saw numerous customers struggling under the burden of myriad CSVs that were the product of even more numerous Excel spreadsheet exports and legacy database dumps. As an engineering leader in biotech, my team struggled to ingest CSVs from research institutions and hospital networks who gave us sample data as CSVs more often than not. Of course if you are working in biotech then you'll work with scientists (horrible coders most of the time) and legacy systems because it's a domain that's very adverse to change that might break stuff. And it's not the CSV format's fault if it's used as the wrong tool for the job. > Another big drawback of CSV is its almost complete lack of metadata. Again, emphasis on the right tool for the job. For simple tasks metada are not needed.
- Stratoscope 5y agoYou mistakenly replied on the wrong thread. Here's the one you meant to reply on: https://news.ycombinator.com/item?id=28221654 https://news.ycombinator.com/item?id=28221654 Your quote "The CSV format itself is notoriously inconsistent" comes from this page which the thread discusses: https://www.bitsondisk.com/writing/2021/retire-the-csv/ https://www.bitsondisk.com/writing/2021/retire-the-csv/
- TeeMassive 5y ago
- nikki93 5y agoThe API side of this library is pretty good. I've generally avoided it due to the compile time, and preferred rapidjson due to both better performance and compile time if I wanted a C++ API, or cJSON if a C API is fine and I really just want an extremely low compile time (lately I've been playing with a live coding C++ context for a game engine, as a side project, I use cJSON there; rapidjson on main project though) (cJSON's tradeoff is it sticks with a single global allocator that you can only customize in one location, and also seems to perf slightly lower on parse). The cJSON's C API or rapidjson's somewhat less ergonomic API has felt fine because I usually have read / write be reflective and don't write that much manual JSON code. Specifically for the case where compile time doesn't matter and I need expressive JSON object mutation in C++ (or some tradeoff thereof), I think this library seems good. Another big feature (which I feel like it could be louder about -- it really is a big one!) is that you can also generate binary data in a bunch of formats (BSON, CBOR, MessagePack and UJBJSON) that have a node structure similar to JSON's -- from the same in-memory instances of this library's data type. That sort of thing is something I've desired for various reasons (smaller file types with potentially better network send time for asynchronous sending / downloads (for real-time multiplayer you still basically want to encode to something that doesn't embed field string names)). I do think I may end up doing it at one layer above in the engine level and just have a different backend other than cJSON etc. too though...
- ephaeton 5y agoPlusses: as a user of nlohmann-json, I also like the support for json pointer, json patch and json merge patch which comes in handy at times. I like their way of handling to/from_json (declare functions with appropriate signature and namespacing and the lib will pick them up seamlessly in implicit conversions). "standard" container facades are appreciated. Minusses: although I'd like a way to append a (C-)array into a JSON-array "at once" and not iteratively (i.e., O(n) instead of O(n log n)). Also, lack of support for JSON schema is .. slightly annoying.
- nikki93 5y agoYeah the support for pointer / patch etc. is a definite plus. The customization point thing I tend to do with my own customization points, but it's pretty good if a bunch of libraries settle on a standard customization point (eg. I think serde in Rust has achieved that a bit due to the ecosystem there) (definitely want it to be static). I didn't realize that about the array complexity. Can you not just initialize a JSON array of N elements in O(N) (conceding a `.reserve(N)` style call beforehand if required)? rapidjson is pretty good about that sort of thing, cJSON's arrays are linked list so I basically think of its performance as at a different level and it's mostly about compile time for me.
- SloopJon 5y agoThis is the library I settled on for a project a couple of years ago. Although I was intrigued by the magic of expression templates, my usage is actually pretty straightforward and boring. One thing it has that some of the other libraries I evaluated lack is the ability to parse numeric strings yourself, which I wanted for a decimal floating-point type. There are lots of other libraries, and surely some faster ones, but I haven't felt any need to change. I was nervous when I got an inexplicable overload ambiguity in xlclang++ for AIX, but it went away when I grabbed a newer version of the library.
- iamvnraju 5y agoI remember having used nlohmann's single header JSON parser library around 2018 on a client app and found it to be pretty nifty. For REST, I had had used Boost Beast.
- newusertoday 5y agooff topic, is there any tool which can be used to query json file that also has autocomplete functionality for keys?
- modeless 5y agoHuh, I was just in some code that uses this library a few days ago. I wish it included a base64 implementation for more efficiently encoding arrays of bytes or floats. JSON is inefficient for large arrays of numbers.
- jcelerier 5y agoWhy wouldn't you use a base64 library ? This is prime library bloat mindset. Base64 has nothing to do in a JSON lib; there are a ton of implementations out there.
- deleted 5y ago[deleted]
- modeless 5y agoA base64 implementation is very small, but not completely trivial to the point where you should implement it yourself, or paste in a one liner from Stack Overflow. It's widely used with JSON. C++ has no package manager to make it easy to import tiny libraries for purposes like this, so C++ libraries should be a little more self contained than npm packages or rust crates. It would absolutely be reasonable to include a base64 implementation in a C++ JSON library for those reasons.
- tsbinz 5y agoUh, it has plenty of methods for more compact encodings? https://github.com/nlohmann/json#binary-formats-bson-cbor-messagepack-and-ubjson https://github.com/nlohmann/json#binary-formats-bson-cbor-me...
- modeless 5y agoSure, if you want to change the format of the whole message. If you're communicating with something that accepts JSON and you can't or don't want to change that, then having a more efficient option is nice.
- knorker 5y ago"Modern" C++. The second paragraph talks about setting a #define before including a header, as if this is still the 90's.
- mike_hock 5y agoUnfortunately, this antipattern has persisted way beyond the 90s. But, you know, if the only way the standard provides to achieve a certain goal is a shitty hack, people will use the shitty hack.
- a-dub 5y agowhat's wrong with #define(s)?
- einpoklum 5y agoThe compiler is blind to them, so they can't be reasoned about / carried forward anywhere / allowed for in one way or another. Moreover, because of `#define`s, the effect of including a file can be entirely different, so a compiler can never say "Oh, I know what's in this include file already, I don't need to include it again in other translation units". Of course #define's have many uses, and at present cannot be avoided entirely, but C++ has been making an effort to have other mechanisms put in place so as to obviate the use of #define's In particular, look for CppCon talks about Modules.
- a-dub 5y agointeresting. so traditionally the preprocessor has been used to work around incompatibilities across various compilers/platforms and to completely enable/disable features at compile time. what's the new thinking there? can you build big projects across clang/g++/intel without preprocessor workarounds now? has the number of viable compilers in use dropped and compatibility amongst those that live on increased? how about stuff like debugging/instrumentation? or is the current thinking on all that stuff to always build with all of it and enable/disable via runtime branch? (along with some argument that configuration at build time was more trouble than it's worth on modern spacious/fast machines)?
- einpoklum 5y agoFrom the repository's README: > Speed. There are certainly faster JSON libraries out there. However, if your goal is to speed up your development by adding JSON support with a single header, then this library is the way to go. It is essentially admitting to being slow(er). We are not supposed to pay for code modernity with performance. In fact, it should be the other way around. Does someone know the innards of this library as opposed to, say, jsoncpp or other popular libraries, and describe the differences?
- nlohmann 5y agoThe library aims at making anything related to JSON straightforward to implement. For some applications, this is a good compromise. The comment in the README is like the answer to a FAQ "How fast is it?"
- einpoklum 5y ago> This is a good compromise You're making the false assumption that there is a compromise to be made - but there isn't. A modern JSON library should be the fastest (or about the same speed as the fastest), and vice-versa.
- lazypenguin 5y agoThe comments here are uncharitable. For some reason when someone posts a C++ project all the nitpickers come out of the woodwork. Is it as fast as simdjson? No but simdjson is read only. Is it as memory efficient as rapidjson sax writer? No, but it’s a hell of a lot more ergonomic. Is it “modern” C++? Yes it is, just open the source and you can see you for yourself. Nitpicking which standard is allowed to be call modern is silly. I’d argue modern C++ is a style or way of thinking. Would I use it for all projects? Maybe not but this is a good, high quality library. Dear @nlohmann and contributors. Thank you for this library, I’ve used it in the past and appreciate your work and efforts.
- microtherion 5y agoAn awesome library, wonderfully convenient to use. I particularly like being able to use the same code for JSON and CBOR.
- bdavis__ 5y agobeen using nlohmann::json for several years. changing from something else (no reason to name it), has proven to be a good decision. away from SFO, "Modern C++" means C+11 and above. mr. lohmann, much thanks for an easy to use, robust, complete, and performant library provided for the world to use. every piece of software has tradeoffs, you have developed a library that is not perfect, rather it is the one people use. nicely done.