12 ms·
Writing a simple JSON library from scratch: a tour through modern C++
- jcelerier 5y agoThat struct JSONValue { std::optional<std::string> string; std::optional<double> number; std::optional<bool> boolean; std::optional<std::vector<JSONValue>> array; std::optional<std::map<std::string, JSONValue>> object; JSONValueType type; }; really ought to be a variant, that would simplify things a lot. It'd just be struct JSONValue; using variant_type = std::variant< std::string , double , bool , std::vector<JSONValue> , std::map<std::string, JSONValue> >; struct JSONValue : variant_type { using variant::variant; }; then instead of switches you'd do: std::string_view print_type(const variant_type& jtt) { using namespace std::literals; struct { auto operator()(const std::string&) const noexcept { return "String"sv; } auto operator()(double) const noexcept { return "Number"sv; } auto operator()(bool) const noexcept { return "Bool"sv; } auto operator()(const std::vector<JSONValue>&) const noexcept { return "Array"sv; } auto operator()(const std::map<std::string, JSONValue>&) const noexcept { return "Dict"sv; } auto operator()(std::monostate) const noexcept { return "Null"sv; } } vis; return std::visit(vis, jtt); } which is much safer than switch/cases
- eatonphil 5y agoI do get that it's the type-safe approach but your resulting code looks more complicated than what I did (to me). Still, thanks for sharing so that other people can see for themselves. One of the biggest turnoffs about std::variant to me (which isn't relevant here) is that you can't name the types so the hacks for having multiple of the same type in std::variant look even hairier.
- gravypod 5y agoA nice follow up would also be looking at std::string_view as this has the potential to save a lot of memory.
- eatonphil 5y agoSince I'm unfamiliar with the underlying concept, could you give an example of how you could see that being used?
- gravypod 5y ago`std::string_view` is essentially something like this: struct string_view { char *data; size_t length; } It is a pointer to a subset of an existing string. It has all of the existing features of `std::string` but passing it around is zero copy. As long as the original memory allocation exists your `string_view`s are still valid. Also, it has an implicit constructor from `const std::string` and `const char #` allowing you you define a single function like `Thing ParseThing(std::string_view line)` and accept `char #` and `std::string` as an input. More examples are here: https://abseil.io/tips/1 https://abseil.io/tips/1 # -> *. HN formats bad
- Arnavion 5y ago\* (backslash asterisk) will escape the asterisk. >Also, it has an implicit constructor from `const std::string` and `const char *` allowing you you define a single function like `Thing ParseThing(std::string_view line)` and accept `char *` and `std::string` as an input.
- jcelerier 5y agoevery function where you are reading from a string without writing to it or saving it, you can use std::string_view. Same for vector / array: anything non-owning, read-only should use std::span instead.
- jcelerier 5y agoit's not only type-safe, it uses much less memory, takes less time to construct and does not duplicate the "null" state: - your JSONStruct takes 160 bytes (on 64-bit systems with libstdc++), the variant one takes 56, almost three times less. - it needs std::optional<T> for each member to avoid construction, while std::variant does that by default. But the compiler still have to initialize all these optionals to their default state. - you need on every access two branches: * one for the type switch * one for the optional the variant only has one level of indirection.
- eatonphil 5y agoThat's fair. I assumed it would be clear to anyone who knows C++ better than I that the point of this post wasn't exactly performance. Actually I was most interested in just how simple and readable I could make it. But I didn't say that in the post anywhere. Will add.
- jcelerier 5y ago> I assumed it would be clear to anyone who knows It's just not possible to assume that at all on the internet. Every year I have students who learn from blog posts like this for instance.
- gravypod 5y agoAs someone who does quite a bit of C++ recently: for most uses readability is king. If you did want squeeze out the extra performance you could also do something where you store `std::variant<>` inside a struct and provide an API like this: if (auto int_value = value.i()) { // *int_value == your int } and expose a `std::variant<> &variant();` for use cases where you want the more complex/faster stuff.
- tcbawo 5y agoUsing variant as the storage type while providing a nice public accessor API makes sense given the huge difference in semantics between a struct containing optional fields and a variant.
- quietbritishjim 5y agoRather than using std::visit, you can just use a chain of if statements with get_if [1]. You keep the most important benefits of std::variant but the code is much more readable (IMO). [1] https://news.ycombinator.com/item?id=25316192 https://news.ycombinator.com/item?id=25316192
- jcelerier 5y agoInteresting, I find the list of of ifs much less easy on the eyes. I wonder why that is.
- quietbritishjim 5y agoAgreed that it's a personal preference. If you geniunely have a standalone function, like print_type shown here, then std::visit seems roughly even with std::get_if to me (but I'd tend to use the latter for consistency with my other code). std::visit would clearly come out ahead if you had some sort of complex nested variant and you called out to other visitors. get_if is clearly ahead IMO if you're using a variant in passing in a broader function, and especially if you when you want to use control flow (if you return in an if branch you return from the wider function).
- jcelerier 5y agoYes, this case (checking for a type and returning early) is pretty much my only usage of get_if (and in this case it makes a lot of sense)
- jcelerier 5y agoSince you're interested in readability, you may also like std::format: std::string format_parse_error(std::string base, JSONToken token) { std::ostringstream s; s << "Unexpected token '" << token.value << "', type '" << JSONTokenType_to_string(token.type) << "', index "; s << std::endl << base; return format_error(s.str(), *token.full_source, token.location); } becomes std::string format_parse_error(std::string_view base, JSONToken token) { return format_error( std::format( "Unexpected token '{}', type '{}', index '{}'\n{}" , token.value , JSONTokenType_to_string(token.type) , index , base) , *token.full_source , token.location ); }
- eatonphil 5y agoI was excited to use it! But I don't think it's available in Clang 12 (noted in this post) which is the latest version that comes with latest Fedora. I didn't want to have to spring a newer compiler version than came with the distro. Unless it's just a flag I did not turn on.
- lbhdc 5y agoYou can install it. dnf install fmt-devel
- agent327 5y agoIf you really cared about readability , you'd put your commas where normal human beings put them . Instead of in a silly place like that . Prefering structure over normal communication is not , typically , seen as a good thing .
- Kranar 5y agoYour alternative is undefined behavior due to using an incomplete type in std::map. C++17 allows std::vector to use an incomplete type but std::map does not. Reference this paper for further details: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/n4510.html http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/n451...
- thethirdone 5y agoWhy doesn't the compiler error when compiling if it is undefined behavior? It seems really simple to detect if a type is incomplete.
- jcelerier 5y agoIt's undefined because the standard says so explicitly for these containers, but if you copy-pasted all the code of libstdc++ or libc++'s std::map in a new namespace and gave it your name it would be perfectly defined. It's a bit like naming a global variable _Weeeeeeeeee. It's technically UB because the standard says so, but in practice the compiled code will be fine if your stdlib does not use that variable, the compiler isn't actively looking for elements of the spec to check and enforce, it's just using UB where it can as a tool to optimize
- thethirdone 5y agoI was only really aware of undefined behavior that was relevant to runtime behavior. I wasn't aware using compiler reserved identifiers was undefined behavior; I assumed it was required to error. I don't see the purpose in allowing trivially detectable UB to exist. In this case and in the case of reserved identifiers, I don't see how any optimization could be made using the UB. Noticing the UB allows you to do anything, so you might as well rewrite the program to do nothing.
- account42 5y ago> I assumed it was required to error. They are reserved because the compiler (and OS) writers need identifiers they can use in their own headers without breaking valid user code. Since there is no good way for the compiler to say which headers are user headers, erroring on those symbols would be counter-productive.
- forrestthewoods 5y agostd::variant is an abomination. The idea is stellar. But actually using it is a frustrating nightmare. I prefer the simple way, unfortunately.
- ojeda 5y agoC++ may get pattern matching with P1371, which may make visiting look more reasonable: std::string_view print_type(const variant_type& jtt) { using namespace std::literals; return inpect (jtt) { <double> __ => "Number"sv; <bool> __ => "Bool"sv; ... }; } In any case, `std::variant` should have been a language feature all along, instead of a library one; cf. Rust: enum Value { Number(f64), Bool(bool), ... } fn print_type(v: &Value) -> &str { match v { Value::Number(_) => "Number", Value::Bool(_) => "Bool", ... } }
- jcelerier 5y agoYes, pattern-matching like this would be the bee's knees
- deleted 5y ago[deleted]
- henning 5y agoWhy do people make websites that are so narrow that the code snippets have a horizontal scrollbar? I have a 4k monitor. Modern web design is almost as stupid as modern C++.
- cpp2013wow 5y agoNoobs writing C++ tutorials are a crime against humanity. Compilers too, but I digress. Are you a time traveler from ten years ago or something, what is this even supposed to be.
- nwellnhof 5y agoI know it's only a tutorial but recursive function calls are a bad idea for a general purpose JSON parser. At the very least, limit the recursion depth. Otherwise, it's trivial to make the parser crash with a stack overflow.
- cpp2013wow 5y agoThat would require knowing how to iterate a tree without recursion, tough stuff. Hi dang, love our private chat.
- trzeci 5y agoTitle: > a tour through modern C++ Then: ``` #ifndef JSON_H #define JSON_H // ... #endif ``` Still I didn't get how #pragma once didn't end up in a standard, when every major compiler supports that.
- kitkat_new 5y agoI thought modules would be modern C++ now? Am I missing something?
- eatonphil 5y agoI tried but couldn't get modules working, personally. The various sites online I found differed in how they work in the first place. I couldn't find what the actual state of modules are in Clang.
- pjmlp 5y agoHere, https://blog.ecosta.dev/en/tech/cpp-modules-with-clang https://blog.ecosta.dev/en/tech/cpp-modules-with-clang
- DeathMetal3000 5y agoThat lengthy blog post is a perfect example of everything that is wrong with C++. Using modules should be a trivial part of the language. Instead, we have to take a detour via historical Objective-C (a language already superseded) and there are enough caveats to make a lawyer blush.
- pjmlp 5y agoSome languages care to keep 30 years of code compiling, regardless of the new shinny features.
- int_19h 5y agoIt's so modern (C++20), it's not actually implemented fully by all compilers yet. https://en.cppreference.com/w/cpp/compiler_support#cpp20 https://en.cppreference.com/w/cpp/compiler_support#cpp20
- Uptrenda 5y agoI'd just like to interject for a moment that C++ is literal dog shit and object orientation is the greatest troll perpetuated in the history of software engineering. It can be shown quite trivially that any 'class' can be re-written as a series of functions whose organization is both more aesthetically pleasing, easier to maintain, test, and reason about than a tightly coupled object-orientated, turd pie. Polymorphism is about intentionally creating objects that scramble type purity and make it harder to debug simple programs. While inheritance obfuscates program flow and makes it harder to read code. All of this is great if you like wasting hours over-complicating problems to stay in the job. But actual productivity is about simplicity and not abstract meta-meta class::template::library->vector->classes.
- qmmmur 5y agoCan you give me an example? Not asking disingenuously. I'm a noob and would be keen to learn more.
- villasv 5y agoInternet hygiene best practices tell you to ignore advice that starts up saying that some tech/tool/language is “literally” dog shit. The argument might even be valid sometimes, but the extreme and onesided view damages your learning.
- BenFrantzDale 5y agoI don’t know exactly what the troll you are responding to was getting at, but modern C++ generally agrees with them that OO isn’t the best way to do most things. For better and/or for worse, C++ can’t remove features, so classes aren’t going anywhere, but you don’t have to make big OO hierarchies.
- plorkyeran 5y agoWhile the C++ community is largely moving away from OO, the features added to support OO are still in heavy use. For example, std::function is typically implemented by holding a pointer to a templated subclass of a private abstract base class. There's still inheritance and virtual calls and such going on, and that's just not exposed as part of the API.
- cgrealy 5y ago>std::shared_ptr means I don't have to manage any memory; no more new/delete! (But try as I might to understand std::unique_ptr, I'm just not there yet.) Eh? unique_ptr is just a scoped resource. You create a thing (or take ownership of said thing) and it gets deleted when it goes out of scope. Am I missing something? It should be the first thing you reach for if you need to dynamically create an object (certainly way before shared_ptr).
- kilotaras 5y agoA simple variable would be a scoped resource, unique_ptr can be transferred. { RAII file = ....; // do stuff } // file is no longer open vs unique_ptr<RAII> outer_ptr; { unique_ptr<RAII> file = make_unique(...); // do stuff outer_ptr = file; } // resource still open
- spacechild1 5y agoYour “RAII“ object can also be transferred if it supports move semantics (it really should). Both classes hold a unique resource (a file handle resp. a heap allocated object) that can be moved but not copied and the resource gets freed when the variable goes out of scope.
- cgrealy 5y agoYeah, I’m aware of that. I still don’t think it adds much to the complexity of unique_pet though. It’s certainly less complicated than its ancestor auto_ptr.
- random9e7de4c1 5y agoI am curious to know why every single in function parameters in this project are passed by value. Too much RAM left unused in our modern desktops?
- stockerta 5y agoIn the world of electron applications eating ram like nothing, this pales in comparison.
- benibela 5y agoThere is a good chance that electron comes with a highly optimized JSON parser that is vastly faster the one in the article
- pkolaczk 5y agoWhy do you think pass-by-value takes up more memory? The data gets packed on the stack pretty densely, so as long as the data structures are tiny, that shouldn't be a problem. On the flip side heap allocation is typically prone to some level of fragmentation, which actually does waste memory.
- random9e7de4c1 5y agoI agree with your point for small size data structures. But in general, and in the case of the OP source code, he doesn't even need to copy its arguments to pass them by value, he should pass them via a const reference, and just pass the address.
- spacechild1 5y agoPassing by value can fine, but then the author should use std::move when passing it on to other functions. But you are right that passing by (const) reference is standard practice and less error prone.
- random9e7de4c1 5y ago
- ncmncm 5y agoGood modern C++ uses shared_ptr very, very sparingly. Most programs do not use it at all. Overuse of shared_ptr is called Java Disease. unique_ptr is, at base, very simple. A variable of this type owns whatever it points at. Since only one pointer can own the memory, ownersip passes from one instance to the next "by move". It is easy to know, at any place in the code, which owns the pointed-at thing. It is generally a mistake to expose unique_ptr in an interface. It is an implementation primitive. Very often a class has only just one data member, a unique_ptr to a private-typed object in storage. Users of the type treat it as an ordinary value, but don't need to know anything at all about the implentation object. All the code that operates on it can be off in a ".cc" file, and you can change it without need to rebuild clients. A vector of unique_ptr is another common class member, to hold ownership of a bunch of objects. Once in a great while you want to take over control of how the memory is released when an owning unique_ptr goes out of scope. Maybe the memory was allocated with mmap and must be freed with munmap. This is easy to do with one-liner functions, or lambdas.
- yakubin 5y agostd::unique_ptr still suffers from The RAII Syndrome, where freeing memory, instead of being a constant-time operation, becomes linear, when one std::unique_ptr owns a struct, which owns other std::unique_ptrs and so on. Another problem with The RAII Syndrome is encouraging large numbers of microallocations, over allocating memory in bulk, increasing the number of places in your program where something may fail. And although not strictly RAII, a related issue with the C++ model of constructors, destructors and whatnot (The Rule of N, where N goes to inf as the version of C++ goes to inf), is that when you use std::vector::push_back, and the std::vector needs to be resized, instead of executing a simple memcpy (or equivalent), move constructors for every element need to be called.
- tialaramex 5y ago> instead of executing a simple memcpy (or equivalent), move constructors for every element need to be called. But does that actually result in different machine code? If the compiler ends up concluding that you want to move all this contiguous data and by the as-if rule it does memmove() then the fact that notionally a bunch of other C++ code ran doesn't matter. I agree it's theoretically cleaner to be able to just say "You can move these, it's fine" about a type and know that optimisation falls out automatically, but the end result machine code may be identical.
- ahaferburg 5y ago> std::shared_ptr means I don't have to manage any memory You really should have left this out. It's very, very distracting, and makes me question if I even want to read the article. shared_ptr is one tool to manage memory. It certainly doesn't mean you don't have to manage it, and it doesn't mean you should reach for it most of the time. > I don't do a lot of C++ so I wanted to get a sense for what it can look like today. I think this would be a great intro for the post.
- spacechild1 5y agoNo offence, but you should make it clear in the beginning that you are not too familiar with (modern) C++. This blog post might teach how to write a simple JSON parser, but it does not teach how to write good modern C++. There a many red flags which others have already pointed out.
- DennisL123 5y agoArguably, a Json type can be implemented with just std library thing in a less involved way: https://medium.com/@dennis.luxen/breaking-circular-dependencies-in-recursive-union-types-with-c-17-the-curious-case-of-4ab00cfda10d https://medium.com/@dennis.luxen/breaking-circular-dependenc...
- klaussilveira 5y agoIf you are looking into learning Modern C++ through library development, I highly recommend the following: https://pragprog.com/titles/lotdd/modern-c-programming-with-test-driven-development/ https://pragprog.com/titles/lotdd/modern-c-programming-with-... https://stroustrup.com/Tour.html https://stroustrup.com/Tour.html
- benibela 5y agoIf you want to have that in production, you should run a test suite on it. There is a popular test suite by nst. Although some go too far. I just had to write my own JSON parser (resp. rewrite one), because the parser I was using did not pass the test suite I had.