4 ms·

The trick with Serde is that it decodes straight into a native Rust struct. You get most of validation for free (it also nicely takes advantage of Rust enums wi

by pornel 5y ago

The trick with Serde is that it decodes straight into a native Rust struct. You get most of validation for free (it also nicely takes advantage of Rust enums with data), and struct access is maximally fast. A generic JSON decoder would give you a dynamic structure that can contain anything, and then you'd have to pick it apart.

UK-Al05 5y ago
Most deserializers in typed languages allow you to deserialize straight into structs or typed objects... Serde just doesn't use reflection to do it.
- Taywee 5y ago
  C++ isn't super easy about it, though. The best C++ json library (in my opinion), nlohmann's json, still requires you to define to_json and from_json for your types to use them. It's not too bad as a one-off, but when you have dozens of types, it gets really tedious compared to Serde, and managing std::variant with it is way more annoying than Serde with Rust enums.
- Jweb_Guru 5y ago
  APIs like GSON and Jackson have a lot more footguns, because (1) they attempt to (de)serialize types whether or not they were ever intended to be (de)serialized, (2) they are missing information about important constraints due to losing information at runtime (the excuse usually used here is that "they don't do validation" when in reality they are violating expectations of package modularity and encapsulation in a pretty blunt way). For example, they will happily ignore non-null constraints on your fields. In practice, this is a major annoyance and ends up making you very paranoid about (de)serializing any structure you don't have full control over. Performance of a lot of the reflection-based parsing APIs also leaves something to be desired, which means that projects with more stringent performance requirements often have to resort to stuff like Avro. This still happens with serde, but much more rarely--besides having more compile-time information at its disposal and needing to allocate less, it also provides relatively straightforward hooks to achieve things like zero-copy deserialization for strings where the whole buffer is available at once.
  - dcow 5y ago
    I’m pretty sure the non-null issue is fixable with a single config option, at least in GSON IIRC.
    - Jweb_Guru 5y ago
      I don't believe it's fixable at all if you're using someone's third-party library that wasn't designed for GSON (unless something has changed a lot since I last used it). The annotation has to be stuck on the original field, AFAIK. And similar stuff apply to other invariants on the type. I believe these kinds of issues are somewhat fundamental when you decide you're going to start (de)serializing structures that never thought about whether or how they'd like that to happen.
      - unnah 5y ago
        In Jackson, you can use mixin classes to define annotations for 3rd party libraries.
berkut 5y ago
However, if you just want a very minimal subset of a JSON response (i.e. a 'status' string field out of say 20 other values in a map), is it not better to just look for one field, as opposed to have to create an entire struct in Rust to represent the full response? How does Serde work with optional extra fields? I know you can use Option<>, but that implies you know they exist - what happens if fields get added in the future silently? I guess the schema changes then, which isn't good, but that might happen?
- deleted 5y ago
  [deleted]
- nemothekid 5y ago
  An "entire" Rust struct containing a single string field is just a struct with a single field. You don't have to specify the fields you don't need and it's unlikely your handrolled parser would be faster than serde (unless you are using memchr or something, then in that case you aren't parsing JSON).
- rokob 5y ago
  You can add a catch all field that captures those in a map. You can also choose to ignore things you don’t explicitly mention. It is up to you how strict you want the parsing to be. So you can define a struct with one status field and the right annotations and you will get exactly what you describe without having to write the code and it will still be almost as fast as doing that parsing yourself.
- kroltan 5y ago
  As far as I understand, unmapped fields are simply ignored, so you can write your code like #[derive(Deserialize)] struct Response { status: u64 }
- codys 5y ago
  If you only want a single field in json, you define a struct with just that single field. Serde does have the ability to return an error about unknown fields, but it is not enabled by default. This also informs your second question. If new optional fields in json are added, unless you tell serde to complain about them, it won't, and you can add the new Option at your leisure.

by pornel 5y ago

UK-Al05 5y ago

Most deserializers in typed languages allow you to deserialize straight into structs or typed objects... Serde just doesn't use reflection to do it.

Taywee 5y ago
C++ isn't super easy about it, though. The best C++ json library (in my opinion), nlohmann's json, still requires you to define to_json and from_json for your types to use them. It's not too bad as a one-off, but when you have dozens of types, it gets really tedious compared to Serde, and managing std::variant with it is way more annoying than Serde with Rust enums.
Jweb_Guru 5y ago
APIs like GSON and Jackson have a lot more footguns, because (1) they attempt to (de)serialize types whether or not they were ever intended to be (de)serialized, (2) they are missing information about important constraints due to losing information at runtime (the excuse usually used here is that "they don't do validation" when in reality they are violating expectations of package modularity and encapsulation in a pretty blunt way). For example, they will happily ignore non-null constraints on your fields. In practice, this is a major annoyance and ends up making you very paranoid about (de)serializing any structure you don't have full control over. Performance of a lot of the reflection-based parsing APIs also leaves something to be desired, which means that projects with more stringent performance requirements often have to resort to stuff like Avro. This still happens with serde, but much more rarely--besides having more compile-time information at its disposal and needing to allocate less, it also provides relatively straightforward hooks to achieve things like zero-copy deserialization for strings where the whole buffer is available at once.
- dcow 5y ago
  I’m pretty sure the non-null issue is fixable with a single config option, at least in GSON IIRC.
  - Jweb_Guru 5y ago
    I don't believe it's fixable at all if you're using someone's third-party library that wasn't designed for GSON (unless something has changed a lot since I last used it). The annotation has to be stuck on the original field, AFAIK. And similar stuff apply to other invariants on the type. I believe these kinds of issues are somewhat fundamental when you decide you're going to start (de)serializing structures that never thought about whether or how they'd like that to happen.
    - unnah 5y ago
      In Jackson, you can use mixin classes to define annotations for 3rd party libraries.

berkut 5y ago

However, if you just want a very minimal subset of a JSON response (i.e. a 'status' string field out of say 20 other values in a map), is it not better to just look for one field, as opposed to have to create an entire struct in Rust to represent the full response? How does Serde work with optional extra fields? I know you can use Option<>, but that implies you know they exist - what happens if fields get added in the future silently? I guess the schema changes then, which isn't good, but that might happen?

deleted 5y ago
[deleted]
nemothekid 5y ago
An "entire" Rust struct containing a single string field is just a struct with a single field. You don't have to specify the fields you don't need and it's unlikely your handrolled parser would be faster than serde (unless you are using memchr or something, then in that case you aren't parsing JSON).
rokob 5y ago
You can add a catch all field that captures those in a map. You can also choose to ignore things you don’t explicitly mention. It is up to you how strict you want the parsing to be. So you can define a struct with one status field and the right annotations and you will get exactly what you describe without having to write the code and it will still be almost as fast as doing that parsing yourself.
kroltan 5y ago
As far as I understand, unmapped fields are simply ignored, so you can write your code like #[derive(Deserialize)] struct Response { status: u64 }
codys 5y ago
If you only want a single field in json, you define a struct with just that single field. Serde does have the ability to return an error about unknown fields, but it is not enabled by default. This also informs your second question. If new optional fields in json are added, unless you tell serde to complain about them, it won't, and you can add the new Option at your leisure.