3 ms·

> Comment from that issue: "For example, ElasticSearch returns a 8,683KB document, I deser it into Value and the next RAM reading gives me delta of 98,484KB of

> Comment from that issue: "For example, ElasticSearch returns a 8,683KB document, I deser it into Value and the next RAM reading gives me delta of 98,484KB of RAM use. That's more than 10x the original size." I think this is misleading. `serde_json` is designed to parse JSON documents directly into your user-defined structs. Thus the raw `Value` type isn't optimized for direct manipulation. That said, what do you expect? The `Value` enum has the `Value::Array` variant that contains a `Vec<Value>`, which is 24 bytes in itself. One for the heap pointer, one for the length, and one for the capacity. Because of alignment requirements the enum discriminant must thus also be aligned at the 8-byte boundary. This gives us a 32-byte type. It is possible to reduce the variant to 16 bytes by making the value type `Box<[Value]>` instead. Ignoring the `Value::Object` variant for a second, that would make `Value` a 24-byte type. However, you've now also made the type immutable. It's a tradeoff. 8 bytes for mutable values. Again, do remember that deserializing into `Value`s is _not_ the primary path when working with JSON through `serde_json`.

lenkite 4y ago
Yes, I fully understand how enum variants are implemented under the hood. My point was in response to the enthusiastic use of enum variants which generally causes excessive memory to be consumed for even moderate inputs. Folks are surprised by this and then re-write their code to avoid enum variants or use pointer tagging. Enum variants should come with a STRICT warning in the Rust book and Rust reference that their real-world use should be incorporated very carefully. Most proponents of Rust tend to never mention their costs or caveats. They are most certainly NOT a zero-cost abstraction and tend to trip up lots of programmers. "Thus the raw `Value` type isn't optimized for direct manipulation." Maybe this statement this should be explicitly mentioned in the documentation: DO NOT USE `serde_json::value::Value` for moderate or large sized JSON inputs in production! Stack overflow answers merrily recommend the use of `Value` to a get a piece of data out.
- kajaktum 4y ago
  How else would you implement tagged union? Rust already optimize away the discriminant when its Option<NonZero>

lenkite 4y ago

Yes, I fully understand how enum variants are implemented under the hood. My point was in response to the enthusiastic use of enum variants which generally causes excessive memory to be consumed for even moderate inputs. Folks are surprised by this and then re-write their code to avoid enum variants or use pointer tagging. Enum variants should come with a STRICT warning in the Rust book and Rust reference that their real-world use should be incorporated very carefully. Most proponents of Rust tend to never mention their costs or caveats. They are most certainly NOT a zero-cost abstraction and tend to trip up lots of programmers. "Thus the raw `Value` type isn't optimized for direct manipulation." Maybe this statement this should be explicitly mentioned in the documentation: DO NOT USE `serde_json::value::Value` for moderate or large sized JSON inputs in production! Stack overflow answers merrily recommend the use of `Value` to a get a piece of data out.

kajaktum 4y ago
How else would you implement tagged union? Rust already optimize away the discriminant when its Option<NonZero>