4 ms·
This is premature optimization. A very small percentage of APIs return so much data that you need to worry about stream parsing. The vast majority of APIs in th
by stickfigure 5y ago
This is premature optimization. A very small percentage of APIs return so much data that you need to worry about stream parsing. The vast majority of APIs in the world, including probably yours, returns structures that are easy to manage in memory.
If you render this `{"MyMessageType":{...}}` structure, people who live in statically typed languages will hate you. At best you've created a union type with a truly vast number of possible cases. In practice you've forced this to be read as `Map<String, JsonNode>` and dynamically interpreted.
Unless your problem domain routinely streams gigabytes of data across an API, don't do this. And if it does stream gigabytes of data across an API, figure out how to chunk the data and still don't do this.
- Zababa 5y agoHow is this premature optimization? Since it's equivalent to the alternative, it should be known that this way has an advantage.
- stickfigure 5y agoAdding a ton of structural noise is not "equivalent". This pattern will be repeated throughout your hierarchy, and your clients will have to deal with this: GET /assets/123 { "Asset": { "owner": { "Person": { "name": "Bob", "contact": { "Address": { "street1": "blah", ... } } } } } } On the other hand, the traditional approach: GET /assets/123 { "owner": { "type": "Person", "name": "Bob", "contact": { "type": "Address", "street1": "blah", ... } } } } This is much more sane to navigate. And when you end up making some previously-static node polymorphic (as happens), the structure does not change.
- FeepingCreature 5y agoI think making a previously-static node polymorphic in a way that doesn't drop any required field is actually quite rare. But also, I just think the first one is better.
- stickfigure 5y agoIt's much easier to migrate adding/dropping fields than to make structural changes. Adding/dropping fields is the easiest form of migration. I honestly have a hard time believing that you prefer: asset.Asset.owner.Person.contact.Address.street1 to: asset.owner.contact.street1 If so, I think you should really announce this first and foremost in your article so that everyone can take a proper measure of your sanity :-)
- FeepingCreature 5y agoI mean, I don't actually believe you that all those levels are polymorphic. :-) IME, we have one, at most two polymorphic fields per message. And for an example chosen deliberately to be annoying, I still like the deeper tree better- at least it advertises that something unusual is going on.
- Spivak 5y agoHow does { "type": "huge union of types" } help you here? You still have to read the whole thing as a generic Map just to find the type key and then dynamically unmarshall to your real object. Like what even is the return type for parseJSONBlobToOneOfFiftyMessageTypes(String json)? Guess you could return a generic APIMessage and a type object so the caller can cast it but now you're back to playing dynamic games.
- agency 5y agoComing from TypeScript, having an explicit discriminator field (like "type") is a lot easier to deal with[1] (which to be fair is compatible with the last example given in the OP where they have a "type" field and a generic "data" payload field). [1] https://www.typescriptlang.org/docs/handbook/2/narrowing.html#discriminated-unions https://www.typescriptlang.org/docs/handbook/2/narrowing.htm...
- TheCoelacanth 5y agoI don't think it's any easier. Narrowing works the same way with the in operator. If A, B and C each have a single property with the same name: function f(x: A | B | C) { if ('A' in x) { // x is narrowed to A } else { // x is narrowed to B | C } }
- stickfigure 5y agoIn Javaland, Jackson and GSON both have easy ways of deserializing polymorphically based on a type property.
- FeepingCreature 5y agoI've ran into business-relevant slowdowns due to this issue. I live in a statically typed language. Your "at best" is actually very straightforward. Also, sum types exist. The problem is that by the time you stream gigabytes of data, if you have done it the wrong way, you now have to add a new API revision for the right format, update your whole ecosystem - if the protocol is even in your control. If you did (or specced) it right from the start, you'd only have to adjust the clients at that point, and it would have been just as easy up until then.
- kevincox 5y agoI don't know about other languages but this is perfectly fine in Rust. It is actually the default formatting for enums (unions) and does streaming parsing. #[derive(Debug,serde::Deserialize)] struct MyMessage { data: i32, } #[derive(Debug,serde::Deserialize)] enum Example { MyMessageType(MyMessage) } fn main() { dbg!(serde_json::from_str::<Example>(r#"{"MyMessageType":{"data":5}}"#)); } Running: [src/main.rs:12] serde_json::from_str::<Example>(r#"{"MyMessageType":{"data":5}}"#) = Ok( MyMessageType( MyMessage { data: 5, }, ), ) https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=cc01d0de9934266ba4b8fb6eb123b96c https://play.rust-lang.org/?version=stable&mode=debug&editio... I'm not sure why you are forced to read it into a Map<String, JsonNode> unless you are doing your parsing without type information in which case any solution is going to require a map.
- selfhoster11 5y ago1. I live in a statically typed language, and honestly what's purposed in the article is not such a burden. It'd much prefer to receive the object type data via that Content-Type header or equivalent, but this will do as well. 2. If you're sending me gigabytes of JSON, something has gone horribly wrong. Split it up, or send me the result in a more suitable format.
- vba616 5y ago>If you're sending me gigabytes of JSON, something has gone horribly wrong. Split it up, or send me the result in a more suitable format. Gigabytes of XML, coming right up!
- selfhoster11 5y agocue incoherent, horrified scream
- deleted 5y ago[deleted]
- deleted 5y ago[deleted]