5 ms·

I didn't really read JSON as the core issue to begin with. In practice, JSON serialization libraries are used to map between domain objects and a JSON represent

by Twisol 6y ago

I didn't really read JSON as the core issue to begin with. In practice, JSON serialization libraries are used to map between domain objects and a JSON representation. The author notes that this really breaks into two problems, one of which is specific to JSON and the other of which is truly dependent on what exactly your domain model looks like. The whole point is that no single library can solve a problem of unbounded (domain-dependent) complexity, and yet we rely on libraries like GSON to do that for us anyway. This leads to some problems, which the author calls out. > This is actually a very trivial problem indeed! I'm not sure I agree. The author specifically mentions the problem of coupling field names to names in the representation, such that an otherwise safe refactoring ends up breaking format compatibility. The more general problem is a strict coupling between the structure of your domain objects and the structure of your representation. For instance, if your domain models cyclically-related objects, you have to decouple these structures somehow -- JSON cannot directly represent cycles. Sometimes you can tell whatever domain mapping library you're using to cut cycles in certain ways, but the author rightly calls this a Faustian bargain. Retaining full control over these representational issues obviates a lot of the problems you'd otherwise face when adapting a library to your domain.

whalesalad 6y ago
You can use field names or you can encode the type alongside. Ex: { “birthdate”: [“date”, “01-02-1991”] } And then in your codebase you know that all the values in your data are actually a list of (type, value). In this specific case I’d decide a “date” as “mm-dd-yyyy” These are trivial problems. We have all solved them. The root of the problem is wanting to put all your trust/hope/faith in the regular old JSON.encode and JSON.decode methods. Reading between the lines that is what I believe the author is frustrated with. It sounds like they want the equivalent to a Python’s pickle or PHP’s serialize. But of course you will quickly run into limitations there, too. The lesson being: we will all need to augment vanilla serialization tools with little tweaks and enhancements to fit our specific use cases. Again, blaming the tool is not productive. JSON is not the problem. JSON can do all of the things the author wants it to do. The issue is that JSON.stringify cannot.
- Twisol 6y ago
  > You can use field names or you can encode the type alongside. Ex: > { “birthdate”: [“date”, “01-02-1991”] } > And then in your codebase you know that all the values in your data are actually a list of (type, value). In this specific case I’d decide a “date” as “mm-dd-yyyy” This addresses neither of the problems I referenced from the article. The problem you've chosen to demonstrate is, indeed, trivial. > It sounds like they want the equivalent to a Python’s pickle or PHP’s serialize. They very much want the opposite. I wonder how you can read the article and conclude that they want an all-in-one serialization facility. The image at the end of the post well summarizes their position [0]. [0] https://einarwh.files.wordpress.com/2020/05/json-serialization-decoupled-by-hand-1.png https://einarwh.files.wordpress.com/2020/05/json-serializati...
  - whalesalad 6y ago
    This is how everyone is already using JSON. This post is complaining about a problem that doesn’t exist.
    - Twisol 6y ago
      Sadly, I can personally attest that the problem does exist, and that not everybody is performing serialization in such a reasonable, decoupled way. It sounds like you agree with the article, but find its existence unnecessary. That's fine.
    - smaudet 6y ago
      I have to agree with the other bloke, this is very much a problem in certain 'magical' JSON 'serializers' in certain very popular server languages, e.g. Java, Spring, FasterXML. I've seen JSON ser done right where you do just ser/deser direct from/to a JSON object, then perform a manual mapping to domaim or business objects, I've seen it done the wrong way where the domain and data objects are highly coupled, by default. I think the OP is bemoaning that being a common practice.
dreamcompiler 6y ago
It can even happen without cycles. If object A contains two references to object B and you want to read A back into memory the same way (rather than have it contain two different but identical objects B) you have to deal with the issue of referential integrity. No cycles needed. If your domain model is strictly that of rows in an RDBMS, there's no problem. Otherwise you're not storing objects but graphs of objects and overlooking that simple fact is the source of many serialization problems.
stickfigure 6y ago
> safe refactoring ends up breaking format compatibility I have a moderately strict rule for my teams that API objects must exist separately from normal domain objects (ie, persistence). And any APIs that have separate lifecycles (say, private webclient API vs official published API) get separate DTOs (which is what they really are). This works fine? It's not much work, not even in Java (thanks to lombok). There's clear migration strategies for these api objects and you can refactor your domain objects without risk of breaking something. I guess this is the "by hand" mapping that the article concludes with, but honestly it seems like a lot of words just to say "keep your API objects separate from your domain objects". > JSON cannot directly represent cycles It's incredibly easy to tweak JSON to allow it, and you don't even need a special parser. I wrote this five years ago: https://github.com/jsog/jsog https://github.com/jsog/jsog
- Twisol 6y ago
  > I guess this is the "by hand" mapping that the article concludes with Not "the", "a". One of the comments under the OP itself links to another blog post [0] describing an alternative to explicit DTOs, which I personally prefer. The point isn't just to pick some other solution and be done with it; the point is to understand the problem in the first place. [0] http://www.ballofcode.com/ddd/boundaries/serialization/2016/02/20/avoid-serialization-objects.html http://www.ballofcode.com/ddd/boundaries/serialization/2016/... > It's incredibly easy to tweak JSON to allow it Sure, but that was just an example of a general class of coupled-representation problems. As discussed elsewhere in this thread, the choice of JSON isn't even essential to the problem being discussed. Also, even though (or precisely because) JSOG is 100% JSON, the fact remains that after your JSON parser finishes reading your JSOG document, you still have to hook up all the cycles. Either you do this by hand (per the article), or you wrap it up into parser-level knowledge (which breaks somewhat from the "100% JSON" intent).
  - stickfigure 6y ago
    static JObject CreateMessageRepresentation(Customer customer) { return new JObject( new JProperty("customer", new JObject( new JProperty("name", customer.Name), new JProperty("address", new JObject( new JProperty("street", customer.Address.Street), new JProperty("zipCode", customer.Address.Zip), new JProperty("town", customer.Address.City) ) ) ) ) ) } Yuck. Just make CustomerData and AddressData classes, even if you only use them for that one API response. And even if you have ten other versions of CustomerData and AddressData for ten other methods. You get type safety and your tests refactor nicely. @Value static class AddressData { String street; String zipCode; String town; } @Value static class CustomerData { String name; AddressData address; } @Value static class Message { CustomerData customer; } public Message createMessage(Customer customer) { final Address addy = customer.getAddress(); return new Message(new CustomerData(customer.getName(), new AddressData(addy.getStreet(), addy.getZip(), addy.getCity()))); } You could format this nicer, and adding some constructors would help, but at least the typechecker is doing work for you.

by Twisol 6y ago

whalesalad 6y ago

You can use field names or you can encode the type alongside. Ex: { “birthdate”: [“date”, “01-02-1991”] } And then in your codebase you know that all the values in your data are actually a list of (type, value). In this specific case I’d decide a “date” as “mm-dd-yyyy” These are trivial problems. We have all solved them. The root of the problem is wanting to put all your trust/hope/faith in the regular old JSON.encode and JSON.decode methods. Reading between the lines that is what I believe the author is frustrated with. It sounds like they want the equivalent to a Python’s pickle or PHP’s serialize. But of course you will quickly run into limitations there, too. The lesson being: we will all need to augment vanilla serialization tools with little tweaks and enhancements to fit our specific use cases. Again, blaming the tool is not productive. JSON is not the problem. JSON can do all of the things the author wants it to do. The issue is that JSON.stringify cannot.

Twisol 6y ago
> You can use field names or you can encode the type alongside. Ex: > { “birthdate”: [“date”, “01-02-1991”] } > And then in your codebase you know that all the values in your data are actually a list of (type, value). In this specific case I’d decide a “date” as “mm-dd-yyyy” This addresses neither of the problems I referenced from the article. The problem you've chosen to demonstrate is, indeed, trivial. > It sounds like they want the equivalent to a Python’s pickle or PHP’s serialize. They very much want the opposite. I wonder how you can read the article and conclude that they want an all-in-one serialization facility. The image at the end of the post well summarizes their position [0]. [0] https://einarwh.files.wordpress.com/2020/05/json-serialization-decoupled-by-hand-1.png https://einarwh.files.wordpress.com/2020/05/json-serializati...
- whalesalad 6y ago
  This is how everyone is already using JSON. This post is complaining about a problem that doesn’t exist.
  - Twisol 6y ago
    Sadly, I can personally attest that the problem does exist, and that not everybody is performing serialization in such a reasonable, decoupled way. It sounds like you agree with the article, but find its existence unnecessary. That's fine.
  - smaudet 6y ago
    I have to agree with the other bloke, this is very much a problem in certain 'magical' JSON 'serializers' in certain very popular server languages, e.g. Java, Spring, FasterXML. I've seen JSON ser done right where you do just ser/deser direct from/to a JSON object, then perform a manual mapping to domaim or business objects, I've seen it done the wrong way where the domain and data objects are highly coupled, by default. I think the OP is bemoaning that being a common practice.

dreamcompiler 6y ago

It can even happen without cycles. If object A contains two references to object B and you want to read A back into memory the same way (rather than have it contain two different but identical objects B) you have to deal with the issue of referential integrity. No cycles needed. If your domain model is strictly that of rows in an RDBMS, there's no problem. Otherwise you're not storing objects but graphs of objects and overlooking that simple fact is the source of many serialization problems.

stickfigure 6y ago

> safe refactoring ends up breaking format compatibility I have a moderately strict rule for my teams that API objects must exist separately from normal domain objects (ie, persistence). And any APIs that have separate lifecycles (say, private webclient API vs official published API) get separate DTOs (which is what they really are). This works fine? It's not much work, not even in Java (thanks to lombok). There's clear migration strategies for these api objects and you can refactor your domain objects without risk of breaking something. I guess this is the "by hand" mapping that the article concludes with, but honestly it seems like a lot of words just to say "keep your API objects separate from your domain objects". > JSON cannot directly represent cycles It's incredibly easy to tweak JSON to allow it, and you don't even need a special parser. I wrote this five years ago: https://github.com/jsog/jsog https://github.com/jsog/jsog

Twisol 6y ago
> I guess this is the "by hand" mapping that the article concludes with Not "the", "a". One of the comments under the OP itself links to another blog post [0] describing an alternative to explicit DTOs, which I personally prefer. The point isn't just to pick some other solution and be done with it; the point is to understand the problem in the first place. [0] http://www.ballofcode.com/ddd/boundaries/serialization/2016/02/20/avoid-serialization-objects.html http://www.ballofcode.com/ddd/boundaries/serialization/2016/... > It's incredibly easy to tweak JSON to allow it Sure, but that was just an example of a general class of coupled-representation problems. As discussed elsewhere in this thread, the choice of JSON isn't even essential to the problem being discussed. Also, even though (or precisely because) JSOG is 100% JSON, the fact remains that after your JSON parser finishes reading your JSOG document, you still have to hook up all the cycles. Either you do this by hand (per the article), or you wrap it up into parser-level knowledge (which breaks somewhat from the "100% JSON" intent).
- stickfigure 6y ago
  static JObject CreateMessageRepresentation(Customer customer) { return new JObject( new JProperty("customer", new JObject( new JProperty("name", customer.Name), new JProperty("address", new JObject( new JProperty("street", customer.Address.Street), new JProperty("zipCode", customer.Address.Zip), new JProperty("town", customer.Address.City) ) ) ) ) ) } Yuck. Just make CustomerData and AddressData classes, even if you only use them for that one API response. And even if you have ten other versions of CustomerData and AddressData for ten other methods. You get type safety and your tests refactor nicely. @Value static class AddressData { String street; String zipCode; String town; } @Value static class CustomerData { String name; AddressData address; } @Value static class Message { CustomerData customer; } public Message createMessage(Customer customer) { final Address addy = customer.getAddress(); return new Message(new CustomerData(customer.getName(), new AddressData(addy.getStreet(), addy.getZip(), addy.getCity()))); } You could format this nicer, and adding some constructors would help, but at least the typechecker is doing work for you.