4 ms·

This really has nothing at all to do with JSON. The author is really complaining about encoding and decoding domain objects (happening to use JSON as a represen

• Scarbutt 6y ago
  It's not like everyone has the choice to not use JSON.
  • whalesalad 6y ago
    ??? There is absolutely nothing inherent to JSON that would cause the woes the author is experiencing. Any data can be serialized with JSON.
    • Scarbutt 6y ago
      Any data can be serialized with JSON. Yep, by writing ad-hoc and context-dependent formats on top of JSON for every language on your system. JSON set of types are just too limited.
    • blandflakes 6y ago
      Can't any data also be serialized as a string? Does that mean "string payload" is good enough for exchanging data between systems?
      • whalesalad 6y ago
        Bingo.
        • giantDinosaur 6y ago
          Everything is encoded in bits. What's like, the difference between anything?
        • blandflakes 6y ago
          I guess my point was that I'm rather on the side of "Ok then maybe it is still worth finding a serialization format that maps better to my domain, like perhaps that can serialize numbers with the same precision without having to route them through a string and back." This is like claiming that it's not worth comparing programming languages because machine code is Turing complete.
• cratermoon 6y ago
  Your response is technically correct but completely useless. Of course if we all just programmed in the JSON data model everything would be simple. The author correctly points out the uselessness of that. He correctly identifies that the complexity lies in providing a useful JSON serialization library that is both compliant in input/output yet flexible enough to be configured to go from JSON object model to business model. It's actually a complex problem, going from one type system to another. Want to know how complex? Just look at the compilers and interpreters that transform instructions for complex data types down to x86 (or M1, or RISC, or whatever) opcodes.
  • whalesalad 6y ago
    You can’t model the entire world. Well, you can, but you can’t. Take a look at schema.org to see where that rabbit hole will take you. So instead we use more primitive tools that allow us to model our domain without needing a tool that magically knows how to do it for us. My domain has unique nouns that differ from someone else’s. JSON offers building blocks of primitive types like integers and strings and can be used to build more sophisticated types like “timezone aware datetime”. We use ISO8601 for this. We can use that inside JSON. Now you and I can agree on what a date is. We could build this into the language itself - but many would argue the beauty of JSON is in the fact that it is very naive and can be used quite literally to model any scenario.
    • cratermoon 6y ago
      > We use ISO8601 Ah yes, the old "stringly typed" data model. https://gist.github.com/timvisee/fcda9bbdff88d45cc9061606b4b923ca https://gist.github.com/timvisee/fcda9bbdff88d45cc9061606b4b... Application developers aren't trying to model the entire world, they are trying to model their domain, and I'm sorry, but primitive strings, numbers, lists, and maps are the building blocks of a domain model, but they are not sufficient. See for example https://fsharpforfunandprofit.com/ddd/ https://fsharpforfunandprofit.com/ddd/
      • earthboundkid 6y ago
• Twisol 6y ago
  I didn't really read JSON as the core issue to begin with. In practice, JSON serialization libraries are used to map between domain objects and a JSON representation. The author notes that this really breaks into two problems, one of which is specific to JSON and the other of which is truly dependent on what exactly your domain model looks like. The whole point is that no single library can solve a problem of unbounded (domain-dependent) complexity, and yet we rely on libraries like GSON to do that for us anyway. This leads to some problems, which the author calls out. > This is actually a very trivial problem indeed! I'm not sure I agree. The author specifically mentions the problem of coupling field names to names in the representation, such that an otherwise safe refactoring ends up breaking format compatibility. The more general problem is a strict coupling between the structure of your domain objects and the structure of your representation. For instance, if your domain models cyclically-related objects, you have to decouple these structures somehow -- JSON cannot directly represent cycles. Sometimes you can tell whatever domain mapping library you're using to cut cycles in certain ways, but the author rightly calls this a Faustian bargain. Retaining full control over these representational issues obviates a lot of the problems you'd otherwise face when adapting a library to your domain.
  • whalesalad 6y ago
    You can use field names or you can encode the type alongside. Ex: { “birthdate”: [“date”, “01-02-1991”] } And then in your codebase you know that all the values in your data are actually a list of (type, value). In this specific case I’d decide a “date” as “mm-dd-yyyy” These are trivial problems. We have all solved them. The root of the problem is wanting to put all your trust/hope/faith in the regular old JSON.encode and JSON.decode methods. Reading between the lines that is what I believe the author is frustrated with. It sounds like they want the equivalent to a Python’s pickle or PHP’s serialize. But of course you will quickly run into limitations there, too. The lesson being: we will all need to augment vanilla serialization tools with little tweaks and enhancements to fit our specific use cases. Again, blaming the tool is not productive. JSON is not the problem. JSON can do all of the things the author wants it to do. The issue is that JSON.stringify cannot.
    • 6y ago
• forrestthewoods 6y ago
  > You gotta remember that JSON doesn’t support all of the types that many languages do. So you need to engineer around that by annotating your attributes with a type or relying on a naming convention on keys. You can’t get mad at JSON for not being good at this. Oh I absolutely can get mad at it. JSON was not designed to be good at this. That’s the problem! Well, that and for various reasons (some good, some bad) programmers decided to use JSON for _everything_. JSON sucks. If it didn’t exist the world would probably invent and standardize on something better. But it does exist, and it sucks, and we’re stuck with it.
  • deleted 6y ago
    [deleted]
  • whalesalad 6y ago
    I guess you haven’t spent a lot of time time with XML? JSON is not perfect but I wouldn’t say it sucks.
    • forrestthewoods 6y ago
      XML is a dumpster fire. JSON merely sucks.
  • deleted 6y ago
    [deleted]
  • jitl 6y ago
    JSON replaced the previous iteration XML which also had a million ways to do things, plus rarely-used schemas, and miles of abstractions between your object and the text XML representation. The standard libraries for working with XML also had extremely insecure defaults around entities that were tantamount to remote code execution. Overall JSON simplified serialization tasks for the 80% case. The remaining 20% of special situations leads to the pain and suffering, especially in static typed languages.
• eyelidlessness 6y ago
  It actually is a JSON problem, specifically because ... > You gotta remember that JSON doesn’t support all of the types that many languages do. So you need to engineer around that And now you have N+1 problems. > You can’t get mad at JSON for not being good at this. It's not about being mad at JSON or even critical. It's about recognizing that the format's simplicity pushes some logic into either your language, your codec/serdes or your application itself. Unfortunately, what I've seen is that people opt for the "application itself" approach, do it completely ad hoc, and trust all kinds of godawful things that have no reason to be trusted. If you're using JSON and represent richer types (edit to add: and almost every JSON API does, there's no Date type so you're definitely making some assumptions about names or string [or bless your heart number] data), you should either: - Use a transport format that extends it in a consistent way like Transit. - Use the same codec logic on coupled client/server, and make your server's codec logic a first class part of your SDK which you should provide. There's good tools for sharing and translating type definitions. Use those too. There are better tools even than that (one of which I'm working on, and hope to have a `Show HN` soon).
  • Spivak 6y ago
    I think a shorter way of saying this is that everyone ad-hoc invents new formats that are subsets of JSON but don’t implement validators or parsers for them.
    • eyelidlessness 6y ago
      That’s true (except I assume you mean superset) but I think some of the shortening of it has a risk of masking just how awful that is. And in this case it’s not a JSON problem, it’s general data interchange design. I’ve worked on too many applications that have HTTP body, query, header, cookie logic deep in business logic. Or distributed systems just pulling rando values out of messages. Or database queries making assumptions about blob data. It’s pervasive. Engineers routinely design systems where they overload the types of their tools and just whistle while their world burns.
• kstenerud 6y ago
  Yes, JSON was not designed for this sort of thing; it was designed to follow the Javascript object model, which is VERY limited. It was a quick and clever hack to get serialization working between Javascript systems, so it did mostly solve the problem of rehydrating whole objects (in JS). The only tricky problem is if you change the data structure and then try to rehydrate from an old document (the same problem you encounter in databases when you change the schema). And most use cases have done alright despite that. But of course once you take JSON and try to make it work in other languages, you run into trouble with the lack of types. There have been many attempts at serialization formats that cover the 80% case, each with varying levels of success. The thing is, JSON and XML opened the door to a new requirement: human readability. Nowadays we want to be able to inspect and edit the data without needing some kind of specialized binary editor to do it. We want to be able to load it up into our text editor of choice, make changes, and expect it to work when fed into the machine. That's huge, and marked a paradigm shift in data communications and serialization (despite the terrible inefficiencies this introduces). But it's always been the lack of types that tripped up nontrivial use cases, requiring all sorts of unportable hacks and workarounds to shoehorn it into your particular use case. I've been developing Concise Encoding [1] over the past 3 years specifically to tackle this problem. For the 80% use case, the format must: - Be ad-hoc capable (most real-world use cases don't actually need a schema) - Be human readable/writable as text - Be efficient (don't waste energy constantly serializing/deserializing text) - Support all common types (bool, int, float, date, URL, UUIDs, list, map, array, etc) - Support recursive data - Support pseudo-objects like comments and metadata - Be precise (no lossy conversions from things such as floats) Also another pet peeve of mine is date formats. Pretty much every serialization format either gets it wrong, or doesn't support all of the cases it should (this includes ISO-8601). [1] https://concise-encoding.org https://concise-encoding.org