4 ms·

JEP 540: Simple JSON API (Now in Incubator)

whartung 3mo ago
Well, this will be fun. I already have one of these (I'm sure I'm not alone). It's about 500 lines. I found I'm not a huge fan of the JAX-B-ish style serialization of Java objects for JSON. I don't want to really downplay them, they certainly have their uses, they're very popular, I just don't like fighting them. Hand writing JSON marshaling code has not been arduous for me (notably with my utility layer). (I also, philosophically, strive to avoid "magic" in my code as much as practical.) Of course, I still need a parser, I'm using GSONs parser, which means I'm still dragging in the whole bean level serialization infrastructure. I just don't use it. And while I've done JSON parsers before, I felt it was something better to import than maintain myself. So, in that sense, it's a mixed bag. But, I do enjoy using it. This will be a worthwhile JDK capability. Ideally it can replace mine.
gavinray 3mo ago
It used to be the case that if you wanted to build a web service on the JVM, you wanted 2 things: 1. An HTTP server library/framework 2. A JSON library We got a decently-performing and unopionated HTTP server in JDK 18 with "HttpHandlers" and "SimpleFileServer" plus "jwebserver" CLI It later received Virtual Thread support, which made performance + scalability very competitive. With a JSON module, you finally won't NEED to rely on external deps to build a basic JVM web service without pain. Now, we just need a proper CLI framework like picocli, or at least "argparse" from Python stdlib...
- drdexebtjl 3mo ago
  My understanding from reading this is the complete opposite. This library is explicitly not supporting the features that web servers need to be performant and handle production traffic, like streaming. A web server using this could only start parsing when it receives the last byte, and could only start responding when it’s done serializing, all while holding non-lazy trees of JsonValue objects in memory.
  - wewtyflakes 3mo ago
    I suspect the vast majority of services are not dealing with such massive JSON documents that doing serde on them becomes a material part of their latency breakdown.
  - gavinray 3mo ago
    I've never built or worked on a service where the JSON payloads were so large that (de)serialization accounted for a significant portion of the timing profile I'm sure lots of them exist, but for your typical CRUD API, this has not been a phenomena I've run into.
    - drdexebtjl 3mo ago
      I don’t have any benchmarks, but it’s surprisingly relevant. Not necessarily because parsing JSON is slow, but because managing memory is slow, and you’re dealing with potentially malicious input. A non-streaming implementation needs to copy the request from the network stack into a contiguous GC-managed char array, possibly resizing it a few times as the data is received. Then when it’s time to parse, it goes through this array and allocates an unbounded number of JsonValue nodes. For JsonString and JsonNumber, it probably needs to create defensive copies of the data instead of spans of the input array, otherwise changing the input array corrupts the tree. That’s kinda bad under memory pressure even for benign inputs. But consider malicious inputs, such as {"x":{"x":{"x":{"x":{"x":{…}}}}}. It would make this non-streaming implementation allocate a lot of String and JsonObject instances. The allocations would total multiple times the size of the input, and would be extremely fragmented. On the other hand, a library that does streaming and that binds to objects could parse straight from the buffers in the network stack, and could avoid allocating objects for anything that it will not need to bind.
IanGabes 3mo ago
Including this in the standard library speaks to how much a citizen JSON has become, where a language simply can't afford to not consider it. I find it interesting to note that nowhere in this JEP is the word "serialization", which is what most people might associate with JSON libs. Or rather, they are studiously ignoring that feature and just improving the ergonomics of interacting with JSON.
- ameliaquining 3mo ago
  If I'm guessing correctly what you mean by "serialization", the JEP refers to it as "data binding" and includes a section on why it's not going to be part of this library.
- mcfedr 3mo ago
  its crazy its taken Java so long to realise this
MeteorMarc 3mo ago
Many languages have json marshalling and unmarshalling in their standard libs, e.g. C#, golang, python.
- deepsun 3mo ago
  You forgot Javascript :)
- maleldil 3mo ago
  Even a relatively barebones systems language like Zig has std.json. Odin has it too.
whaley 3mo ago
Meanwhile, Jackson has been going strong for almost 20 years now https://github.com/FasterXML/jackson https://github.com/FasterXML/jackson. One of my favorite things about Jackson was being able to arbitrarily navigate through the document with a rich and fluent API (JsonNode) in jackson.databind, which this JEP at least conceptually borrows from with the JsonValue abstraction. Both of these are better than how some of the other implementations do it, where you are effectively working with glorified Map<String,Object>
- rf15 3mo ago
  But isn't JSON de facto conceptually a glorified Map<String,Object> ?
  - wewtyflakes 3mo ago
    The root of a JSON document can also be an array.
    - svieira 3mo ago
      Or the boolean `true`, or the empty string `""`, or `null`. JSON allows bare "value"s as the root element. https://www.json.org/json-en.html https://www.json.org/json-en.html
exabrial 3mo ago
Is this just including JSR-367 in the sdk? I've used that for years along with its many implementations. Great stuff!
q3k 3mo ago
Happy to see that numbers are arbitrary width/precision until explicitly cast by the user. This goes against so many other JSON libraries that will always cast all numbers to double (or even float) thereby silently corrupting JSON numbers representing large values (eg. memory addresses). Does anyone know how this behaves when encountering a repeated key in an object? (RFC8259 states that keys SHOULD be unique, which makes that generally allowed and implementations all behave slightly differently).
- lmz 3mo ago
  Not sure why they didn't include the BigDecimal conversion directly on the JsonNumber object instead of going through the String representation. Duplicate keys are a parse exception. > Additionally, documents must not have objects with duplicate member names.
  - q3k 3mo ago
    > Duplicate keys are a parse exception. Good, that's the least bad behavior. (Postel's law be damned)
Sankozi 3mo ago
Using JSON as a configuration format is a big mistake. JEP authors could learn a bit from package.json problems. Hope JSON will not be used in anything significant for JDK configuration.
- Groxx 3mo ago
  Yeah - json is fine as a readable mechanical exchange format, but without comments it's essentially unusable for anything humans need to touch :/ I'm somewhat boggled that json5 hasn't grown to be more of a thing.
dfabulich 3mo ago
A stated goal of the API is to have "low ceremony"; this seems like a lot of ceremony. IO.println(JsonObject.of(Map.of("providers", JsonArray.of(List.of(JsonString.of("SUN"), JsonString.of("SunRsaSign"), JsonString.of("SunEC")))))); There's gotta be a better way! Surely there could be some way of creating a JsonArray of native Java Strings, Booleans, Doubles, and Integers without requiring clients to explicitly convert each value into a JsonValue. And why am I forced to convert a native List into a JsonArray just so I can make it the value of a JsonObject? Why can't I write this? JsonObject.of(Map.of("providers", List.of("SUN", "SunRsaSign", SunEC"))));
- nlitened 3mo ago
  There's a nice Java library called Clojure with a lightweight syntax if you need to work with data structures and concurrency in Java a lot: (println (json/generate-string {:providers ["SUN" "SunRsaSign" "SunEC"]}))
  - packetlost 3mo ago
    You really can't appreciate how awesome Clojure is until you're coming from Java, can you...
    - pron 3mo ago
      And vice versa. As someone who likes both languages, I appreciate how they both have their pros and cons. I was a Schemer before I learnt Java (when Java barely existed) and I adore Clojure, but if I were to write 10 MLOC mobile network routing and billing system, an air-traffic control system, or a credit-card transaction processing system etc. etc. that needs to evolve by a large team for 20 years, I would choose Java over Clojure any day. So I came to Java from C, C++, Scheme, and ML, and it's completely unsurprising to me why there has never existed a language that better supports large, long-lasting, important software (I'm not saying such a language couldn't exist, but it currently doesn't). And it's not just about the features Java has and, just as importantly, doesn't have (and if you think keeping features out of a programming language is easy, think again), but also how its evolution is handled. In this particular case, of course it is more straightforward to embed what is effectively an untyped language in another untyped language than in a typed language, at least while keeping the language simple and without adding features we don't wish to add (even in TypeScript, working with JSON directly requires bypassing type checks). Still, as the JEP says, there are good, popular Java libraries that offer more convenient and powerful ways of working with JSON, but that is not the purpose of this particular package.
delusional 3mo ago
They're doing a real API instead of the magic annotation hell that JavaEE fans love. Thank god. I welcome this, because Java has a lot of need for a good, common, performant, and well maintained JSON library that doesn't come along with the complexity of being JSON-B.
nikeee 3mo ago
I don't like this: String body = ... REST response body, which is a JSON document ... ; JsonValue json = Json.parse(body); json.get("properties").get("periods").asList().stream() .mapToInt(j -> j.get("temperature").asInt()) .average() .ifPresent(IO::println); Why am I able to call `.get(string)` or `.get(int)` on a JsonValue? Shouldn't these be on the JsonObject and JsonArray instead? > If the JsonValue instance is of the wrong type, or if the requested member or element does not exist, the access methods throw a JsonValueException. So if I get an exception, I can't tell whether the value was of the wrong type or the object didn't have the requested key? This looks like a footgun to me. They just brought pattern matching to Java. Why not move those .get to JsonArray and JsonObject? That would solve this confusions. So we could just use something like `if (json instanceof JsonObject o) o.get("properties")`
- zbentley 3mo ago
  That is indeed baffling and regrettable. Perhaps they believe that users will just hard-cast what .parse() returns to JsonObject/JsonArray/whatever, and that the resulting ClassCastException will be uglier and harder to debug than whatever errors are currently produced by calling .get() on something other than JsonObject?
- jaen 3mo ago
  The alternative has bad ergonomics, chained `.get`s which are the most common operation become: json.asObject().get("prop1").asObject().get("prop2")...
esprehn 3mo ago
Not supporting comments will be a mistake that haunts this API. They give an example of replacing properties files but those do have standardized comments! The proposed pre-processing step means all the comments are lost during round tripping, and the single line comments they suggest are not enough to even match JSONC. By the time those issues have userland workarounds you might as well use another library instead of the built-in one. This seems to repeat the same mistakes of Go's built in JSON library where the ecosystem is full of workarounds and other libraries that are faster or have better features.
- zbentley 3mo ago
  Eh, I think it's a good tradeoff. If it losslessly supported deserializing comments/JSONC, or trailing commas, or JSON-lines, or whatever, the serialization APIs would get more complicated. Every time you serialize you'd have to decide which of several formats you were producing. Automatic round-trippability would still be impossible in that world, since e.g. "deserialize JSON-ish, set one key=value, reserialize" would then risk producing a not-strictly-JSON object that broke whatever it was sent to, so then you'd need a whole bunch of different serialization configs/settings, which would confuse newbies (either they produce something that's subtly different from what they need, or they accidentally strip out information). As similar as all the almost-JSON formats are, I still think it's best to keep APIs single-purpose: one for JSON, one for JSON-lines, one for JSONC, and so on. It's a larger code surface, but a less potentially surprising one.
jonenst 3mo ago
I find it very surprising that they went for unchecked exceptions. For JsonValueException the following rationale is given "This exception is unchecked, so that scripts and small programs are easier to read and write." But for JsonParseException there is no rationale. This is surprising especially given the pushback from openjdk members against jackson3 moving to unchecked exception.
- crewindream 3mo ago
  I’m very happy that these exes are unchecked (incl jackson3). If parsing fails - how often can you do anything else than just abort and log/show an error.. Checked exceptions are nice in theory, but it is very context dependent if checked’ness is useful, I would prefer that libraries do not expose them.

Well, this will be fun. I already have one of these (I'm sure I'm not alone). It's about 500 lines. I found I'm not a huge fan of the JAX-B-ish style serialization of Java objects for JSON. I don't want to really downplay them, they certainly have their uses, they're very popular, I just don't like fighting them. Hand writing JSON marshaling code has not been arduous for me (notably with my utility layer). (I also, philosophically, strive to avoid "magic" in my code as much as practical.) Of course, I still need a parser, I'm using GSONs parser, which means I'm still dragging in the whole bean level serialization infrastructure. I just don't use it. And while I've done JSON parsers before, I felt it was something better to import than maintain myself. So, in that sense, it's a mixed bag. But, I do enjoy using it. This will be a worthwhile JDK capability. Ideally it can replace mine.

gavinray 3mo ago

It used to be the case that if you wanted to build a web service on the JVM, you wanted 2 things: 1. An HTTP server library/framework 2. A JSON library We got a decently-performing and unopionated HTTP server in JDK 18 with "HttpHandlers" and "SimpleFileServer" plus "jwebserver" CLI It later received Virtual Thread support, which made performance + scalability very competitive. With a JSON module, you finally won't NEED to rely on external deps to build a basic JVM web service without pain. Now, we just need a proper CLI framework like picocli, or at least "argparse" from Python stdlib...

drdexebtjl 3mo ago
My understanding from reading this is the complete opposite. This library is explicitly not supporting the features that web servers need to be performant and handle production traffic, like streaming. A web server using this could only start parsing when it receives the last byte, and could only start responding when it’s done serializing, all while holding non-lazy trees of JsonValue objects in memory.
- wewtyflakes 3mo ago
  I suspect the vast majority of services are not dealing with such massive JSON documents that doing serde on them becomes a material part of their latency breakdown.
- gavinray 3mo ago
  I've never built or worked on a service where the JSON payloads were so large that (de)serialization accounted for a significant portion of the timing profile I'm sure lots of them exist, but for your typical CRUD API, this has not been a phenomena I've run into.
  - drdexebtjl 3mo ago
    I don’t have any benchmarks, but it’s surprisingly relevant. Not necessarily because parsing JSON is slow, but because managing memory is slow, and you’re dealing with potentially malicious input. A non-streaming implementation needs to copy the request from the network stack into a contiguous GC-managed char array, possibly resizing it a few times as the data is received. Then when it’s time to parse, it goes through this array and allocates an unbounded number of JsonValue nodes. For JsonString and JsonNumber, it probably needs to create defensive copies of the data instead of spans of the input array, otherwise changing the input array corrupts the tree. That’s kinda bad under memory pressure even for benign inputs. But consider malicious inputs, such as {"x":{"x":{"x":{"x":{"x":{…}}}}}. It would make this non-streaming implementation allocate a lot of String and JsonObject instances. The allocations would total multiple times the size of the input, and would be extremely fragmented. On the other hand, a library that does streaming and that binds to objects could parse straight from the buffers in the network stack, and could avoid allocating objects for anything that it will not need to bind.

IanGabes 3mo ago

Including this in the standard library speaks to how much a citizen JSON has become, where a language simply can't afford to not consider it. I find it interesting to note that nowhere in this JEP is the word "serialization", which is what most people might associate with JSON libs. Or rather, they are studiously ignoring that feature and just improving the ergonomics of interacting with JSON.

ameliaquining 3mo ago
If I'm guessing correctly what you mean by "serialization", the JEP refers to it as "data binding" and includes a section on why it's not going to be part of this library.
mcfedr 3mo ago
its crazy its taken Java so long to realise this

MeteorMarc 3mo ago

Many languages have json marshalling and unmarshalling in their standard libs, e.g. C#, golang, python.

deepsun 3mo ago
You forgot Javascript :)
maleldil 3mo ago
Even a relatively barebones systems language like Zig has std.json. Odin has it too.

whaley 3mo ago

Meanwhile, Jackson has been going strong for almost 20 years now https://github.com/FasterXML/jackson https://github.com/FasterXML/jackson. One of my favorite things about Jackson was being able to arbitrarily navigate through the document with a rich and fluent API (JsonNode) in jackson.databind, which this JEP at least conceptually borrows from with the JsonValue abstraction. Both of these are better than how some of the other implementations do it, where you are effectively working with glorified Map<String,Object>

rf15 3mo ago
But isn't JSON de facto conceptually a glorified Map<String,Object> ?
- wewtyflakes 3mo ago
  The root of a JSON document can also be an array.
  - svieira 3mo ago
    Or the boolean `true`, or the empty string `""`, or `null`. JSON allows bare "value"s as the root element. https://www.json.org/json-en.html https://www.json.org/json-en.html

exabrial 3mo ago

Is this just including JSR-367 in the sdk? I've used that for years along with its many implementations. Great stuff!

q3k 3mo ago

Happy to see that numbers are arbitrary width/precision until explicitly cast by the user. This goes against so many other JSON libraries that will always cast all numbers to double (or even float) thereby silently corrupting JSON numbers representing large values (eg. memory addresses). Does anyone know how this behaves when encountering a repeated key in an object? (RFC8259 states that keys SHOULD be unique, which makes that generally allowed and implementations all behave slightly differently).

lmz 3mo ago
Not sure why they didn't include the BigDecimal conversion directly on the JsonNumber object instead of going through the String representation. Duplicate keys are a parse exception. > Additionally, documents must not have objects with duplicate member names.
- q3k 3mo ago
  > Duplicate keys are a parse exception. Good, that's the least bad behavior. (Postel's law be damned)

Sankozi 3mo ago

Using JSON as a configuration format is a big mistake. JEP authors could learn a bit from package.json problems. Hope JSON will not be used in anything significant for JDK configuration.

Groxx 3mo ago
Yeah - json is fine as a readable mechanical exchange format, but without comments it's essentially unusable for anything humans need to touch :/ I'm somewhat boggled that json5 hasn't grown to be more of a thing.

dfabulich 3mo ago

A stated goal of the API is to have "low ceremony"; this seems like a lot of ceremony. IO.println(JsonObject.of(Map.of("providers", JsonArray.of(List.of(JsonString.of("SUN"), JsonString.of("SunRsaSign"), JsonString.of("SunEC")))))); There's gotta be a better way! Surely there could be some way of creating a JsonArray of native Java Strings, Booleans, Doubles, and Integers without requiring clients to explicitly convert each value into a JsonValue. And why am I forced to convert a native List into a JsonArray just so I can make it the value of a JsonObject? Why can't I write this? JsonObject.of(Map.of("providers", List.of("SUN", "SunRsaSign", SunEC"))));

nlitened 3mo ago
There's a nice Java library called Clojure with a lightweight syntax if you need to work with data structures and concurrency in Java a lot: (println (json/generate-string {:providers ["SUN" "SunRsaSign" "SunEC"]}))
- packetlost 3mo ago
  You really can't appreciate how awesome Clojure is until you're coming from Java, can you...
  - pron 3mo ago
    And vice versa. As someone who likes both languages, I appreciate how they both have their pros and cons. I was a Schemer before I learnt Java (when Java barely existed) and I adore Clojure, but if I were to write 10 MLOC mobile network routing and billing system, an air-traffic control system, or a credit-card transaction processing system etc. etc. that needs to evolve by a large team for 20 years, I would choose Java over Clojure any day. So I came to Java from C, C++, Scheme, and ML, and it's completely unsurprising to me why there has never existed a language that better supports large, long-lasting, important software (I'm not saying such a language couldn't exist, but it currently doesn't). And it's not just about the features Java has and, just as importantly, doesn't have (and if you think keeping features out of a programming language is easy, think again), but also how its evolution is handled. In this particular case, of course it is more straightforward to embed what is effectively an untyped language in another untyped language than in a typed language, at least while keeping the language simple and without adding features we don't wish to add (even in TypeScript, working with JSON directly requires bypassing type checks). Still, as the JEP says, there are good, popular Java libraries that offer more convenient and powerful ways of working with JSON, but that is not the purpose of this particular package.

delusional 3mo ago

They're doing a real API instead of the magic annotation hell that JavaEE fans love. Thank god. I welcome this, because Java has a lot of need for a good, common, performant, and well maintained JSON library that doesn't come along with the complexity of being JSON-B.

nikeee 3mo ago

I don't like this: String body = ... REST response body, which is a JSON document ... ; JsonValue json = Json.parse(body); json.get("properties").get("periods").asList().stream() .mapToInt(j -> j.get("temperature").asInt()) .average() .ifPresent(IO::println); Why am I able to call `.get(string)` or `.get(int)` on a JsonValue? Shouldn't these be on the JsonObject and JsonArray instead? > If the JsonValue instance is of the wrong type, or if the requested member or element does not exist, the access methods throw a JsonValueException. So if I get an exception, I can't tell whether the value was of the wrong type or the object didn't have the requested key? This looks like a footgun to me. They just brought pattern matching to Java. Why not move those .get to JsonArray and JsonObject? That would solve this confusions. So we could just use something like `if (json instanceof JsonObject o) o.get("properties")`

zbentley 3mo ago
That is indeed baffling and regrettable. Perhaps they believe that users will just hard-cast what .parse() returns to JsonObject/JsonArray/whatever, and that the resulting ClassCastException will be uglier and harder to debug than whatever errors are currently produced by calling .get() on something other than JsonObject?
jaen 3mo ago
The alternative has bad ergonomics, chained `.get`s which are the most common operation become: json.asObject().get("prop1").asObject().get("prop2")...

esprehn 3mo ago

Not supporting comments will be a mistake that haunts this API. They give an example of replacing properties files but those do have standardized comments! The proposed pre-processing step means all the comments are lost during round tripping, and the single line comments they suggest are not enough to even match JSONC. By the time those issues have userland workarounds you might as well use another library instead of the built-in one. This seems to repeat the same mistakes of Go's built in JSON library where the ecosystem is full of workarounds and other libraries that are faster or have better features.

zbentley 3mo ago
Eh, I think it's a good tradeoff. If it losslessly supported deserializing comments/JSONC, or trailing commas, or JSON-lines, or whatever, the serialization APIs would get more complicated. Every time you serialize you'd have to decide which of several formats you were producing. Automatic round-trippability would still be impossible in that world, since e.g. "deserialize JSON-ish, set one key=value, reserialize" would then risk producing a not-strictly-JSON object that broke whatever it was sent to, so then you'd need a whole bunch of different serialization configs/settings, which would confuse newbies (either they produce something that's subtly different from what they need, or they accidentally strip out information). As similar as all the almost-JSON formats are, I still think it's best to keep APIs single-purpose: one for JSON, one for JSON-lines, one for JSONC, and so on. It's a larger code surface, but a less potentially surprising one.

jonenst 3mo ago

I find it very surprising that they went for unchecked exceptions. For JsonValueException the following rationale is given "This exception is unchecked, so that scripts and small programs are easier to read and write." But for JsonParseException there is no rationale. This is surprising especially given the pushback from openjdk members against jackson3 moving to unchecked exception.

crewindream 3mo ago
I’m very happy that these exes are unchecked (incl jackson3). If parsing fails - how often can you do anything else than just abort and log/show an error.. Checked exceptions are nice in theory, but it is very context dependent if checked’ness is useful, I would prefer that libraries do not expose them.