4 ms·

Given that almost all JSON applications are going to be IO bound (reading from disk, or more likely, network) then what’s the fascination in making super-fast J

Given that almost all JSON applications are going to be IO bound (reading from disk, or more likely, network) then what’s the fascination in making super-fast JSON decoders, beyond the engineering challenge? Sure, it’s an overhead to the processing you’re actually interested in, but I hardly imagine that it’s in “straw that breaks the camel’s back” territory.

jonstewart 6y ago
Because it’s not true that JSON parsing is I/O bound. It might be if you have an old 5400 RPM laptop hard drive, otherwise it’s CPU bound. There are many, many benchmarks which will indicate this to you, including ones in TFA.
- throwaway894345 6y ago
  Don't benchmarks like these usually pre-warm the file cache? If so, then these benchmarks wouldn't be evidence that parsing is I/O bound, since they're reading from kernel memory, right?
  - jonstewart 6y ago
    Correct, good parsing benchmarks will ensure the input is in memory or, at least, heavily cached with a linear access pattern. And, the point is, those benchmarks often show libraries parse JSON at a good deal less than 100 MB/s, or, for the fast ones, maybe 100-400 MB/s. Those parsing rates are not fast enough to claim that JSON parsing is I/O bound. Of course, the original comment is saying that most “apps” are I/O bound anyway (shall we assume web apps?). I think this is a lazy argument, or at least an ignorant/self-centered one — plenty of apps are not web apps running in an embarrassingly slow context like Django or Rails. For example, I work in digital forensics/cyber security, and we have to scan through TBs of logs (sometimes in JSON).
maccard 6y ago
Just because something isnt the slowest part of your pipeline doesn't mean it's not worth optimising. It might be the slowest part of the pipeline you control, or your network might be a local network where your throughput is 10G. Modern SSDs can do 3+GB/second. Even if you're aiming at 1GB/second, your json parser has to be able to do 0.1ms/MB to be able to keep with even a normal disk. If JSON parsing is fast enough, you can use it as an interchange format high performance tasks, and if it's your input and output format, you can use existing tools for non performance critical tasks that use json already!
jandrewrogers 6y ago
The assertion that JSON parsing is faster than I/O for most applications does not match my experience. First, achieving 10 GB/s consistent throughput for I/O on a server is pretty ordinary these days. If your systems are designed for performance and efficiency, parsing JSON with throughput that will saturate the I/O capacity is a challenge. Second, a lot of JSON that gets parsed in applications isn’t transiting I/O at all, it is in memory and only a subset of what is parsed may transit I/O. In these cases, there is an even higher bandwidth threshold for peak performance — memory bandwidth. I’ve profiled many applications that were completely and thoroughly bottlenecked on JSON parsing, even when care was taken to not be profligate with JSON parsing because everyone knows it is quite slow.
smarterclayton 6y ago
A concrete example here is that almost everything in the Kubernetes ecosystem (the control plane, operators, integrations, etc) is CPU bound by time spent Marshalling/Unmarshalling JSON or protobuf. So in some domains it’s all that moves the needle (in Kube I personally spent several years in protobuf, alternate JSON impls, fine tuning the hot path). Some problems are marshalling bound, although it’s not all of them.

jonstewart 6y ago

Because it’s not true that JSON parsing is I/O bound. It might be if you have an old 5400 RPM laptop hard drive, otherwise it’s CPU bound. There are many, many benchmarks which will indicate this to you, including ones in TFA.

throwaway894345 6y ago
Don't benchmarks like these usually pre-warm the file cache? If so, then these benchmarks wouldn't be evidence that parsing is I/O bound, since they're reading from kernel memory, right?
- jonstewart 6y ago
  Correct, good parsing benchmarks will ensure the input is in memory or, at least, heavily cached with a linear access pattern. And, the point is, those benchmarks often show libraries parse JSON at a good deal less than 100 MB/s, or, for the fast ones, maybe 100-400 MB/s. Those parsing rates are not fast enough to claim that JSON parsing is I/O bound. Of course, the original comment is saying that most “apps” are I/O bound anyway (shall we assume web apps?). I think this is a lazy argument, or at least an ignorant/self-centered one — plenty of apps are not web apps running in an embarrassingly slow context like Django or Rails. For example, I work in digital forensics/cyber security, and we have to scan through TBs of logs (sometimes in JSON).

maccard 6y ago

Just because something isnt the slowest part of your pipeline doesn't mean it's not worth optimising. It might be the slowest part of the pipeline you control, or your network might be a local network where your throughput is 10G. Modern SSDs can do 3+GB/second. Even if you're aiming at 1GB/second, your json parser has to be able to do 0.1ms/MB to be able to keep with even a normal disk. If JSON parsing is fast enough, you can use it as an interchange format high performance tasks, and if it's your input and output format, you can use existing tools for non performance critical tasks that use json already!

jandrewrogers 6y ago

The assertion that JSON parsing is faster than I/O for most applications does not match my experience. First, achieving 10 GB/s consistent throughput for I/O on a server is pretty ordinary these days. If your systems are designed for performance and efficiency, parsing JSON with throughput that will saturate the I/O capacity is a challenge. Second, a lot of JSON that gets parsed in applications isn’t transiting I/O at all, it is in memory and only a subset of what is parsed may transit I/O. In these cases, there is an even higher bandwidth threshold for peak performance — memory bandwidth. I’ve profiled many applications that were completely and thoroughly bottlenecked on JSON parsing, even when care was taken to not be profligate with JSON parsing because everyone knows it is quite slow.

smarterclayton 6y ago

A concrete example here is that almost everything in the Kubernetes ecosystem (the control plane, operators, integrations, etc) is CPU bound by time spent Marshalling/Unmarshalling JSON or protobuf. So in some domains it’s all that moves the needle (in Kube I personally spent several years in protobuf, alternate JSON impls, fine tuning the hot path). Some problems are marshalling bound, although it’s not all of them.