3 ms·

50MB/s was my guess for a json parser written in a direct way in a moderately fast language that doesn’t read into some optimised representation (eg I assume th

50MB/s was my guess for a json parser written in a direct way in a moderately fast language that doesn’t read into some optimised representation (eg I assume the python one constructs python data structures). The json file is 64k (which may have been what threw you off) so that comes to ~750 per second. But my guess of 50MB/a could be way off (might be imagining a slower implementation but then the computer for the tests is also a bit slow, I think)

tmtvl 3y ago
The test computer may be a little slow-ish but it doesn't seem "commodity laptop picked up from local supermarket" slow. It has an SSD, for example.
- dan-robertson 3y ago
  I think it’s ‘thermally limited and a few years old’ which makes it a bit slower than I expected. I’ll accept that ‘old’ was a bad characterisation.
ilyt 3y ago
I probably underestimated how much does it take to convert it from "fast JSON decoder written in C" (which many interpreted languages use in backend) to actual language implementation, but in perl it's pretty fast: -> ᛯ ls -s -h big.json 101M big.json [11:53:30] ^ [/tmp] -> ᛯ cat /tmp/1.pl #!/usr/bin/perl use v5.24; use JSON::XS; use File::Slurp; my $j = decode_json(read_file('/tmp/big.json')); say scalar @{$j->{'a'}} [11:53:39] ^ [/tmp] -> ᛯ time /tmp/1.pl 34952534 /tmp/1.pl 0,81s user 0,13s system 99% cpu 0,939 total so just around 100MB/s
Measter 3y ago
For comparison, using Rust's Serde-Json library on my desktop (Intel 11700k, and with computer doing other things), benchmarked with Criterion: Benchmarking json_read_struct: Collecting 100 samples in estimated 5.2012 s (81k iterations) json_read_struct time: [64.126 µs 64.376 µs 64.660 µs] change: [+0.2646% +0.7525% +1.2538%] (p = 0.00 < 0.05) Change within noise threshold. Found 7 outliers among 100 measurements (7.00%) 6 (6.00%) high mild 1 (1.00%) high severe My benchmark code[0] takes advantage of knowing the shape of the data, and also knowing that I can avoid allocations for the strings so we're not measuring the allocator performance. With 64 microseconds per iteration, that comes out to about 15,600 parses per second. [0] https://gist.github.com/Measter/acbae474ba8e1451946630da2a2cb177 https://gist.github.com/Measter/acbae474ba8e1451946630da2a2c...
- dan-robertson 3y ago
  I don’t really understand what this is trying to prove: - you don’t seem to specify the size of the input. This is the most important omission - you are constructing an optimised representation (in this case, strict with fields in the right places) instead of a generic ‘dumb’ representation that is more like a tree of python dicts - rust is not a ‘moderately fast language’ imo (though this is not a very important point. It’s more about how optimised the parser is, and I suspect that serde_json is written in an optimised way, but I didn’t look very hard). I found[1], which gives serde_json to a dom 300-400MB/s on a somewhat old laptop cpu. A simpler implementation runs at 100-200, a very optimised implementation gets 400-800. But I don’t think this does that much to confirm what I said in the comment you replied to. The numbers for simd json are a bit lower than I expected (maybe due to the ‘dom’ part). I think my 50MB/a number was probably a bit off but maybe the python implementation converts json to some C object and then converts that C object to python objects. That might half your throughput (my guess is that this is what the ‘strict parse’ case for rustc_serialise is roughly doing). [1] https://github.com/serde-rs/json-benchmark https://github.com/serde-rs/json-benchmark

tmtvl 3y ago

The test computer may be a little slow-ish but it doesn't seem "commodity laptop picked up from local supermarket" slow. It has an SSD, for example.

dan-robertson 3y ago
I think it’s ‘thermally limited and a few years old’ which makes it a bit slower than I expected. I’ll accept that ‘old’ was a bad characterisation.

ilyt 3y ago

I probably underestimated how much does it take to convert it from "fast JSON decoder written in C" (which many interpreted languages use in backend) to actual language implementation, but in perl it's pretty fast: -> ᛯ ls -s -h big.json 101M big.json [11:53:30] ^ [/tmp] -> ᛯ cat /tmp/1.pl #!/usr/bin/perl use v5.24; use JSON::XS; use File::Slurp; my $j = decode_json(read_file('/tmp/big.json')); say scalar @{$j->{'a'}} [11:53:39] ^ [/tmp] -> ᛯ time /tmp/1.pl 34952534 /tmp/1.pl 0,81s user 0,13s system 99% cpu 0,939 total so just around 100MB/s

Measter 3y ago

For comparison, using Rust's Serde-Json library on my desktop (Intel 11700k, and with computer doing other things), benchmarked with Criterion: Benchmarking json_read_struct: Collecting 100 samples in estimated 5.2012 s (81k iterations) json_read_struct time: [64.126 µs 64.376 µs 64.660 µs] change: [+0.2646% +0.7525% +1.2538%] (p = 0.00 < 0.05) Change within noise threshold. Found 7 outliers among 100 measurements (7.00%) 6 (6.00%) high mild 1 (1.00%) high severe My benchmark code[0] takes advantage of knowing the shape of the data, and also knowing that I can avoid allocations for the strings so we're not measuring the allocator performance. With 64 microseconds per iteration, that comes out to about 15,600 parses per second. [0] https://gist.github.com/Measter/acbae474ba8e1451946630da2a2cb177 https://gist.github.com/Measter/acbae474ba8e1451946630da2a2c...

dan-robertson 3y ago
I don’t really understand what this is trying to prove: - you don’t seem to specify the size of the input. This is the most important omission - you are constructing an optimised representation (in this case, strict with fields in the right places) instead of a generic ‘dumb’ representation that is more like a tree of python dicts - rust is not a ‘moderately fast language’ imo (though this is not a very important point. It’s more about how optimised the parser is, and I suspect that serde_json is written in an optimised way, but I didn’t look very hard). I found[1], which gives serde_json to a dom 300-400MB/s on a somewhat old laptop cpu. A simpler implementation runs at 100-200, a very optimised implementation gets 400-800. But I don’t think this does that much to confirm what I said in the comment you replied to. The numbers for simd json are a bit lower than I expected (maybe due to the ‘dom’ part). I think my 50MB/a number was probably a bit off but maybe the python implementation converts json to some C object and then converts that C object to python objects. That might half your throughput (my guess is that this is what the ‘strict parse’ case for rustc_serialise is roughly doing). [1] https://github.com/serde-rs/json-benchmark https://github.com/serde-rs/json-benchmark