8 ms·

PyPy's New JSON Parser

192 points by gok 7y ago

slovenlyrobot 7y ago
This post touches briefly on ujson. I found it a long time ago, and while I can't vouch for the quality of its implementation, it's very popular and seems to work just fine. Any time I've encountered a bottleneck due to parsing, dropping ujson.loads() in has made it completely vanish from profiling output.
- superwayne 7y ago
  I think there are even faster JSON parsers out there nowadays. I have recently used orjson and found it to be extremely fast for our use case (serializing a dict with about 100 MB).
- ambivalence 7y ago
  Please note that according to the benchmark in this blogpost, CPython's builtin JSON decoder is faster than ujson.
  - wyldfire 7y ago
    It's not strictly the case (`reddit` case). But the difference is pretty insignificant compared to Node and PyPyFull.
  - gshulegaard 7y ago
    That result is actually why I am healthily skeptical of the performance comparisons specifically in this post. Like the previous comment I too have been in a situation where JSON encode/decode on large payloads were significant enough that we decided we needed to figure out something more optimal. Our solution was to use ujson instead (of the built in) and that was good enough that we moved on. When I say skeptical what I mean is that I have no doubt the measurements are accurate (as in not fabricated); however, I wonder what the full context of the benchmarks are and whether or not it's representative of situations like mine where the built-in parser started to fall over so we replaced it. It may also be the case that the PyPy parser would also beat ujson in this case, but if we are limiting our test case to just those where the CPython implementation is marginally faster than ujson then I'm not sure it's that useful a benchmark. I too cannot comment on the quality of the implementation or the relative performance against other parsers...but it solved a performance problem for at least one project of mine in the past.
    - kmod 7y ago
      Agreed, I've gotten huge speedups from ujson in the past and the numbers in the post are markedly different from my experience, making me think that they are measuring something different / there is something significant about this use case that is different than the ones I've had in the past.
      - weberc2 7y ago
        The post is profiling large payloads? Maybe ujson is optimized for smaller payloads?
umvi 7y ago
> In the last year or two I have worked on and off on making PyPy's JSON faster, particularly when parsing large JSON files I think this is the key takeaway. If you are parsing huge JSON documents, this will make a difference. Otherwise, it's super hard to beat Python3's native implementation. I've tried them all - hyperjson, orjson, ujson... none of them provide any significant gains on parse speed of smallish JSON documents, even if you are parsing thousands of small(ish) JSON documents in a row.
- bjoli 7y ago
  I have had pretty large speedups in the past when switching JSON implementations. About 5x. It doesn't matter much when the times are 0.2 Vs 1s, but when you are working with a couple of MB of JSON the difference will definitely be noticeable. Python's STDlib is nice, but rarely spectacular. An old naive CSV parser I wrote for Chicken scheme (using a simple state machine) beat python's C-based one.
  - ehsankia 7y ago
    You're basically agreeing with the above, and I have experienced the same. The only place where this makes a difference is when the JSON file is in MBs. For smaller file, it's not really worth the extra library.
    - IanCal 7y ago
      Depends on how many of the small docs you deal with, I see useful differences on small documents because I process reasonable amounts of them. A 10s change on a million docs for just decoding for a single library is easily worth it for me.
  - acdha 7y ago
    > Python's STDlib is nice, but rarely spectacular. An old naive CSV parser I wrote for Chicken scheme (using a simple state machine) beat python's C-based one. The stdlib is also conservative: I've run into quite a few cases where someone had reported huge gains until they realized they weren't properly handling escaping, Unicode, etc. and once updated the gap narrowed considerably.
mhd 7y ago
From the Perl side, there's been an interesting talk by Graham TerMarsch[1] where he tried quite a few of the allegedly fastest JSON parsers and compared them to the standard solution (JSON::XS). Apparently a lot got lost "in translation", when converting the C/C++ data. [1]: https://www.youtube.com/watch?v=BToZ_E3vU0Y https://www.youtube.com/watch?v=BToZ_E3vU0Y
- rurban 7y ago
  Interesting comparison of most fast json parsers. Just https://github.com/lemire/simdjson https://github.com/lemire/simdjson is missing. Purple beating RapidJSON. The standard is btw. my Cpanel::JSON::XS and he didn't compare with cperl which would be much faster and more secure. And he misinterpreted why glibc made a big difference. - march=native got better, not malloc.
- fnord123 7y ago
  I would like a json-game like csv-game[0] that benchmarks all of these together. Here is an example with a /lot/ of C++ parsers: https://github.com/miloyip/nativejson-benchmark https://github.com/miloyip/nativejson-benchmark [0] https://github.com/miloyip/nativejson-benchmark https://github.com/miloyip/nativejson-benchmark
xtreak29 7y ago
Python 3.9 would also include some optimizations for parsing large strings : https://bugs.python.org/issue37587 https://bugs.python.org/issue37587
- jfkw 7y ago
  while reading that bug, I am reminded how amazed I am that the Roundup issue tracker [1] has continued to serve Python development after all this time and through transitions in hosting and workflow. Roundup is neat, and I'm glad it exists. [1] http://roundup.sourceforge.net/ http://roundup.sourceforge.net/
  - linsomniac 7y ago
    Ka-Ping Yee, the original author of Roundup, has done some pretty awesome things. I remember one time I was BSing with a recruiter in the hallway at PyCon, and Ping walked by behind the recruiter. The recruiter stopped talking to me mid-sentence and gave chase: "Ping! Ping! Can I talk to you for a minute ..."
  - xtreak29 7y ago
    There is an accepted PEP to transition to GitHub issues : https://www.python.org/dev/peps/pep-0581/ https://www.python.org/dev/peps/pep-0581/ . But there is another PEP to keep using roundup to address concerns raised.
    - Nicksil 7y ago
      >But there is another PEP to keep using roundup to address concerns raised. Seems to be this one: www.python.org/dev/peps/pep-0595/ Please correct if I'm wrong.
      - xtreak29 7y ago
        Yes, maintainer of the bpo also showed some demos of REST API addition to build other features and gsoc student working on few features like markdown support during this year core dev sprints. There is a separate pep for actual migration https://www.python.org/dev/peps/pep-0588/ https://www.python.org/dev/peps/pep-0588/ . I am not sure when it would take effect or so given the depth of bpo in core dev workflow. There is also an open issue to track workflows that are missing in GitHub but present in bpo : https://github.com/python/core-workflow/issues/359 https://github.com/python/core-workflow/issues/359
sontek 7y ago
Python RapidJSON: https://github.com/python-rapidjson/python-rapidjson https://github.com/python-rapidjson/python-rapidjson Is pretty amazing. Its the first library we add to most of our projects. I wonder if it would make sense to port it to using cffi so that pypy can benefit?
gsaga 7y ago
>This is completely transparent to the Python programmer, the dictionary will look completely normal to the Python program but its internal representation is different. Shouldn't it be opaque.
- ptx 7y ago
  The optimization is transparent because the representation is opaque, I guess?

192 points by gok 7y ago

This post touches briefly on ujson. I found it a long time ago, and while I can't vouch for the quality of its implementation, it's very popular and seems to work just fine. Any time I've encountered a bottleneck due to parsing, dropping ujson.loads() in has made it completely vanish from profiling output.

superwayne 7y ago
I think there are even faster JSON parsers out there nowadays. I have recently used orjson and found it to be extremely fast for our use case (serializing a dict with about 100 MB).
ambivalence 7y ago
Please note that according to the benchmark in this blogpost, CPython's builtin JSON decoder is faster than ujson.
- wyldfire 7y ago
  It's not strictly the case (`reddit` case). But the difference is pretty insignificant compared to Node and PyPyFull.
- gshulegaard 7y ago
  That result is actually why I am healthily skeptical of the performance comparisons specifically in this post. Like the previous comment I too have been in a situation where JSON encode/decode on large payloads were significant enough that we decided we needed to figure out something more optimal. Our solution was to use ujson instead (of the built in) and that was good enough that we moved on. When I say skeptical what I mean is that I have no doubt the measurements are accurate (as in not fabricated); however, I wonder what the full context of the benchmarks are and whether or not it's representative of situations like mine where the built-in parser started to fall over so we replaced it. It may also be the case that the PyPy parser would also beat ujson in this case, but if we are limiting our test case to just those where the CPython implementation is marginally faster than ujson then I'm not sure it's that useful a benchmark. I too cannot comment on the quality of the implementation or the relative performance against other parsers...but it solved a performance problem for at least one project of mine in the past.
  - kmod 7y ago
    Agreed, I've gotten huge speedups from ujson in the past and the numbers in the post are markedly different from my experience, making me think that they are measuring something different / there is something significant about this use case that is different than the ones I've had in the past.
    - weberc2 7y ago
      The post is profiling large payloads? Maybe ujson is optimized for smaller payloads?

umvi 7y ago

> In the last year or two I have worked on and off on making PyPy's JSON faster, particularly when parsing large JSON files I think this is the key takeaway. If you are parsing huge JSON documents, this will make a difference. Otherwise, it's super hard to beat Python3's native implementation. I've tried them all - hyperjson, orjson, ujson... none of them provide any significant gains on parse speed of smallish JSON documents, even if you are parsing thousands of small(ish) JSON documents in a row.

bjoli 7y ago
I have had pretty large speedups in the past when switching JSON implementations. About 5x. It doesn't matter much when the times are 0.2 Vs 1s, but when you are working with a couple of MB of JSON the difference will definitely be noticeable. Python's STDlib is nice, but rarely spectacular. An old naive CSV parser I wrote for Chicken scheme (using a simple state machine) beat python's C-based one.
- ehsankia 7y ago
  You're basically agreeing with the above, and I have experienced the same. The only place where this makes a difference is when the JSON file is in MBs. For smaller file, it's not really worth the extra library.
  - IanCal 7y ago
    Depends on how many of the small docs you deal with, I see useful differences on small documents because I process reasonable amounts of them. A 10s change on a million docs for just decoding for a single library is easily worth it for me.
- acdha 7y ago
  > Python's STDlib is nice, but rarely spectacular. An old naive CSV parser I wrote for Chicken scheme (using a simple state machine) beat python's C-based one. The stdlib is also conservative: I've run into quite a few cases where someone had reported huge gains until they realized they weren't properly handling escaping, Unicode, etc. and once updated the gap narrowed considerably.

mhd 7y ago

From the Perl side, there's been an interesting talk by Graham TerMarsch[1] where he tried quite a few of the allegedly fastest JSON parsers and compared them to the standard solution (JSON::XS). Apparently a lot got lost "in translation", when converting the C/C++ data. [1]: https://www.youtube.com/watch?v=BToZ_E3vU0Y https://www.youtube.com/watch?v=BToZ_E3vU0Y

rurban 7y ago
Interesting comparison of most fast json parsers. Just https://github.com/lemire/simdjson https://github.com/lemire/simdjson is missing. Purple beating RapidJSON. The standard is btw. my Cpanel::JSON::XS and he didn't compare with cperl which would be much faster and more secure. And he misinterpreted why glibc made a big difference. - march=native got better, not malloc.
fnord123 7y ago
I would like a json-game like csv-game[0] that benchmarks all of these together. Here is an example with a /lot/ of C++ parsers: https://github.com/miloyip/nativejson-benchmark https://github.com/miloyip/nativejson-benchmark [0] https://github.com/miloyip/nativejson-benchmark https://github.com/miloyip/nativejson-benchmark

xtreak29 7y ago

Python 3.9 would also include some optimizations for parsing large strings : https://bugs.python.org/issue37587 https://bugs.python.org/issue37587

jfkw 7y ago
while reading that bug, I am reminded how amazed I am that the Roundup issue tracker [1] has continued to serve Python development after all this time and through transitions in hosting and workflow. Roundup is neat, and I'm glad it exists. [1] http://roundup.sourceforge.net/ http://roundup.sourceforge.net/
- linsomniac 7y ago
  Ka-Ping Yee, the original author of Roundup, has done some pretty awesome things. I remember one time I was BSing with a recruiter in the hallway at PyCon, and Ping walked by behind the recruiter. The recruiter stopped talking to me mid-sentence and gave chase: "Ping! Ping! Can I talk to you for a minute ..."
- xtreak29 7y ago
  There is an accepted PEP to transition to GitHub issues : https://www.python.org/dev/peps/pep-0581/ https://www.python.org/dev/peps/pep-0581/ . But there is another PEP to keep using roundup to address concerns raised.
  - Nicksil 7y ago
    >But there is another PEP to keep using roundup to address concerns raised. Seems to be this one: www.python.org/dev/peps/pep-0595/ Please correct if I'm wrong.
    - xtreak29 7y ago
      Yes, maintainer of the bpo also showed some demos of REST API addition to build other features and gsoc student working on few features like markdown support during this year core dev sprints. There is a separate pep for actual migration https://www.python.org/dev/peps/pep-0588/ https://www.python.org/dev/peps/pep-0588/ . I am not sure when it would take effect or so given the depth of bpo in core dev workflow. There is also an open issue to track workflows that are missing in GitHub but present in bpo : https://github.com/python/core-workflow/issues/359 https://github.com/python/core-workflow/issues/359

sontek 7y ago

Python RapidJSON: https://github.com/python-rapidjson/python-rapidjson https://github.com/python-rapidjson/python-rapidjson Is pretty amazing. Its the first library we add to most of our projects. I wonder if it would make sense to port it to using cffi so that pypy can benefit?

gsaga 7y ago

>This is completely transparent to the Python programmer, the dictionary will look completely normal to the Python program but its internal representation is different. Shouldn't it be opaque.

ptx 7y ago
The optimization is transparent because the representation is opaque, I guess?