8 ms·
My experience with PyPy was that it was faster for mapreduce jobs without JSON parsing and slower for JSON-heavy jobs. Because, I assume, CPython benefits from
by gdulli 11y ago
My experience with PyPy was that it was faster for mapreduce jobs without JSON parsing and slower for JSON-heavy jobs. Because, I assume, CPython benefits from a C implementation of simplejson and PyPy doesn't. Does anyone know if this has changed? (I'll probably install 4.0 and see for myself, but curious if anyone knows what I should expect.)
- fijal 11y agoWe did put some effort into making built-in json faster. If you're using an optimized version like uJson, it'll be faster under cpython so far, but it all depends whether there is more processing or more parsing involved. We're happy to help you with benchmarks though
- gdulli 11y agoI've only used simplejson. A mapreduce job could be reading billions of lines and possibly doing very simple logic on each line, so the parsing/encoding is a big percentage of the processing, sometimes nearly all of it. I'll try 4.0 and see how it does now.
- fijal 11y agoTry builtin json instead of simplejson for pypy (it's API-equivalent)
- dorfsmay 11y agoOn big jobs (millions of json input/output) I have found ujson significantly faster than json + pypy.
- fijal 11y agoyop, as I said, ujson is probably your best bet if all you do is parse json.
- sontek 11y agoIf you are on python3 python-rapidjson is faster and safer than uJSON: https://github.com/kenrobbins/python-rapidjson https://github.com/kenrobbins/python-rapidjson >>> data2 = 1.23456789e-13 >>> rapidjson.loads(rapidjson.dumps(data2)) 1.23456789e-13 >>> ujson.loads(ujson.dumps(data2)) 0.0
- ant6n 11y agoOne has to be careful with strings in pypy, because they don't have a string-builder optimization. Or maybe they do now?