4 ms·
Of course, the smaller the JSON file, the shorter the amount of time spent in native code, and proportionally more time will be spent in the interpreter around
by AaronFriel 7y ago
Of course, the smaller the JSON file, the shorter the amount of time spent in native code, and proportionally more time will be spent in the interpreter around whatever looping construct is used to feed data into json.loads (and presumably process that data).
# interpreter time spent here looping
for file_name in stuff:
# interpreter: do stuff with file name, load file into string, etc.
str_data = ...
# interpreter: dispatch call to native function
# native: actually perform json.loads
data = json.loads(str_data)
# interpreter: time spent processing the data
process_data(data)
# what happens as native time goes to zero (json parse time decreases)?
# answer: interpreter time dominates
- weberc2 7y agoSo "interpreter time" is defined as "time not spent parsing JSON" and thus "the less time you spend parsing JSON the less time you spend parsing JSON"? Of course that's correct, but it's so obvious as to be unsubstantial. Surely the OP is making a more interesting point...
- AaronFriel 7y agoNo, that's it - for most json documents, the time the interpreter spends in all the stuff surrounding (and executing) json.loads dominates the time spent in native code parsing. That's why replacing json.loads with a faster native implementation doesn't result in noticeable improvements - it's speeding up the fastest bit, and if it's already 0.1% of the wall clock time, then you can't get much more than about 0.1% faster. PyPy is a JITed language though, so the interpreted bits are much faster. OP is making the point that Python 3/CPython is so slow that speeding up json.loads is a premature (and unnecessary) optimization.