3 ms·
> Performance is not good when dataset is huge. Program usually needs to load all data into memory first. You could use a streaming parser, or, more easily, ND
by drej 8y ago
> Performance is not good when dataset is huge. Program usually needs to load all data into memory first.
You could use a streaming parser, or, more easily, NDJSON (assuming your data is huge, because it's a list of stuff). Just save each JSON as a line in a file and then stream the file line by line, only parsing one line at a time. That's fairly fast and allocates very little memory.