3 ms·
If the improvements scale, you can still get benefit for smaller use cases. And even if each json is small, if you have millions of them the tiny improvements a
by Epa095 5y ago
If the improvements scale, you can still get benefit for smaller use cases. And even if each json is small, if you have millions of them the tiny improvements add up. We parse quite a bit of json, we fetch and store json regularly from external api's. We can't ask for a different format, and for each of the individual requests json makes sense, the response is only a few hundred kbs. But over time there is a lot of data. We convert them to parquet, but each json needs to be read at least once.