3 ms·
ndjson(new line delimited json) is what we should use as a human friendly and machine friendly alternative to csv. json is a universal format. Works in every m
by nojvek 6y ago
ndjson(new line delimited json) is what we should use as a human friendly and machine friendly alternative to csv.
json is a universal format. Works in every major language. It's spec is a state machine defining what to do with every possible encountered byte. https://www.json.org/json-en.html https://www.json.org/json-en.html
SIMDJSON parses json at RAM speeds (~3Ghz).
CSV should die. It's a terrible format. Just send arrays of ndjson instead. You'll probably have a few more " and [, but hey, you can deeply nest arrays inside arrays of objects with arrays.
https://github.com/simdjson/simdjson https://github.com/simdjson/simdjson
- tatersolid 6y agoJSON doesn’t handle int64 or explicit date-time data types, which makes it nearly as painful as CSV in practice when used for data integration. Both of those are explicit types present in nearly every real-world database schema. So you end up with many of the same problems as CSV. You basically get an application-specific JSON dialect to interpret. Header row or not? Is a long series of digits in quotes an int64? What happens when someone puts an ISO-formatted date string into a field that is supposed to be a simple string?