2 ms·
Maybe I'm reading too generously, but my sense is that the format described in the article is intended to be used for encoding and transferring data to be used
by saghm 3y ago
Maybe I'm reading too generously, but my sense is that the format described in the article is intended to be used for encoding and transferring data to be used in some other application rather than something where you can assume constraints like "no punctuation". I could imagine using the "JSON lines" format for things like encoding a large number of records to insert for test cases or as a way of sharing a test dataset with another developer; I'm not convinced that either of those would work well without any punctuation whatsoever, and even if they could, I suspect I'd still rather pay the cost of slower importing in order to make my testing code (and any manual reading/transformation outside of my code) easy to do without having to add some additional processing step to deal with the punctuation.
The best argument against this format to me is that it's possible to just use JSON itself for this by wrapping all of the values in an array. I'm not totally convinced that the quality of life improvements of not needing to add commas along with the single pair of square braces necessitate an entirely new file format, but realistically I think I'll just end up following whatever happens to be supported by the tooling I use for a given project. If "JSON lines" became widely used overnight, I wouldn't be bothered by it, but if I never end up seeing it supported by anything, I doubt I'll miss it.
- Arch485 3y agoOne particular downside to using an actual array instead of something like JSON lines is that you can't really stream the data. If you have so much data that you need to stream it, you probably shouldn't be using JSON, though...
- fermisea 3y agoShopify, for instance, uses this format if you request bulk downloads of records, for this exact reason.
- vsnf 3y ago>If you have so much data that you need to stream it, you probably shouldn't be using JSON, though... Why not? I have a bunch of large/real-time files I'd like to read piece by piece and json is a convenient format for parsing it to something useful on the client. I understand using other, more efficient formats, but I think just wanting to stream the data isn't enough of a reason to move to them.
- Arch485 3y agoWell, I said _need_ to stream it. I think that's slightly different: I _want_ to stream data almost all the time, but I _need_ to stream data (e.g. due to RAM limitations) fairly infrequently. If you have so much data (e.g. 8GB+, yes my workstation "only" has 8GB of RAM) that it _must_ be streamed, in my personal opinion you should opt for a better format than JSON. It's likely, too, that with a more specialized/space-efficient format, you may no longer need to stream the data!
- baq 3y agoYou can, this has been done since xml was a thing (yes some of us are old). The api to do that is necessarily horrible, so no one sane does, though.