well, jsonlines works extremly well for gigabytes of short json objects. You can mix and match usual unix tools, or simple scripts with just "read" in a loop, a

well, jsonlines works extremly well for gigabytes of short json objects. You can mix and match usual unix tools, or simple scripts with just "read" in a loop, and you can parse each json object without worrying it'll take all your memory. jq can convert arrays to json lines and back. I love the format.