A useful file format to deal with this kind of problem is JSONL, which is just a list of newline separated json blobs. You can then simply append things to the

A useful file format to deal with this kind of problem is JSONL, which is just a list of newline separated json blobs. You can then simply append things to the file rather than having to write the whole thing each time.