4 ms·
One pattern that I've used when parsing large json responses is to have a table schema like frequently|used|data|in|these|cols|full_json_response So you p
by remus 8y ago
One pattern that I've used when parsing large json responses is to have a table schema like
frequently|used|data|in|these|cols|full_json_response
So you pull out the fields you need frequently but have the full json so you can dig in to details the response on an adhoc basis. Not very efficient as you can end up storing a big blob of json, but you're not throwing away any data when you parse the response and your schema stays very manageable. Also gives you the flexibility to add new fields to the schema by pulling the data out of the json.
- TremendousJudge 8y agoCompletely agree, I do this all the time. This is a useful pattern.
- oefrha 8y agoI use this pattern a lot, too. Just make sure not to `SELECT *`, which could be much slower and waste a lot of RAM in some cases (ORMs using e.g. active record pattern are often susceptible to this when used naively).
- tracker1 8y agoI've also done similar where there are differing systems where the extra data is different, but common bits are, well common. For example supporting direct CC, Paypal and Amazon pay. Putting the common parts of the transaction as separate columns, with the full details in a JSON column.