3 ms·
I've been using duckdb's json import and parsing to deal with a O(50k) item scrape of an API into newline delimited json to convert it into something more usabl
by wiredfool 3y ago
I've been using duckdb's json import and parsing to deal with a O(50k) item scrape of an API into newline delimited json to convert it into something more usable and found it quite nice to use.
syntax like:
create table foo as (select * from read_json_auto('foos.jsonl'))
and for a bit that was ... overly nested.
create table nested as
select unnest(json) from (
select unnest(arr) as json from (
select json as arr from read_json_auto('nested.jsonl')));
And then exporting that as a fixture for django is as easy as:
copy (select uid as pk, 'app.Model' as model, tbl as fields from tbl) to 'tbl.jsonl' (format json, array false);