3 ms·
Running into a couple issues right out of the gate: 1) Needed to increase maximum_object_size 2) Unexpected yyjson tag in ValTypeToString Couldn't find a refe
by spullara 4y ago
Running into a couple issues right out of the gate:
1) Needed to increase maximum_object_size
2) Unexpected yyjson tag in ValTypeToString
Couldn't find a reference anywhere to that error. Loads into Snowflake without a hitch - which is where I normally query large JSON files.
- mytherin 4y agoThanks for trying it out! Could you perhaps open an issue [1] or share the file with us so we could investigate the problem? [1] https://github.com/duckdb/duckdb/issues https://github.com/duckdb/duckdb/issues
- atombender 4y agoI tried to do "select * from ... limit 1" from a 1.7GB JSON file (array of objects), and I had to increase maximum_object_size to 1GB to make it not throw an error. But DuckDB then consumed 8GB of RAM and sat there consuming 100% CPU (1 core) for ever — I killed it after about 10 minutes. Meanwhile, doing the same with Jq ("jq '.[0]'") completed in 11 seconds and consumed about 2.8GB RAM. I love DuckDB, but it does seem like something isn't right here.
- jjwiseman 4y agoTried "select * from 'data.json' limit 10" on a 6.3 MB file (which feels relatively tiny…) and got the same `unexpected end of data. Try increasing "maximum_object_size"` error. (This is my very first attempt to use duckdb, so with respect I'm not invested enough to open an issue).