3 ms·
I think parsing once into a faster format (sqlite3 or parquet) would be more beneficial. https://simdjson.org/ https://simdjson.org/
by emmelaich 2y ago
I think parsing once into a faster format (sqlite3 or parquet) would be more beneficial.
https://simdjson.org/ https://simdjson.org/
- llimllib 2y agoFor kicks, I downloaded the file and compared duckdb with the spatial extension against jq. jq was about 2x as fast for interactive use, but if you had multiple queries to run, paying a small cost to create a duckdb database and then querying it would be vastly faster
- datacynic 2y agoI also was nerdsniped into trying this and found that after extracting the features array into a newline delimited json file, DuckDB finishes the example query in 500 ms (M1 Mac), querying the 1.3 GB json file directly with read_json!
- wiredfool 2y agoYeah, any of GeoPackage (sqllite db with spatial extensions), FlatGeoBuf, or GeoArrow/GeoParquet would be far better for querying.