5 ms·
I'm just curious - is duckdb too slow for people? This benchmark from clickhouse shows it being fairly slow compared to some options: https://jsonbench.com/ htt
by snissn 4mo ago
I'm just curious - is duckdb too slow for people? This benchmark from clickhouse shows it being fairly slow compared to some options: https://jsonbench.com/ https://jsonbench.com/
- conradkay 4mo agoThat's for their `JSON` data types. In DuckDB it's just a string meaning lots of queries will have to do JSON parsing on every row, but the inserts are very fast. Definitely a bit of a footgun and when you actually just need STRUCT or MAP. There's a talk about ClickHouse's approach from its creator: https://www.youtube.com/watch?v=xHj9mysh0GI https://www.youtube.com/watch?v=xHj9mysh0GI , but the gist is that it maintains (sub)columns to store different paths in the JSON In other ways DuckDB has very good JSON support, like you can do `CREATE TABLE name AS `SELECT * FROM 'data.json';` and it'll infer the schema when possible.
- hilariously 4mo agoNot just that, you can SELECT * FROM read_json('folder/*') and read an entire schematically consistent folder of json files, or parquet files, or basically anything it supports with its various functions. Duckdb is insanely useful.
- thijsbruin 3mo agoHi, DuckDB engineer here, to tackle this problem of efficiently storing and querying semi-structured data (aka JSON), we've put in a lot of work into the VARIANT type and still continue to pour a lot more work into it regularly, so expect to see some developments on this area in the upcoming major release!