3 ms·
Yes, the native format is very similar to Parquet. It works for Parquet as well: SELECT * FROM url('https://clickhouse-public-datasets.s3.amazonaws.com/hits
by zX41ZdbW 4y ago
Yes, the native format is very similar to Parquet.
It works for Parquet as well:
SELECT * FROM url('https://clickhouse-public-datasets.s3.amazonaws.com/hits.parquet') LIMIT 1
And for CSV or TSV:
SELECT * FROM url('https://clickhouse-public-datasets.s3.amazonaws.com/github_events/tsv/github_events_v3.tsv.xz') LIMIT 1
And for ndJSON:
SELECT repo_name, created_at, event_type FROM s3('https://clickhouse-public-datasets.s3.amazonaws.com/github_events/partitioned_json/github_events_*.gz', JSONLines, 'repo_name String, actor_login String, created_at String, event_type String') WHERE actor_login = 'simonw' LIMIT 10
Note: the query above is kind of slow.
Here is the query from preloaded data - your activity in GitHub issues:
https://play.clickhouse.com/play?user=play#U0VMRUNUIGNyZWF0ZWRfYXQsIGFjdG9yX2xvZ2luLCBmb3JtYXQoJ2h0dHBzOi8vZ2l0aHViLmNvbS97fS9pc3N1ZXMve30nLCByZXBvX25hbWUsIG51bWJlcjo6U3RyaW5nKSBBUyB1cmwsIHRpdGxlIEZST00gZ2l0aHViX2V2ZW50cyBXSEVSRSBldmVudF90eXBlIElOICgnSXNzdWVzRXZlbnQnLCAnSXNzdWVDb21tZW50RXZlbnQnKSBBTkQgYWN0b3JfbG9naW4gPSAnc2ltb253JyBPUkRFUiBCWSBjcmVhdGVkX2F0IERFU0MgTElNSVQgMTAw https://play.clickhouse.com/play?user=play#U0VMRUNUIGNyZWF0Z...
- simonw 4y agoAnother question about that demo. https://clickhouse.com/docs/en/getting-started/example-datasets/github-events/ https://clickhouse.com/docs/en/getting-started/example-datas... says "Dataset contains all events on GitHub from 2011 to Dec 6 2020" - but I'm seeing results in there from a couple of hours ago. Do you know if that's continually updated and, if so, is that documented anywhere?
- zX41ZdbW 4y agoYes, it's continuously updated. The source code is here: https://github.com/ClickHouse/github-explorer https://github.com/ClickHouse/github-explorer This shell scripts updates it: https://github.com/ClickHouse/github-explorer/blob/main/update.sh https://github.com/ClickHouse/github-explorer/blob/main/upda...
- simonw 4y agoThanks for the info! I wrote this up as a TIL: https://til.simonwillison.net/clickhouse/github-explorer https://til.simonwillison.net/clickhouse/github-explorer