4 ms·
ClickHouse includes a tool which allows to run SQL queries over files/data in any supported format including TSV [1] E. g. you can use this command to see whic
by citrin_ru 5y ago
ClickHouse includes a tool which allows to run SQL queries over files/data in any supported format including TSV [1]
E. g. you can use this command to see which unix users use most memory (RSS):
ps aux | awk 'NR > 1 { print $1"\t"$6 }' | clickhouse-local -S "user String, mem UInt64" -q "SELECT user, formatReadableSize(sum(mem)*1024) as mem_total FROM table GROUP BY user ORDER BY sum(mem) DESC LIMIT 20 FORMAT PrettyCompact"
It is pretty fast - I've generated 7.7G test data for this query and it took 23 second to run the query above (using multiple threads). For comparison wc -l for this file takes 9 seconds.
[1] https://clickhouse.tech/docs/en/operations/utilities/clickhouse-local/ https://clickhouse.tech/docs/en/operations/utilities/clickho...