3 ms·
> What if there was a way to take some SQL's logical plan and turn it into a physical plan that uses compute resources from a pool This is how ClickHouse works
by zX41ZdbW 3y ago
> What if there was a way to take some SQL's logical plan and turn it into a physical plan that uses compute resources from a pool
This is how ClickHouse works on a cluster.
For example, writing the same query as
SELECT sum(size)
FROM urlCluster('default', 'https://huggingface.co/datasets/vivym/midjourney-messages/resolve/main/data/0000{01..55}.parquet')
(note the urlCluster usage)
will give the result in 0.3 seconds on a cluster: https://pastila.nl/?00ef0aac/a54918ef6d3536fad34a5eca0e1157f3#4/SenMvm+mngkbiCGnQIXg== https://pastila.nl/?00ef0aac/a54918ef6d3536fad34a5eca0e1157f...