3 ms·
Do you know how much data it had to download to run that query? Did it pull all 8GB?
by simonw 3y ago
Do you know how much data it had to download to run that query? Did it pull all 8GB?
- zX41ZdbW 3y agoThe same query takes only 1.4 seconds on my server, so I assume that the query does not read all 8 GB.
- richraposa 3y agoClickHouse just reads the values from the one column being summed: https://www.markhneedham.com/blog/2023/11/15/clickhouse-summing-columns-remote-files/ https://www.markhneedham.com/blog/2023/11/15/clickhouse-summ...
- deleted 3y ago[deleted]
- hipadev23 3y agoI ran a network monitor while running that query, it pulled down ~290MB
- simonw 3y agoAlmost exactly the same as DuckDB then - I measured ~287MB.
- riku_iki 3y agoProbably compression
- deleted 3y ago[deleted]
- daveguy 3y agoIf that's raw network transfer, it's probably just a difference in headers or MTU size. Larger MTU -> fewer headers required. Maybe a difference in network configuration that requires more or less data in the header.
- qxip 3y agoI ran the same queries and got similar results but the bandwidth utilization I measured was significantly different. On the same fly.io instance with 1vCPU/256MB both queries completed successfully but ClickHouse/chdb reached 10MB/s (max) and logically completed the count faster, while DuckDB only peaked at around 2.5MB/s. This might be due to the tiny resources but I like rock bottom measurements. Did anyone else notice a similar bandwidth utilization gap?