4 ms·
How do you determine query complexity? Do you go off of the query plan or something like looking at the columns, joins, etc.?
by yladiz 4y ago
How do you determine query complexity? Do you go off of the query plan or something like looking at the columns, joins, etc.?
- zX41ZdbW 4y agoThere are only a few ways to determine it before running the query - by the amount of rows/bytes to scan. But most of the logic happens when query is already started - by the possibility to stop it and throw exception if some metrics became too large. For example, this query will not start, because the amount of rows to scan is greater than the limit: https://play.clickhouse.com/play?user=play#U0VMRUNUICogRlJPTSB3aWtpc3RhdA== https://play.clickhouse.com/play?user=play#U0VMRUNUICogRlJPT... This query will fail with the error: https://play.clickhouse.com/play?user=play#U0VMRUNUICogRlJPTSByZWRkaXQgV0hFUkUgYm9keSBJTElLRSAnJUNsaWNrSG91c2UlJw== https://play.clickhouse.com/play?user=play#U0VMRUNUICogRlJPT... Code: 160. DB::Exception: Estimated query execution time (580.1882010677873 seconds) is too long. Maximum: 60. Estimated rows to process: 9946243959: While executing MergeTreeThread. (TOO_SLOW) (version 22.13.1.294 (official build))
- yladiz 4y agoI see. How do you determine when to kill the query before running it then? Of course you could do “if the process is running longer than X seconds kill and returns NNN status code” but that feels somewhat wasteful if you can be smarter when calculating the possible cost.
- zX41ZdbW 4y agoThe cost can be calculated, but not always. Example 1: SELECT ... FROM table WHERE key BETWEEN c1 AND c2 - here we can get the number of records and bytes to scan before even starting the query. Example 2: SELECT ... FROM table WHERE key = (SELECT ...) - here we cannot know how many records will be scanned until we will finish the subquery. Maybe some estimations can be applied... but we don't do that.