4 ms·
From my experience, planning is often the first headache I have to deal with (join order, hash sizing, operator choice), before concurrency and memory even come
by hero-24 8mo ago
From my experience, planning is often the first headache I have to deal with (join order, hash sizing, operator choice), before concurrency and memory even come into play.
- exagolo 8mo agoYou mean the "execution plan" for your queries? Ideally, those types of decisions are automatically done by the database.
- hero-24 8mo agoideally? yes. in practice? big nope. How you actually interpret what you're seeing here? does it look like more like optimizer fragility (plans that assume ideal memory conditions) or more like runtime memory management limits (good plans, but no adaptive behavior under pressure)?
- exagolo 8mo agoI think the issue in the tests was the lack of a proper resource management of Clickhouse that led to queries failing under pressure. Although I have to admit that the level of pressure was minimal. Just a few concurrent users shouldn't be considered pressure. Also, having far more RAM than the whole database size means very little pressure. And the schema model is quite simple, just two fact tables and a few dimension tables. Any database should be able to handle 100 concurrent queries robustly, even if this means to slow down the execution of queries.