4 ms·

With an analytical database like ClickHouse, you can write many rows with a single INSERT statement (thousands of rows, millions of rows, and more). In fact, th

With an analytical database like ClickHouse, you can write many rows with a single INSERT statement (thousands of rows, millions of rows, and more). In fact, this kind of batching is recommended. Larger inserts will consume more write units than smaller inserts. Check out our billing FAQ for some examples, and we will be enhancing it with more detail as questions from our users come in (we'll work on clarifying this specific point): https://clickhouse.com/docs/en/manage/billing/ https://clickhouse.com/docs/en/manage/billing/ We also provide a $300 credit free trial to try out your workload and see how it translates to some of the usage dimensions. Finally, this is a Beta product, so keep the feedback coming!

gsanderson 4y ago
Thanks, I agree batching inserts would indeed be a good idea and it makes sense that's recommended. However that link you mention (as of now) does not specify what a write unit is. So if that could be clarified, that would be great. Since from your reply it sounds like one INSERT would indeed (at a minimum) incur one write unit. And thus 100 writes could indeed cost $1.25. Which could get expensive, fast.
- tbragin 4y ago
  An INSERT can consume less than one write unit, depending on how many rows and bytes it writes to how many partitions, columns, materialized views, data types, etc. So, a "write unit", which corresponds to hundreds of low-level write operations, typically translates to many batched INSERTS. We are working to improve our examples in the FAQ to clarify - thank you so much for asking the question!
  - ok_dad 4y ago
    You know what would be really helpful? Some tooling around measuring "write units", like a fake local ClickHouseCloud API you could submit a write to and see how many "write units" that particular write would take. Of course, that would make pricing transparent and easy and SaaS companies don't seem like that. I challenge you folks to actually figure out a way to prevent surprise bills, until then all this "write unit" stuff is bullshit.
  - mike_d 4y ago
    > We are working to improve our examples in the FAQ to clarify Based on the comments here, and my own confusion, I think figuring out a different way of billing read/write operations is in order.
mrwnmonm 4y ago
"Each write operation (INSERT, DELETE, etc) consumes write units depending on the number of rows, columns, and partitions it writes to."
benjaminwootton 4y ago
How about if you are streaming in from Kafka and inserting each event as if arrives? Clickhouse is ideal for rapid analytics over event data so to introduce batching would be disappointing. Batch upload is of course more cost effective, but I would expect that to be more typical in a traditional data warehouse where we are uploading hourly or daily extracts. Clickhouse would likely show up in more real time and streaming architectures where events arrive sporadically. I am a huge fan and advocate of Clickhouse, but the concept of a write unit is strange and the likely charges if you have to insert small batches would make this unviable to use. A crypto analytics platform I built on top of Clickhouse would cost in the $millions per month vs $X00 or low $X000 with competitors or self hosting.

gsanderson 4y ago

Thanks, I agree batching inserts would indeed be a good idea and it makes sense that's recommended. However that link you mention (as of now) does not specify what a write unit is. So if that could be clarified, that would be great. Since from your reply it sounds like one INSERT would indeed (at a minimum) incur one write unit. And thus 100 writes could indeed cost $1.25. Which could get expensive, fast.

tbragin 4y ago
An INSERT can consume less than one write unit, depending on how many rows and bytes it writes to how many partitions, columns, materialized views, data types, etc. So, a "write unit", which corresponds to hundreds of low-level write operations, typically translates to many batched INSERTS. We are working to improve our examples in the FAQ to clarify - thank you so much for asking the question!
- ok_dad 4y ago
  You know what would be really helpful? Some tooling around measuring "write units", like a fake local ClickHouseCloud API you could submit a write to and see how many "write units" that particular write would take. Of course, that would make pricing transparent and easy and SaaS companies don't seem like that. I challenge you folks to actually figure out a way to prevent surprise bills, until then all this "write unit" stuff is bullshit.
- mike_d 4y ago
  > We are working to improve our examples in the FAQ to clarify Based on the comments here, and my own confusion, I think figuring out a different way of billing read/write operations is in order.

mrwnmonm 4y ago

"Each write operation (INSERT, DELETE, etc) consumes write units depending on the number of rows, columns, and partitions it writes to."

benjaminwootton 4y ago

How about if you are streaming in from Kafka and inserting each event as if arrives? Clickhouse is ideal for rapid analytics over event data so to introduce batching would be disappointing. Batch upload is of course more cost effective, but I would expect that to be more typical in a traditional data warehouse where we are uploading hourly or daily extracts. Clickhouse would likely show up in more real time and streaming architectures where events arrive sporadically. I am a huge fan and advocate of Clickhouse, but the concept of a write unit is strange and the likely charges if you have to insert small batches would make this unviable to use. A crypto analytics platform I built on top of Clickhouse would cost in the $millions per month vs $X00 or low $X000 with competitors or self hosting.