4 ms·

Thanks for the answers Mike! One more follow-up on the scale side (which I mentioned with sibling comment), it isn't so much about clickhouse itself, but about

Thanks for the answers Mike! One more follow-up on the scale side (which I mentioned with sibling comment), it isn't so much about clickhouse itself, but about scaling up ingest. From my own experience and from talking with quite a few APM players (I previously worked in streaming space), a Kafka / durable log storage kind of becomes a requirement, so I was curious if you think at some point you need a log to further scale ingest. For enterprise side, I was previously in data streaming space and had quite a few conversations with APM players and companies building their own observability platforms, happy to chat and share more if that would be useful!

mikeshi42 3y ago
Ah got it, yeah a queue of some sort is definitely useful when scaling up to buffer pre-inserted data. This is something on the OSS side we've kept open to implementation. However it's something that is highly coupled with infra footprint and internal SLA guarantees the user wants to preserve. It can range anywhere from just rely on client-side retries to setting up a HA Kafka cluster early in the ingestion pipeline. Similar to Elastic - I think a lot of architectures are available to choose on that side when users want to scale. Will reach out to connect!