4 ms·
What amount of data are you ingesting per day and what is the cost? How much time do you spend maintaining Druid? Do you have dedicated SRE/devops people mana
by mej10 10y ago
What amount of data are you ingesting per day and what is the cost?
How much time do you spend maintaining Druid?
Do you have dedicated SRE/devops people managing it or is it something a dev could do with minimal time investment?
Answers to any of these would be super helpful -- it is hard to find details on these aspects of real world uses of Druid.
- gillh 10y agoWe are packaging our product as a self-hosted machine images (www.netsil.com/download). We are using DC/OS runtime to orchestrate all our internal services and have written various health check scripts to make sure the data pipeline components such as our stream-processor, Kafka and Druid stay operational. This blog post describes Netsil's architecture: https://blog.netsil.com/listen-to-your-apis-see-your-apps-a91ece5aa5bd https://blog.netsil.com/listen-to-your-apis-see-your-apps-a9... In-terms of data being ingested -- we are stream-processing service interactions (network packets!) in real-time, storing interaction logs in memory and rolling them up to generate multidimensional timeseries data emitted every second. Therefore, the actual timeseries ingestion rates by Druid vary across deployments.