3 ms·
I'm curious about where the threshold lies between "ELK Stack is good enough" and "We need Parquet".
by whatnotests 10y ago
I'm curious about where the threshold lies between "ELK Stack is good enough" and "We need Parquet".
- coredog64 10y agoMy employer is doing a lot of this, mostly for cost reduction. Keeping live data in ElasticSearch (or some other horizontally scalable system) is expensive. Far cheaper to keep a small portion live and the rest in S3. Also worth mentioning that managed ES in AWS has a hard limit of 20 data nodes with 512GB of EBS. We've outgrown that and are now looking hard at alternatives like this.