3 ms·
You write to S3 in chunks for long term storage and keep hot data live. You don’t write every metric directly. Along with compression It shouldn’t be that 5 bil
by deepakhj 6y ago
You write to S3 in chunks for long term storage and keep hot data live. You don’t write every metric directly. Along with compression It shouldn’t be that 5 billion/s.
- RhodesianHunter 6y agoEven with chunks it's prohibitively expensive at this scale, and you then need some intermediate system batching the inbound data either to disk (then why use S3 instead of a database) or in memory (more expensive, more risk of data loss).