4 ms·
Microservices for days. I worked on lifecycle ~5 years ago and just the Standard -> Glacier transition path involved no fewer than 7 microservices. Just deter
by riknos314 1y ago
Microservices for days.
I worked on lifecycle ~5 years ago and just the Standard -> Glacier transition path involved no fewer than 7 microservices.
Just determining which of the 400 trillion keys are eligible for a lifecycle action (comparing each object's metadata against the lifecycle policy on the bucket) is a massive big data job.
Always was a fun oncall when some bucket added a lifecycle rule that queued 1PB+ of data for transition or deletion on the same day. At the time our queuing had become good enough to handle these queues gracefully but our alarming hadn't figured out how to differentiate between the backlog for a single customer with a huge job and the whole system failing to process quickly enough. IIRC this was being fixed as I left.
- rubiquity 1y agoI used to work on the backing service for S3's Index and the daily humps in our graphs from lifecycle running were immense!
- jiggawatts 1y agoI work on tiny systems now, but something I miss from "big" deployments is how smooth all of the metrics were! Any bump was a signal that really meant something.