5 ms·
Not shipping on Fridays in 2023 is an infrastructure smell.
by bradhe 4y ago
Not shipping on Fridays in 2023 is an infrastructure smell.
- noman-land 4y agoTo me it seems like just good, clean living. If you have to deploy on Friday then you can, but if you don't have to, then it just seems like a good idea to do it while many people are paying attention instead of over a weekend when most people are off.
- pcdevils 4y agoInfra can make it easier to do it safer; it's not a panacea. Sure you can roll back; alert based on metrics / logs / error rates, but the knowledge required to automate around these data points tends to require the data to already exist, which won't be true with new features.
- bradhe 4y agoOkay so don't always ship on a Friday but that doesn't seem like rationale to put a blanket rule in place.
- Jensson 4y agoNo matter how you structure your infrastructure shipping an update can bring down your system. You can make it less likely, but I don't know any company that never bring down their systems with bad updates. Google does it, Amazon does it, Microsoft does it etc.
- bradhe 4y agoYour infrastructure should heal itself if it comes “down.” A decent release pipeline can fix this. Those companies don’t practice release moratoriums on Fridays.
- erik_seaberg 4y agoInfra won’t amend or refund bad transactions that were committed, I’ve seen a few people have to burn their weekends recovering from that.
- bradhe 4y agoAn exception should not dictate the rule. Sounds like there were some safety mechanisms missing. You indeed need a mature system (and engineering team) to manage this process.
- deleted 4y ago[deleted]
- ablob 4y agoFollowing your logic, is not taking of on a plane during a thunderstorm an "engineering smell"?
- bradhe 4y agoYeah...those aren't the same thing.