4 ms·
Don't deploy on Friday and 3 other unwritten rules of software engineering
- aliswe 8y ago"Slack is famous for its role in increasing connectivity between team members, but, at what cost (aside from the monthly fee)?" And what's bikeshedding?
- g-erson 8y ago> The term was coined as a metaphor to illuminate Parkinson’s Law of Triviality. Parkinson observed that a committee whose job is to approve plans for a nuclear power plant may spend the majority of its time on relatively unimportant but easy-to-grasp issues, such as what materials to use for the staff bikeshed. -- https://en.wiktionary.org/wiki/bikeshedding https://en.wiktionary.org/wiki/bikeshedding
- Sarki 8y agoPoint "2. Maintain regular backups." should rather be "2. Maintain regular backups and validate your restoration process." How painful it can be when you realize that your backup process was wrong from the beginning and you're left with a useless gigabytes sized blob when in need to restore your DB...
- curtis 8y agoA corollary to "Don't deploy on Friday": "Don't deploy on Monday." Unless you really want to work over the weekend.
- lojack 8y agoWhy would deploying on a Monday result in working over the weekend?
- blacksmith_tb 8y agoI assume preparing for the deploy? But of course that could also have happened in the previous week, and just be ready to deploy on Monday...
- curtis 8y agoI guess it depends on why you're doing the deploy. If the whole point of doing the deploy is to pick up new code changes, then the people responsible for doing those code changes run the risk of having to work over the weekend to make sure they're ready. This can be a big problem at (for example) startups where the same engineers are responsible for writing new code and doing deploys. But it can happen at organizations of all sizes. If all you're doing is rolling out a new version of Postgres and it's supposed to just work, then a Monday deploy makes perfect sense. However, as a software engineer, the primary reason I've been involved in deployments is because we've written new code and we need to release it. In an ideal world, the code would be completed and rigorously tested by no later than Friday at 5pm. In the not-so-ideal world we actually live in, this isn't always true, which is why I prefer to target deploys for Wednesdays.
- lojack 8y ago> If the whole point of doing the deploy is to pick up new code changes, then the people responsible for doing those code changes run the risk of having to work over the weekend to make sure they're ready. I feel like you're conflating deadlines with deploys. If its ready to go on a Monday then I see no reason why you should hold off doing the deploy. If you have to work all weekend to get it ready to go by Monday then its probably because there's a deadline.
- curtis 8y agoFair enough, but I'll observe that in my experience deadlines and deploys are often strongly correlated. Management generally doesn't care when code is done, they care when it's done and deployed. I suppose my experience is skewed by the fact that I've spent a lot of my career in startups where sales (or even the CEO) want the code deployed ASAP because they want to demo it.
- esrauch 8y ago"Don't have a deployment process that takes a day ahead"
- koneles 8y agoIt's funny because I'm reading this from my work, instead of doing somethink productive
- clintonb 8y agoIf reading this increases/reaffirms your knowledge of good practices, that is productive.
- cphoover 8y agohmm I dunno... If you have CD working correctly you should be shipping multiple times a day depending on the product. If there is a problem with the deploy it should be as simple as rolling back to the container image of the last version...with K8s or ECS this takes seconds. I really hate the whole "let's plan for a midnight deploy" paradigm... and find it very unconstructive.
- sbergot 8y agoHow do you rollback a DB schema change?
- clintonb 8y agoYou deploy in phases. You only remove data in the final phase. I wrote about this at https://engineering.edx.org/django-migration-donts-f4588fd11b64 https://engineering.edx.org/django-migration-donts-f4588fd11....
- tracker1 8y agoYou can absolutely push to development, qa, uat, demo even... just don't deploy to production.
- karmakaze 8y agoThis point seems targeted more toward those who deploy infrequently, e.g. weekly deploy on Tuesdays. When working on a few microservices with several deploys per service per day per programmer-pair, not deploying on a Friday means a ~20% productivity reduction. We had one of thosr days as there was an important demo day and our work got blocked in staging. When it finally does deploy to prod, you're exposing your users to all the changes at once and if there's an issue it's much harder to isolate the cause. Our basic rule is don't deploy and leave any day of the week. If something does happen on the weekend, we're unofficially on-call via slack. I can only remember this happening maybe twice in two years.
- ogn3rd 8y agoIt's called Read-Only Friday for a reason. Unless of course you enjoy spending your weekends debugging and rolling back. No thanks.