3 ms·

My deep seated fear is that some fundamental infrastructure will fail, Database, dns, DDoS, there are so many things that can go wrong. Esp with big old code ba

by zz865 5y ago

My deep seated fear is that some fundamental infrastructure will fail, Database, dns, DDoS, there are so many things that can go wrong. Esp with big old code bases. Many developers really dont care what happens in production, leaving a subset to stress out. Perhaps the solution is just be like the people who just expect problems and accept it.

pseudoramble 5y ago
I hear you, it isn't great when people are apathetic to what happens in prod, especially when there are things that can be done. That being said, here's another way to view it: of all of those deep seated fears you have, what do you know you can do to avoid certain issues and what can you actually not do? Let's take the database for example. What's the deep seated fear here? Maybe: > The disks crash, the data center bursts into flames and all the data is gone! A solution to those is perform backups and get the backups off-prem. You have to explain the risk and justifications to the company as best as you can. Once you've done that, what is left? Technically speaking, bugs in the backup script could exist, or maybe the backups get corrupted. Beyond that, there are the human ones. Maybe somebody else manages the script and breaks it, or maybe backups get unintentionally poisoned by malware getting into the backups. Or maybe even before that happens the company goes "eh, risk worth taking to save costs!" and you don't even get to implement the backups. Here's the thing: these things are _not_ in your control at this point! They are out of your hands, and with humans like us who make mistakes like us. So problems are going to be expected, especially ones you don't realize right now. But they're not in your control! And yet, you did your part, and things will continue on.
- deleted 5y ago
  [deleted]
rconti 5y ago
As someone with a background in infrastructure operations, it always felt like we were the ones who cared and stressed out, and development didn't (have to) care. These days, particularly in microservices, more and more developers are on call for their work, so they care more. Still, I find a radical misalignment between my mentality, where you CANNOT make mistakes because you're working without a net, and development, where typically there are more layers of safety. I'm far, far more cautious and anxious about making changes and breaking things; they've historically been paid to produce. If I try to have a conversation about it, they're unable to grok that their threshold for the acceptability of making mistakes is much higher than mine is. You NEED to be able to make mistakes in order to be able to produce, but it's a hard mentality shift.
- zz865 5y ago
  Yeah my sympathies, its tougher than it used to be. Wish we had you on our team, our first level support doesn't do anything except bounce processes then escalate. :)

by zz865 5y ago

pseudoramble 5y ago

I hear you, it isn't great when people are apathetic to what happens in prod, especially when there are things that can be done. That being said, here's another way to view it: of all of those deep seated fears you have, what do you know you can do to avoid certain issues and what can you actually not do? Let's take the database for example. What's the deep seated fear here? Maybe: > The disks crash, the data center bursts into flames and all the data is gone! A solution to those is perform backups and get the backups off-prem. You have to explain the risk and justifications to the company as best as you can. Once you've done that, what is left? Technically speaking, bugs in the backup script could exist, or maybe the backups get corrupted. Beyond that, there are the human ones. Maybe somebody else manages the script and breaks it, or maybe backups get unintentionally poisoned by malware getting into the backups. Or maybe even before that happens the company goes "eh, risk worth taking to save costs!" and you don't even get to implement the backups. Here's the thing: these things are _not_ in your control at this point! They are out of your hands, and with humans like us who make mistakes like us. So problems are going to be expected, especially ones you don't realize right now. But they're not in your control! And yet, you did your part, and things will continue on.

deleted 5y ago
[deleted]

rconti 5y ago

As someone with a background in infrastructure operations, it always felt like we were the ones who cared and stressed out, and development didn't (have to) care. These days, particularly in microservices, more and more developers are on call for their work, so they care more. Still, I find a radical misalignment between my mentality, where you CANNOT make mistakes because you're working without a net, and development, where typically there are more layers of safety. I'm far, far more cautious and anxious about making changes and breaking things; they've historically been paid to produce. If I try to have a conversation about it, they're unable to grok that their threshold for the acceptability of making mistakes is much higher than mine is. You NEED to be able to make mistakes in order to be able to produce, but it's a hard mentality shift.

zz865 5y ago
Yeah my sympathies, its tougher than it used to be. Wish we had you on our team, our first level support doesn't do anything except bounce processes then escalate. :)