4 ms·

An irritatingly overlooked issue is the tension between procedure and results. Obviously, a well-refined procedure learns from past issues and evolves, growing

An irritatingly overlooked issue is the tension between procedure and results. Obviously, a well-refined procedure learns from past issues and evolves, growing ever more adaptable and useful, the way that aircraft rules are "writ in blood." Still, every one of the adaptations was prompted by an event that was Not in the Script. I think, for my next job, one of those questions I will ask when the time comes to ask if I have anything I am curious about in the interview is the choice, when a choice must be made, between procedure and results ... which am I expected to prioritize? Probably won't win me any callbacks.

treflop 2y ago
I believe this is not a question that can be asked. Recently I pushed a change straight to production by myself with no approval and violated many rules but it saved us and carried us for weeks. Worse case if it broke, rollback would happen under in under 30 seconds. I did it because not only did I triple check, I’ve kept mental track of the number of regressions and issues that have been logged against all of my work throughout my career. I’m good at determining risk and I know my bug rate is very low (I git blame every bug to find out who and why caused it. I don’t tell my coworkers but it does play a role in who gets what kind of tickets.) I did what I did because frankly I know it was going to work and no one was going to complain. And truthfully, I’ve done this at different companies several times. Of course, it’s still never a light decision and I rarely ever do it. But if someone asks me if they could do the same thing, I would not be able to tell them. They would have to keep track of the same details and to be honest, if someone is asking if they can break rules, they probably shouldn’t. This is the “tension.”
- ted_dunning 2y ago
  I once had an engineer push changes directly to production because they were confident that they would do it correctly. In fact, they were completely misguided about their own accuracy because they had systematically ignored or not understood the errors that they had made in the past. The challenge as a tech lead or manager is telling the difference between you and them. Or even telling whether you are them. My own tendency would be to fire both of you.
  - treflop 2y ago
    Your tendency may be that but the reality is my past companies want to ask me to contract for them after I leave, even 7 years later. I uphold a certain quality of work and I expect my peers to do the same. Everyone makes mistakes but even mistakes can be modeled.
eapressoandcats 2y ago
There’s not really a clear tension though. Procedures exist with an intent to get results. Airlines have lots of procedures that are carefully followed and the result is “the plane always arrives safely”. If you asked a pilot the question to “make a choice between procedures and results” they’d think you were crazy. Obviously there can be misguided procedures, but any decision made or operation done can be misguided.
- at_a_remove 2y ago
  Again, and continuing with the flight safety, all of it was an iterative process, learned the hard way. Flight safety rules did not emerge whole and unchanged with the Wright brothers. Every change was prompted by one or more incidents during which the current procedure was not appropriate. What then? Consider the current thread on the whole "toaster in the dishwasher" topic, during which someone related an incident wherein an entire server site was immersed in water but still functioning (https://news.ycombinator.com/item?id=41251234 https://news.ycombinator.com/item?id=41251234). The site manager followed procedure (wait a while, not cut the power, perform risk assessment) and it resulted in total loss, but the poster wanted to "cut the power, pump the water out of the bunker ASAP and immediately clean the whole lot with pure water." Here we have a tension between procedure and results. Procedure ended up causing total site loss, which was completely avoidable. Similarly, a current thread on an ER doctor not following the usual procedures during a mass casualty event was lauded. A choice had to be made. Here, results won. I just like to know this sort of thing about a work culture in advance. Letter of the law versus the spirit of the law, and so on.
  - pas 2y ago
    How would you know? How would they know? Okay, sure you will have a hunch based on the answer, but it seems something that only experience can tell. (Okay, you can ask them about their experience.. and who know how forthcoming and honest and relevant their answer will be, right?)
    - Retric 2y ago

treflop 2y ago

I believe this is not a question that can be asked. Recently I pushed a change straight to production by myself with no approval and violated many rules but it saved us and carried us for weeks. Worse case if it broke, rollback would happen under in under 30 seconds. I did it because not only did I triple check, I’ve kept mental track of the number of regressions and issues that have been logged against all of my work throughout my career. I’m good at determining risk and I know my bug rate is very low (I git blame every bug to find out who and why caused it. I don’t tell my coworkers but it does play a role in who gets what kind of tickets.) I did what I did because frankly I know it was going to work and no one was going to complain. And truthfully, I’ve done this at different companies several times. Of course, it’s still never a light decision and I rarely ever do it. But if someone asks me if they could do the same thing, I would not be able to tell them. They would have to keep track of the same details and to be honest, if someone is asking if they can break rules, they probably shouldn’t. This is the “tension.”

ted_dunning 2y ago
I once had an engineer push changes directly to production because they were confident that they would do it correctly. In fact, they were completely misguided about their own accuracy because they had systematically ignored or not understood the errors that they had made in the past. The challenge as a tech lead or manager is telling the difference between you and them. Or even telling whether you are them. My own tendency would be to fire both of you.
- treflop 2y ago
  Your tendency may be that but the reality is my past companies want to ask me to contract for them after I leave, even 7 years later. I uphold a certain quality of work and I expect my peers to do the same. Everyone makes mistakes but even mistakes can be modeled.

eapressoandcats 2y ago

There’s not really a clear tension though. Procedures exist with an intent to get results. Airlines have lots of procedures that are carefully followed and the result is “the plane always arrives safely”. If you asked a pilot the question to “make a choice between procedures and results” they’d think you were crazy. Obviously there can be misguided procedures, but any decision made or operation done can be misguided.

at_a_remove 2y ago
Again, and continuing with the flight safety, all of it was an iterative process, learned the hard way. Flight safety rules did not emerge whole and unchanged with the Wright brothers. Every change was prompted by one or more incidents during which the current procedure was not appropriate. What then? Consider the current thread on the whole "toaster in the dishwasher" topic, during which someone related an incident wherein an entire server site was immersed in water but still functioning (https://news.ycombinator.com/item?id=41251234 https://news.ycombinator.com/item?id=41251234). The site manager followed procedure (wait a while, not cut the power, perform risk assessment) and it resulted in total loss, but the poster wanted to "cut the power, pump the water out of the bunker ASAP and immediately clean the whole lot with pure water." Here we have a tension between procedure and results. Procedure ended up causing total site loss, which was completely avoidable. Similarly, a current thread on an ER doctor not following the usual procedures during a mass casualty event was lauded. A choice had to be made. Here, results won. I just like to know this sort of thing about a work culture in advance. Letter of the law versus the spirit of the law, and so on.
- pas 2y ago
  How would you know? How would they know? Okay, sure you will have a hunch based on the answer, but it seems something that only experience can tell. (Okay, you can ask them about their experience.. and who know how forthcoming and honest and relevant their answer will be, right?)
  - Retric 2y ago