4 ms·

What's the betting someone pushed something to production on a Friday afternoon and has now gone to the pub?

krona 8y ago
At least to me, the fact this happens on a Friday evening suggests it's deliberate, precisely because nobody with any sense would deploy something on a Friday unless it's an emergency.
- falcon620 8y ago
  Maybe someone forgot the golden rule: fire people on Monday mornings. Never on Fridays.
  - stingraycharles 8y ago
    I thought the MBA handbook said you were supposed to fire them on Friday to minimize the chance they go rogue?
    - deleted 8y ago
      [deleted]
    - falcon620 8y ago
      https://www.businessmanagementdaily.com/1387/know-best-days-of-the-week-to-hire-fire-and-do-evaluations https://www.businessmanagementdaily.com/1387/know-best-days-... That random site happens to agree with my kind of thinking, so I copied that content in here: "Fire on Mondays. This lets dismissed workers start looking for a job right away. Make job offers on Thursdays. If candidates need time to think, you can give them one extra day. If you give them the whole weekend, they may find another offer. Give good job reviews on Fridays. It sets the mood for a good weekend, which can be a reward in itself. It also prevents satisfied workers from "kicking back" for the rest of the week. Give poor job reviews on Mondays. This provides employees time to work out improvements during the week, instead of stewing about them all weekend."
      - ihattendorf 8y ago
        One day to accept a job offer seems extremely short. If someone gives me one day to accept a job offer, chances are I'm still interviewing elsewhere and will decline unless it's my primary choice.
        deleted 8y ago
        [deleted]
        JeanMarcS 8y ago
        I guess that’s the point, to avoid thinking time. It’s from the interviewer point of view
        usrusr 8y ago
        The best candidates might come to an immediate decision if pressured that way.
        yjftsjthsd-h 8y ago
        ...yes, an immediate decision to reject an offer from a company trying to pressure them like that. Or is that what you meant?
        ZWoz 8y ago
        Companies usually don't need best for job.
- jimrandomh 8y ago
  There're plenty of other explanations. Not all failures are linked to deploys. The root cause could turn out to be a hardware failure somewhere. It could be a botched rollback of a change earlier in the week that had lesser consequences. It could be a change that was scheduled to take place on the 1st of the month, where the person who chose the live-date didn't think about the day of the week. Or it could be, as you describe it, someone without any sense.
- EpicEng 8y ago
  >because nobody with any sense would deploy something on a Friday unless it's an emergency So, this is completely irrelevant in this case due to usage patterns (a CC network should not push on a Friday unless they are staffed over the weekend), but we push on Fridays at my company. We are a clinical testing laboratory and we do not receive samples or new orders over the weekend, so it makes perfect sense to deploy on a Friday and deal with potential issues when no one else needs to get work done.
- gargravarr 8y ago
  A friend: >now I'm imagining a Visa IT bloke going to the pub, presenting his card and then shouting "oh SHIT!" and running back to work
subcosmos 8y ago
Someone can't push the fix right now because they are stuck in vim trying to save the git commit message
- subcosmos 8y ago
  AAAND they are ambien-committing ....
mmagin 8y ago
One would expect that this is some kind of very hard to anticipate cascading failure in a distributed system. I find it hard to believe that it would be possible for something affecting all Visa transactions in a region would be deployable by a individual or without extensive integration testing.
kalleboo 8y ago
A newer article says hardware failure https://www.theguardian.com/world/live/2018/jun/01/visa-outage-payment-chaos-after-card-network-crashes-live-updates https://www.theguardian.com/world/live/2018/jun/01/visa-outa...

At least to me, the fact this happens on a Friday evening suggests it's deliberate, precisely because nobody with any sense would deploy something on a Friday unless it's an emergency.

falcon620 8y ago
Maybe someone forgot the golden rule: fire people on Monday mornings. Never on Fridays.
- stingraycharles 8y ago
  I thought the MBA handbook said you were supposed to fire them on Friday to minimize the chance they go rogue?
  - deleted 8y ago
    [deleted]
  - falcon620 8y ago
    https://www.businessmanagementdaily.com/1387/know-best-days-of-the-week-to-hire-fire-and-do-evaluations https://www.businessmanagementdaily.com/1387/know-best-days-... That random site happens to agree with my kind of thinking, so I copied that content in here: "Fire on Mondays. This lets dismissed workers start looking for a job right away. Make job offers on Thursdays. If candidates need time to think, you can give them one extra day. If you give them the whole weekend, they may find another offer. Give good job reviews on Fridays. It sets the mood for a good weekend, which can be a reward in itself. It also prevents satisfied workers from "kicking back" for the rest of the week. Give poor job reviews on Mondays. This provides employees time to work out improvements during the week, instead of stewing about them all weekend."
    - ihattendorf 8y ago
      One day to accept a job offer seems extremely short. If someone gives me one day to accept a job offer, chances are I'm still interviewing elsewhere and will decline unless it's my primary choice.
      - deleted 8y ago
        [deleted]
      - JeanMarcS 8y ago
        I guess that’s the point, to avoid thinking time. It’s from the interviewer point of view
        usrusr 8y ago
        The best candidates might come to an immediate decision if pressured that way.
        yjftsjthsd-h 8y ago
        ...yes, an immediate decision to reject an offer from a company trying to pressure them like that. Or is that what you meant?
        ZWoz 8y ago
        Companies usually don't need best for job.
jimrandomh 8y ago
There're plenty of other explanations. Not all failures are linked to deploys. The root cause could turn out to be a hardware failure somewhere. It could be a botched rollback of a change earlier in the week that had lesser consequences. It could be a change that was scheduled to take place on the 1st of the month, where the person who chose the live-date didn't think about the day of the week. Or it could be, as you describe it, someone without any sense.
EpicEng 8y ago
>because nobody with any sense would deploy something on a Friday unless it's an emergency So, this is completely irrelevant in this case due to usage patterns (a CC network should not push on a Friday unless they are staffed over the weekend), but we push on Fridays at my company. We are a clinical testing laboratory and we do not receive samples or new orders over the weekend, so it makes perfect sense to deploy on a Friday and deal with potential issues when no one else needs to get work done.
gargravarr 8y ago
A friend: >now I'm imagining a Visa IT bloke going to the pub, presenting his card and then shouting "oh SHIT!" and running back to work

subcosmos 8y ago

Someone can't push the fix right now because they are stuck in vim trying to save the git commit message

subcosmos 8y ago
AAAND they are ambien-committing ....

mmagin 8y ago

One would expect that this is some kind of very hard to anticipate cascading failure in a distributed system. I find it hard to believe that it would be possible for something affecting all Visa transactions in a region would be deployable by a individual or without extensive integration testing.

kalleboo 8y ago

A newer article says hardware failure https://www.theguardian.com/world/live/2018/jun/01/visa-outage-payment-chaos-after-card-network-crashes-live-updates https://www.theguardian.com/world/live/2018/jun/01/visa-outa...