3 ms·
>I would think that Amazon of all places would automate anything they could. It is all automated. To the point of paging you when an alarm goes off. But when t
by SomeCallMeTim 10y ago
>I would think that Amazon of all places would automate anything they could.
It is all automated. To the point of paging you when an alarm goes off. But when the alarms go off, someone has to figure out how to fix the problem. Apparently some newer developers dug into the problem, they were able to fix it so that things were Much More Sane.
On our particular team, the server guys had just set too many alarms without really:
a) Documenting exactly why a particular alarm was potentially bad
b) Documenting what to do when a particular alarm went off
c) Really thinking about alarm thresholds -- frequently alarms were going off just because a client was using the product
The hell of oncall for our team was that we were the client team. We knew nothing about the server side. So if someone on our team was oncall and got paged, they'd basically have to call someone on the server team at 3am or whenever anyway because WTF is this stupid alarm?!
I had the Weirdest Commute Ever [1], though, and was as a side effect exempt.
[1] I live in Colorado. Amazon send me an offer to be an employee that allowed me to work from home ... but not in Colorado, because sales tax. So I rented a house on the edge of Kansas (3 hour drive), worked M-W, 12 hours/day, and drove home for the weekends. I couldn't do any work in Colorado or Amazon would risk having to pay sales tax on all purchases from Colorado. So QED I couldn't be oncall. Amazon has offices here now, so I could work for them without the weird commute. But as I mentioned elsewhere in this thread, I'm currently obsessing over finishing a game.