4 ms·

AWS here. This varies wildly by team and doesn't match my experience at all. I've been on 3 teams over 6 years in 3 locations (though none in US so maybe this i

AWS here. This varies wildly by team and doesn't match my experience at all. I've been on 3 teams over 6 years in 3 locations (though none in US so maybe this is why? #AmericaNo) on aws and retail. I don't know anyone that works 60 hours regularly. 50, maybe - you know including lunch. Oncall for the vast majority of teams isn't extra hours, just strange hours. But if you get paged at night people cover for you the next day. I didn't work on a Q4 peak team but your math is psychotic. 100 hours is 14 hours a day 7 days a week? If you went through that I'm sorry, I suspected our Ops org is psychotic, but that is too much. Still, i really have to wonder if you are exaggerating since the other 2 numbers were dramatically. Or you got an Outlier team. My org crunches for reinvent, sure, but what company wouldn't with a massive yearly conference where/when most of the products are launched.

thesuperbigfrog 7y ago
I was in Seattle the whole time. It was common on the team I was on to work about 10 hours a day in office and then go home and have meetings with our sister team in India and some follow-up work with them. The sister team was working on standing up an automation service to reduce some of our "manual" work and help reduce the oncall burden, but it took nearly a year and a half for that to happen. >> if you get paged at night people cover for you the next day. That would have been nice. We initially had to provide 24-7 coverage for a week during our oncall rotation. After the sister team in India stood up, it turned into 12 hours a day and a hand off to an oncall engineer in India who would cover the next 12 hours. Both "tag team" engineers would cover a week together for the oncall rotation. >> I didn't work on a Q4 peak team but your math is psychotic. 100 hours is 14 hours a day 7 days a week? Our service directly supported order fulfillment. Therefore it HAD to stay up at all costs, especially during Q4. Granted 100 hours did not happen often, it was usually closer to 70 and mainly consisted of getting everything ready for Q4 and load testing it all to make sure that it was solid for the tremendous holiday volume. Remember, this was BEFORE the retail website had fully transitioned to elastic scaling. All the supporting layers and configuration were pre-scaled manually based on the previous year's peaks plus the expected increase.

thesuperbigfrog 7y ago

I was in Seattle the whole time. It was common on the team I was on to work about 10 hours a day in office and then go home and have meetings with our sister team in India and some follow-up work with them. The sister team was working on standing up an automation service to reduce some of our "manual" work and help reduce the oncall burden, but it took nearly a year and a half for that to happen. >> if you get paged at night people cover for you the next day. That would have been nice. We initially had to provide 24-7 coverage for a week during our oncall rotation. After the sister team in India stood up, it turned into 12 hours a day and a hand off to an oncall engineer in India who would cover the next 12 hours. Both "tag team" engineers would cover a week together for the oncall rotation. >> I didn't work on a Q4 peak team but your math is psychotic. 100 hours is 14 hours a day 7 days a week? Our service directly supported order fulfillment. Therefore it HAD to stay up at all costs, especially during Q4. Granted 100 hours did not happen often, it was usually closer to 70 and mainly consisted of getting everything ready for Q4 and load testing it all to make sure that it was solid for the tremendous holiday volume. Remember, this was BEFORE the retail website had fully transitioned to elastic scaling. All the supporting layers and configuration were pre-scaled manually based on the previous year's peaks plus the expected increase.