5 ms·

How are you proposing to run a complex system that essentially powers a big chunk of the entire internet and can go down any second without on-call? If a big s

How are you proposing to run a complex system that essentially powers a big chunk of the entire internet and can go down any second without on-call? If a big selling point of your system is the ability to be on 24/7, on-call is inevitable. But it definitely can be way less gruesome than it currently is at Amazon for a lot of teams right now. For example, my on-call schedule is extremely relaxed and low-stress, it is never a worry I actually have or something I dread ahead of time. But getting rid of it completely is simply impossible for a lot of "always online" services.

Barrin92 6y ago
>How are you proposing to run a complex system that essentially powers a big chunk of the entire internet and can go down any second without on-call? hire more people?
- filoleg 6y ago
  Hire more people to do what? Be on-call? Hiring more people is a great idea for making on-call less stressful, because it allows for much sparser on-call shifts (e.g., one week every 8 months vs. one week every 2 months). But I don't see how hiring more people gets rid of on-call completely. Someone will always have to be on-call. You cannot just hire people who do on-call-only duties and nothing else, you have to be very closely familiar with the code and features being written in order to resolve issues effectively. Which usually means that you need to be someone who actually worked on those things, not just some glorified extra-fancy customer-support worker.
  - grumple 6y ago
    If the concern is about performance during business hours in China, you should have employees in China. Or hire a night shift.
  - smacktoward 6y ago
    According to TFA, programmers at Amazon are required to be on call for a full week at a time. More programmers could mean sparser on call shifts, but it could also mean shorter on call shifts -- i.e. instead of going from one week every two months to one every eight months, go to two or three days every two months. Being effectively at work 24 hours a day for seven days straight sure sounds like a recipe for burnout to me. As TFA notes, these people aren't doctors, the only thing that happens if a glitch takes two hours to fix rather than one is that Amazon loses some money. Amazon naturally would prefer not to lose that money, but its workers presumably would prefer to be able to sleep too.
- devdas 6y ago
  On-call will still rotate amongst the developers. They wrote the system, they know how it breaks and can fix it.

Barrin92 6y ago

>How are you proposing to run a complex system that essentially powers a big chunk of the entire internet and can go down any second without on-call? hire more people?

filoleg 6y ago
Hire more people to do what? Be on-call? Hiring more people is a great idea for making on-call less stressful, because it allows for much sparser on-call shifts (e.g., one week every 8 months vs. one week every 2 months). But I don't see how hiring more people gets rid of on-call completely. Someone will always have to be on-call. You cannot just hire people who do on-call-only duties and nothing else, you have to be very closely familiar with the code and features being written in order to resolve issues effectively. Which usually means that you need to be someone who actually worked on those things, not just some glorified extra-fancy customer-support worker.
- grumple 6y ago
  If the concern is about performance during business hours in China, you should have employees in China. Or hire a night shift.
- smacktoward 6y ago
  According to TFA, programmers at Amazon are required to be on call for a full week at a time. More programmers could mean sparser on call shifts, but it could also mean shorter on call shifts -- i.e. instead of going from one week every two months to one every eight months, go to two or three days every two months. Being effectively at work 24 hours a day for seven days straight sure sounds like a recipe for burnout to me. As TFA notes, these people aren't doctors, the only thing that happens if a glitch takes two hours to fix rather than one is that Amazon loses some money. Amazon naturally would prefer not to lose that money, but its workers presumably would prefer to be able to sleep too.
devdas 6y ago
On-call will still rotate amongst the developers. They wrote the system, they know how it breaks and can fix it.