3 ms·
Rogers is also known to historically have had a somewhat 'flatter' and less mature network architecture as compared to their peers, Telus and Bell. Likely owing
by 0xbadc0de5 4y ago
Rogers is also known to historically have had a somewhat 'flatter' and less mature network architecture as compared to their peers, Telus and Bell. Likely owing to their roots as a cable television provider as opposed to landline telephony - ie: dumb pipe vs. less-dumb pipe.
Lots of things went wrong here, to name a few:
- lack of rigour in their process that allowed a critical modification without understanding its effects in context
- lack of monitoring and metrics to alert them immediately of the problem
- lack of emergency rollback capability to revert back to the known-good config
- lack of independent business-continuity comms channels. if you're a Rogers CTO or senior responsible adult and you don't have a backup Telus or Bell SIM card (and vice versa), you learned why you need one.
- Scoundreller 4y agoAlso lack of downtime procedures. They seemed to have assumed that nothing could go wrong when you should assume that everything could go wrong. From their own documentation, in mid-2015 (kinda late, but better than never), the providers issued eachother eachother's SIMs to use in these situations. Who qualified exactly is a mystery.