4 ms·
Rogers letter to CRTC on cause of Canada-wide service outage of July 2022
- deeesstoronto 4y agoRelated filings on outage with CRTC: https://crtc.gc.ca/otf/eng/2022/8000/c12-202203868.htm https://crtc.gc.ca/otf/eng/2022/8000/c12-202203868.htm
- _spduchamp 4y agoThat seems like A LOT of redaction.
- mike_d 4y agoNon-doc version: https://preview.ninja/04b08c61c2a02e8255bbe268fe0e4049f7230b1d/f7eb7445f21dc727 https://preview.ninja/04b08c61c2a02e8255bbe268fe0e4049f7230b...
- totony 4y agoVery light on details. The cause is discussed for 1 paragraph and then it's "Rogers understands... will work on..."
- feet 4y agoOf course, this is what happens when we privatize services. They get shittier on purpose as the owners extract as much value as profit for themselves as they can at the cost of everyone else This is the whole point, of course they'll give the same old tired lines of improving and doing better while never actually investing in making their service improve
- zaptheimpaler 4y agoSeems like most of the technical details are redacted :( This reads more like an obligatory corporate/legal response to an angry bureaucrat than a technical post-mortem.
- j_not_j 4y agoA lot of gaps not described in the letter: Lack of independent management plane for core business assets. Driving to the office is not a management plane. No external visibility of Rogers functionalities. The Kentik analysis of BGP and reachability suggests their tech staff had a lot of analysis to do before understanding of the problem. No fallback plan in case of a misstep. No immediate detection of a misstep. Four hour intervals in notifications (which implies lack of visibility into issues and absence of plans). Secrecy regarding social and first-responder impacts (censorship of 911 call volumes for instance) suggests Rogers views these facts as existential risk, or at least of very high risk to their management.