3 ms·
What annoys me isn't so much that they had this outage. It's that they took so long to restore. Carrier systems are supposed to be multiply fault tolerant. Ev
by paranoidrobot 3y ago
What annoys me isn't so much that they had this outage.
It's that they took so long to restore. Carrier systems are supposed to be multiply fault tolerant.
Even if you have to put warm bodies physically in front of them, you should have enough of those people around to get the core networks up and going again within an hour or so by rolling back to a known-good configuration.
Even if they've somehow managed to brick the systems, they should have enough hot/cold spares, and the ability to call $VENDOR to hand-deliver new ones if need be.
- simfree 3y agoThe model for network delivery has changed. Networks like Rogers in Canada, Optus in Australia, and dish in the USA outsource the core knowledge needed to recover their network to vendors like Nokia and Cisco. The employees locally lack the knowledge and the access to restore the network without guidance from external vendors outside the country. From an operating cost perspective, this is the right choice, but for reliability and sovereignty it's terrible. Sadly, with RCS, 5G Standalone and other new technologies that require operating servers with leading edge software, operators repeatedly choose to outsource the entire stack to an external vendor like Nokia rather than replicate that knowledge locally.
- deleted 3y ago[deleted]