4 ms·
In the entire history of the Bell System, no electromechanical exchange was ever down for more than 30 minutes for any reason other than a natural disaster. Wit
by Animats 2y ago
In the entire history of the Bell System, no electromechanical exchange was ever down for more than 30 minutes for any reason other than a natural disaster. With one exception, a major fire in New York City. Three weeks of downtime for 170,000 phones for that.[1] The Bell System pulled in resources and people from all over the system to replace and rewire several floors of equipment and cabling.
That record has not been maintained in the digital era.
The long distance system did not originally need the Bedminster, NJ network control center to operate. Bedminster sent routing updates periodically to the regional centers, but they could fall back to static routing if necessary. There was, by design, no single point of failure. Not even close. That was a basic design criterion in telecom prior to electronic switching. The system was designed to have less capacity but still keep running if parts of it went down.
[1] https://www.youtube.com/watch?v=f_AWAmGi-g8 https://www.youtube.com/watch?v=f_AWAmGi-g8
- benjojo12 2y agoThat electro mechanical system also switched significantly less calls than the digital counterparts! Most modern day telcos that I have seen still have multiple power/line cards/uplinks in place and designed for redundancy. However the new systems can also just do so much more and are so more flexible that they can be configured out of existence just as easily! Some of this as well is just poor software, on some of the big carrier grade routers you can configure many things but the combination of things that you can figure may also just cause things to not work correctly, or even worse pull down the entire chassis, I don't have immediate experience on how good the early 2000s software was, but I would take a guess and say that configurability/flexibility has had a serious cost on reliability of the network
- atoav 2y agoAnd part of the reason why it is software is because people keep saying it is "just" software. Unreliability is unreliability even of it comes through software and we ahould treat broken software as broken, not as "just a software error".
- _kb 2y agoThe core value proposition of software is the ability to implement most system designs relatively quickly and efficiently, including very bad ones.
- cqqxo4zV46cp 2y agoNow there’s a tattoo.
- maxhodges 2y agoI think that's an oversimplification and not quite the core value proposition. Implementing a solution is just the beginning. Systems evolve, so software development is not only about delivering solutions but also about enabling their maintenance and evolution. I view programming languages and software dev tools as fundamentally about the management of complexity. The evolution of software development tools reflects a continuous effort to manage the inherent complexity of building and maintaining software systems. From the earliest programming languages and punch cards to simple text editors, and now to sophisticated IDEs, version control systems, and automated testing frameworks, each advancement enables better ways to manage and simplify the overall development process. These tools help developers handle dependencies, abstract away low-level details, and facilitate collaboration, ultimately enabling the creation of more robust, scalable, and maintainable software. Notable figures in software development emphasize this focus on complexity management. Fred Brooks, in "The Mythical Man-Month," highlights the inherent complexity in software development and the necessity of tools and methodologies to manage it effectively. Eric S. Raymond, in "The Cathedral and the Bazaar," discusses how collaborative tools and practices, especially in open-source projects, help manage complexity. Grady Booch's "Object-Oriented Analysis and Design" underscores how object-oriented principles and tools promote modularity and reuse, aiding complexity management. Martin Fowler, known for advocating the use of design patterns in his book "Patterns of Enterprise Application Architecture," emphasizes how patterns provide proven solutions to recurring problems, thereby managing complexity more effectively. By using patterns, developers can reuse solutions, improve communication through a common vocabulary, and enhance system flexibility. Fowler also advocates for continuous improvement and refactoring in his book "Refactoring" as essential practices for managing and reducing software complexity. Similarly, Robert C. Martin, in "Clean Code," stresses the importance of writing clean, readable, and maintainable code to manage complexity and ensure long-term software health. While speed and efficiency are important, the core value of software development lies in the ability to manage complexity, ensuring that systems are robust, scalable, and maintainable over time.
- tsimionescu 2y agoThe expectation should be that, as you switch more and more, so that the cost of a 30 minute pause gets higher and higher, the situation would improve, and a more modern system might have been expected to boast that it never had a break lasting more than, say, 30s outside of a natural disaster.
- kbenson 2y agoI don't know where you get that expectation from. These are arbitrary engineering constraints informed by business decisions. If they decided that people could deal with up to 30 minutes of service interruption and set that as a goal, they would engineer with that in mind, regardless of how many people. If they used total combined user hours of interrupted service, then they would engineer around reducing possibly outage times for a system as it handled more people (or scale differently, with more systems). I don't think there's any sort of expectation that it would definitely go one way though, as you say. It's all business and legal constraints providing engineering constraints to build against.
- varjag 2y agoNot sure about the calls, since apparently millennials and zoomers never pick up.
- deleted 2y ago[deleted]
- macintux 2y agoA relatively famous example of the extent to which Indiana Bell went to avoid disrupting telephone service: rotating and relocating its headquarters over a few weeks. https://en.wikipedia.org/wiki/AT%26T_Building_(Indianapolis) https://en.wikipedia.org/wiki/AT%26T_Building_(Indianapolis)
- bcrl 2y agoBell Canada had a major outage on July 17th 1999 when a tool was dropped on the bus bar for the main battery power that ignited the hydrogen from the batteries in one of the exchangeds in downtown Toronto. The fire department insisted that all power in the area be shut down which lead to the main switch that handled long distance call routing for all 1-800 numbers being offline for the better part of a day.
- gosub100 2y agothat was back before the phone system was defiled by robo-dialers and scammers.
- maxhodges 2y agoSurely, that downtime quote is apocryphal. The early decades were plagued by poor service due to maintenance, mechanical failure, and human error. In the early decades, manual switchboards were the primary method of connecting calls. The intense workload and physical demands on operators often led to service disruptions . In 1907, over 400 operators went on strike in Toronto, severely impacting phone service. The strike was driven by wage disputes, increased working hours, and poor working conditions They didn't have downtime logs, but that doesn't mean that the rapid growth of telephone demand didn't outpace the Bell System's capacity to provide adequate service. The company struggled to balance expansion with maintaining service quality, leading to intermittent service issues . Bell System faced significant public dissatisfaction due to poor service quality. This was compounded by internal issues such as poor employee morale and fierce competition.