5 ms·

> As the primary net effect of software is to facilitate bureaucratic complexity it is therefor essential that software projects fail if society is to function

> As the primary net effect of software is to facilitate bureaucratic complexity it is therefor essential that software projects fail if society is to function effectively. Not only failed projects can prevent bureaucratic complexity, they can also save lives > automated system assigns a unique identifying code to each incoming call before passing it on — a method of keeping track of phone calls as they move through the system. But on April 9, the software responsible for assigning the codes maxed out at a pre-set limit; the counter literally stopped counting at 40 million calls. As a result, the routing system stopped accepting new calls, leading to a bottleneck and a series of cascading failures elsewhere in the 911 infrastructure. Had that automated system failed at the development phase... https://www.washingtonpost.com/news/the-switch/wp/2014/10/20/how-a-dumb-software-glitch-kept-6600-calls-from-getting-to-911/ https://www.washingtonpost.com/news/the-switch/wp/2014/10/20...

amelius 6y ago
You can blame the software, but you can also blame the organization for not having a backup system.
- perfunctory 6y ago
  > Any backup 911 call centers that could have helped with the bottleneck never did, because they themselves were suffering from the same problems everyone else was, according to the FCC. Go figure.
  - ByteJockey 6y ago
    They mean a backup way to take phone calls, not backup 911 centers.
segfaultbuserr 6y ago
Wow, it sounds like an apocalyptic story that comp.risks has warned in the 1990s. (From those who don't know: comp.risks was a well-known Usenet forum on the systematic risks of computing technologies in society, and they publish a newsletter since the mid-80s.)
an_opabinia 6y ago
I don't know, the contents of this article sound really inaccurate. There's no software counter that idiosyncratically stops at 40 million - maybe 4 billion was the real answer, and the reporter misread the digits. This sounds like a 32-bit TXID exhaustion bug in a Postgres database, which is a notorious and very difficult to recover from problem in live systems. It is probably the #1 arcane Postgres cataclysm. It would be consistent with how this is a "single coding error" (i.e., the call to VACUUM), additionally why it was so difficult to recover from ad-hoc. I'm generously assuming that everything else went well, that people who are building 911 systems aspire to do a really good job, and that it didn't fail anyone how important their work was. Which is funny, I mean what if Postgres "failed" for every TXID exhaustion cataclysm it had. Arguably the best database product in the world "failed" because of this. It probably provides more than a few billion dollars to our tech economy, on top of being the hub for research in databases - research whose goal is the same as the one as this paper, to use automation to reduce bureaucracy and improve society. Anyway we always imagine these rinky-dink consultant shops as writing all this custom stuff, and in actuality they are almost always combining a bunch open source and off the shelf things - which I would like to not see fail.
JJMcJ 6y ago
Numeric limits like this are far more common than you might think. Besides the well known Y2K mitigation, there were some problems on day 10,000 when four characters to represent days since the epoch overflowed, mid 90s for Unix derived systems and I believe VMS as well. None were civilization threatening. We also had Second One Billion in early 2000s. Also no public catastrophes. Next big one is 2038, when 32-bit Unix timekeeping overflows.
- tonyarkles 6y ago
  > We also had Second One Billion in early 2000s. Also no public catastrophes. Thanks for the reminder! We had a small fun party to celebrate!

amelius 6y ago

You can blame the software, but you can also blame the organization for not having a backup system.

perfunctory 6y ago
> Any backup 911 call centers that could have helped with the bottleneck never did, because they themselves were suffering from the same problems everyone else was, according to the FCC. Go figure.
- ByteJockey 6y ago
  They mean a backup way to take phone calls, not backup 911 centers.

segfaultbuserr 6y ago

Wow, it sounds like an apocalyptic story that comp.risks has warned in the 1990s. (From those who don't know: comp.risks was a well-known Usenet forum on the systematic risks of computing technologies in society, and they publish a newsletter since the mid-80s.)

an_opabinia 6y ago

I don't know, the contents of this article sound really inaccurate. There's no software counter that idiosyncratically stops at 40 million - maybe 4 billion was the real answer, and the reporter misread the digits. This sounds like a 32-bit TXID exhaustion bug in a Postgres database, which is a notorious and very difficult to recover from problem in live systems. It is probably the #1 arcane Postgres cataclysm. It would be consistent with how this is a "single coding error" (i.e., the call to VACUUM), additionally why it was so difficult to recover from ad-hoc. I'm generously assuming that everything else went well, that people who are building 911 systems aspire to do a really good job, and that it didn't fail anyone how important their work was. Which is funny, I mean what if Postgres "failed" for every TXID exhaustion cataclysm it had. Arguably the best database product in the world "failed" because of this. It probably provides more than a few billion dollars to our tech economy, on top of being the hub for research in databases - research whose goal is the same as the one as this paper, to use automation to reduce bureaucracy and improve society. Anyway we always imagine these rinky-dink consultant shops as writing all this custom stuff, and in actuality they are almost always combining a bunch open source and off the shelf things - which I would like to not see fail.

JJMcJ 6y ago

Numeric limits like this are far more common than you might think. Besides the well known Y2K mitigation, there were some problems on day 10,000 when four characters to represent days since the epoch overflowed, mid 90s for Unix derived systems and I believe VMS as well. None were civilization threatening. We also had Second One Billion in early 2000s. Also no public catastrophes. Next big one is 2038, when 32-bit Unix timekeeping overflows.

tonyarkles 6y ago
> We also had Second One Billion in early 2000s. Also no public catastrophes. Thanks for the reminder! We had a small fun party to celebrate!