4 ms·

They’re now filtering bad packets so this can’t happen again. No mention of fixing the design flaw in the system that allows a single piece of malfunctioning h

They’re now filtering bad packets so this can’t happen again. No mention of fixing the design flaw in the system that allows a single piece of malfunctioning hardware to knock out 911 service for millions of users for two days.

JustSomeNobody 8y ago
> ... can’t happen again Gotta love this. > No mention of fixing the design flaw in the system that allows a single piece of malfunctioning hardware to knock out 911 service for millions of users for two days. Yep, they put a bandage on it and called it good.
SlowRobotAhead 8y ago
>a single piece of malfunctioning hardware Just wait until writes Bloomberg that this was because China had put a secret microchip on a network card. Months of entertainment to follow.
jka 8y ago
Worth giving some credit though - mitigation is important for something as critical as 911 service. Hopefully they're also tracking the design flaws, and yes that's worth following (and asking whether they're planning to do so?), but bear in mind people have limited time and resources, so don't be too hard on them (or they'll be less willing to help and investigate in future).
- eeeeeeeeeeeee 8y ago
  I completely disagree. Mitigation with a patch applied in a few hours is acceptable, but mitigation DAYS after a multi-day outage of critical services that have life or death consequences is completely unacceptable.
- mikeash 8y ago
  I completely agree. I wouldn’t expect them to have this fix already, since it probably requires some major work. But some mention that they’ll be looking into it would certainly be nice.
- vmchale 8y ago
  > people have limited time and resources, so don't be too hard on them (or they'll be less willing to help and investigate in future). yes, clearly knocking out 911 service for millions of people isn't a problem. won't someone think of the poor programmers??
patrickg_zill 8y ago
I don't have any inside info. But it sounds like either a control plane or a mesh/fabric board that was messing with all the other such boards in their system.

JustSomeNobody 8y ago

> ... can’t happen again Gotta love this. > No mention of fixing the design flaw in the system that allows a single piece of malfunctioning hardware to knock out 911 service for millions of users for two days. Yep, they put a bandage on it and called it good.

SlowRobotAhead 8y ago

>a single piece of malfunctioning hardware Just wait until writes Bloomberg that this was because China had put a secret microchip on a network card. Months of entertainment to follow.

jka 8y ago

Worth giving some credit though - mitigation is important for something as critical as 911 service. Hopefully they're also tracking the design flaws, and yes that's worth following (and asking whether they're planning to do so?), but bear in mind people have limited time and resources, so don't be too hard on them (or they'll be less willing to help and investigate in future).

eeeeeeeeeeeee 8y ago
I completely disagree. Mitigation with a patch applied in a few hours is acceptable, but mitigation DAYS after a multi-day outage of critical services that have life or death consequences is completely unacceptable.
mikeash 8y ago
I completely agree. I wouldn’t expect them to have this fix already, since it probably requires some major work. But some mention that they’ll be looking into it would certainly be nice.
vmchale 8y ago
> people have limited time and resources, so don't be too hard on them (or they'll be less willing to help and investigate in future). yes, clearly knocking out 911 service for millions of people isn't a problem. won't someone think of the poor programmers??

patrickg_zill 8y ago

I don't have any inside info. But it sounds like either a control plane or a mesh/fabric board that was messing with all the other such boards in their system.