7 ms·

The Patriot Missile Failure (2000)

67 points by soheilpro 5y ago

themodelplumber 5y ago
Wow, if you look at the PDF there were issues all over the place, not just in the binary representation. Patriot battery operators could have easily worked around the computer math issues in particular, but various leadership-system-level mistakes were made, and then procedural details added up on top of that. Edit: To be clear, this was not just bad code, as that issue was already measured and known, but not communicated well, for one. It was military leadership systems-level issue in addition to a Raytheon/coder/etc. issue. Software was corrected on February 16. > On February 21, 1991, the Patriot Project Office sent a message to Patriot users stating that very long run times could cause a shift in the range gate, resulting in the target being offset. The message also said a software change was being sent that would improve the system’s targeting. However, the message did not specify what constitutes very long run times. So Israel was running their units for 8h, but the US unit kept theirs running more than 10x longer with no idea what was appropriate. Phew. Then, on Scud-day, February 25, there were _six_ batteries operating around the vicinity, and it so happens that the 100h-uptime Patriot battery was tasked to this missile. (How many scuds flew that day? I'd like to know) And finally on February 26, the next day, "the modified software, which compensated for the inaccurate time calculation, arrived in Dhahran." Ouch. :-( The PDF report is a really nice effort. Things like the table on page 17/print page 15 made it easier to understand the range error situation compared to Patriot battery run time.
- bostonsre 5y ago
  Seems almost criminally negligent.
  - 2OEH8eoCRo0 5y ago
    It's interesting that Raytheon's name is nowhere to be found on this. It sounds like Raytheon's bad code killed 28 US soldiers.
    - mlyle 5y ago
      The Iraqi SCUD killed 28 US soldiers. Raytheon failed to prevent it.
      - 2OEH8eoCRo0 5y ago
        Due to a code defect in their product*
        kevin_thibedeau 5y ago
        That was being used outside of its original design specifications.
        lumost 5y ago
        Missile interceptors are incredibly complicated. Even with perfect code there is a probability range for successful intercept. The probability was much lower in this case as the software had a defect, and the battery was online for 100h.
        codechad 5y ago
      - bostonsre 5y ago
        Yes, the Iraqi scud was the primary factor, but it almost seems like if your seat belt snaps in a car accident with a drunk driver and you go through the windshield and die. The drunk driver is culpable but the car company doesn't get a free pass.
- doodlebugging 5y ago
  I once worked with a person who had an advanced degree in nuclear physics. They were sharp as a tack, a skilled programmer in multiple languages and on several platforms, and grossly over-qualified for the job they had taken but Congress had just killed their position with the SSC in Texas so they needed to switch gears for a while. Though they were super smart and a rapid typist they were also dyslexic. To make it worse, they would never take responsibility for any mistakes resulting from their failure to double-check the figures entered into the software they needed to run. As a result, we were constantly forced to rerun processes they had worked on because the person never reviewed the parameters they entered into the processing modules before running the jobs. Very simple mistakes like entering 1005 instead of 1500 would cost us a day or more rerunning jobs. It took me weeks to detect the pattern of failures and link them conclusively to this individual since other jobs by other people I supervised also failed but for different reasons. Once I saw the problem and could easily document the issue I took a few moments to sit with them privately to show what I had found and made the recommendation that they should double-check everything they typed before submitting the jobs so that we could eliminate that class of failures. This person was adamant that I had to be wrong and that they were a perfect typist and somehow someone else was responsible. Try as I might I could not get them to adopt a simple review-before-submit plan so, since they were a valuable team member who didn't need to lose their job during a period where good jobs were not easy to find, after consulting with my superiors it became my job to review everything they submitted, correct any input parameter fat-fingers or dyslexic inputs, and resubmit those jobs so that we could eliminate those failures. It was easy enough to do so we made it work and they were never informed I was bird-dogging their work so closely. They remained a valued member of the team until one day they put in their notice. They had been applying for any positions anywhere that would allow them to use their programming skills on problems that a physicist might find challenging. They had been offered and had accepted a position as a programmer with Raytheon working on missile guidance systems. After their exit, we joked that one day we would hear about a fat-finger mistake sending a missile to an unintended location. This person was not connected at all to this event since their work with Raytheon was several years after this event occurred (SSC shuttered in 1993). I have no idea what they worked on while there but I hope that someone noticed the issue and insured that two sets of eyes hit everything they typed. Those were the days.
deleted 5y ago
[deleted]
deleted 5y ago
[deleted]
Mountain_Skies 5y ago
The Patriot missile in the first gulf war was mostly a failure when it came to its stated purpose of intercepting and rendering harmless incoming missiles. It was however a huge propaganda success. Each time it deflected a SCUD, people felt safer, even though often the explosive payload survived and still caused damage on the ground. Given the almost non-existent guidance system on the SCUD, their goal was simply to hit enemy territory and hope to get lucky. The Patriot didn't change that. But it did keep Israel from entering the war and kept the American public believing that the SCUDs were being derogated. Instead of being horrified at the helplessness against SCUDs, people were cheering for the Patriot missile.
- ianmf 5y ago
  IIRC from my military time, the first generations of the patriot missiles (PAC-1, PAC-2) were designed to explode when in vicinity of the target. The PAC-3 missile was the first to introduce kinetic damage before the payload would explode.
  - jandrewrogers 5y ago
    This is correct. The Patriot was an anti-aircraft missile that was retrofitted to kinda sorta be adequate for ballistic missile intercept. It was a proverbial dancing bear -- it was impressive that it worked at all, never mind that it wasn't very good.
- qbasic_forever 5y ago
  I remember the US gained air superiority and control of the skies pretty quickly in the first gulf war and that was pretty much game over for the SCUD or other threats. They were forced to keep them hidden and moving or else a squad of Apache helicopters would make short work of the batteries.
  - goatsi 5y ago
    That was the propaganda published at the time. Postwar analysis showed that the air campaign was totally ineffective at hunting down scud launchers. >Second, there is no technical evidence that a single TEL was actually destroyed during the war, despite the claims of some 100 “kills” by aircrews and special forces. [...] The exact impact of coalition operations against mobile systems is more problematic. Iraqi launch operations never stopped and only diminished somewhat over time, although during the last week of the war launch operations increased in tempo. At best, it can be said that counter-Scud efforts only maintained “pressure” on Iraqi missile operations and that Scud CAP operations apparently were successful at harassing but never halting Iraqi launch operations. https://media.defense.gov/2019/Apr/11/2002115481/-1/-1/0/15CROSSBOW.PDF https://media.defense.gov/2019/Apr/11/2002115481/-1/-1/0/15C...
penisverse 5y ago
I always wonder how much damage a bug can do in a war. Maybe nothing tops an hypothetical catastrophic bug in the Minuteman III ICBM during a first strike/MAD scenario.
- 13of40 5y ago
  The interesting twist here is that the programmed target is calculated to be the place where the most damage can be inflicted. A malfunction involving an actual nuclear explosion is likely to happen over somewhere like the polar ice cap.
- andbberger 5y ago
  oh a bug in the ICBM warning system would be far worse. if minuteman's are flying we've already lost
  - penisverse 5y ago
    And that already happened! https://en.wikipedia.org/wiki/1983_Soviet_nuclear_false_alarm_incident https://en.wikipedia.org/wiki/1983_Soviet_nuclear_false_alar...
- elihu 5y ago
  A bug in one's cryptography could be really bad, if there's someone like Alan Turing on the other side capable of noticing and exploiting it. Also any automated threat detection system that looks at satellite photos / night vision camera feeds / radar / sonar and misinterprets enemy military equipment for something innocuous or vice versa could be really bad. Especially if weapons are used on the targets on the basis of the system's recommendation.
  - gambiting 5y ago
    I mean, the world almost came to an end in 1983 when the Soviet early detection system detected with 100% confidence multiple missile launches from US heading towards Russia. The only reason we are still alive is because one man, officer in the Soviet Air Defence forces, took a personal decision to wait for radar confirmation before informing his superiors(who would have almost certainly ordered a counterstrike before the missiles could arrive). The most terrifying point in all of this is that multiple systems were all coming up with confirmation that the attack is really happening, he had his entire department checking and checking and checking everything over, and it was coming back as 100% positive. I don't think Soviets ever publicly said what the issue actually was, we can only guess. https://en.wikipedia.org/wiki/1983_Soviet_nuclear_false_alarm_incident https://en.wikipedia.org/wiki/1983_Soviet_nuclear_false_alar...
nomilk 5y ago
There's an incredibly interesting video from inside a US command centre of an inbound scud being detected, announced, tracked, and eventually destroyed (presumably by patriot). I tried searching youtube for it, but couldn't spot it. Side-note: search on youtube is a mess. 'For you' and 'Previously watched' collectively took up 16 of the 28 videos displayed on the first page of results, not including '+3 MORE' and '+6 MORE' messages.
- ncmncm 5y ago
  Anyway the Patriot hit the fuel tank after the warhead had separated, turning the tank from one big projectile into lots of smaller projectiles spreading out to hit lots of targets, and leaving the warhead untouched.
hurl 5y ago
Click-bait title, good article. Not surprised there was a computational error in 1991. Glad there was a failure so more R&D could be done to improve.
gspr 5y ago
> Velocity is a real number that can […] Strange definition of velocity!
sllabres 5y ago
If you like such 'stories' i can recommend "Computer-Related Risks by Peter G. Neumann". A bit older (1994), but still an interesting read.
- avnigo 5y ago
  Thanks for the recommendation. A lighter read that includes this kind of errors is Humble Pi: A Comedy of Maths Errors by Matt Parker.

67 points by soheilpro 5y ago

Wow, if you look at the PDF there were issues all over the place, not just in the binary representation. Patriot battery operators could have easily worked around the computer math issues in particular, but various leadership-system-level mistakes were made, and then procedural details added up on top of that. Edit: To be clear, this was not just bad code, as that issue was already measured and known, but not communicated well, for one. It was military leadership systems-level issue in addition to a Raytheon/coder/etc. issue. Software was corrected on February 16. > On February 21, 1991, the Patriot Project Office sent a message to Patriot users stating that very long run times could cause a shift in the range gate, resulting in the target being offset. The message also said a software change was being sent that would improve the system’s targeting. However, the message did not specify what constitutes very long run times. So Israel was running their units for 8h, but the US unit kept theirs running more than 10x longer with no idea what was appropriate. Phew. Then, on Scud-day, February 25, there were _six_ batteries operating around the vicinity, and it so happens that the 100h-uptime Patriot battery was tasked to this missile. (How many scuds flew that day? I'd like to know) And finally on February 26, the next day, "the modified software, which compensated for the inaccurate time calculation, arrived in Dhahran." Ouch. :-( The PDF report is a really nice effort. Things like the table on page 17/print page 15 made it easier to understand the range error situation compared to Patriot battery run time.

bostonsre 5y ago
Seems almost criminally negligent.
- 2OEH8eoCRo0 5y ago
  It's interesting that Raytheon's name is nowhere to be found on this. It sounds like Raytheon's bad code killed 28 US soldiers.
  - mlyle 5y ago
    The Iraqi SCUD killed 28 US soldiers. Raytheon failed to prevent it.
    - 2OEH8eoCRo0 5y ago
      Due to a code defect in their product*
      - kevin_thibedeau 5y ago
        That was being used outside of its original design specifications.
      - lumost 5y ago
        Missile interceptors are incredibly complicated. Even with perfect code there is a probability range for successful intercept. The probability was much lower in this case as the software had a defect, and the battery was online for 100h.
      - codechad 5y ago
    - bostonsre 5y ago
      Yes, the Iraqi scud was the primary factor, but it almost seems like if your seat belt snaps in a car accident with a drunk driver and you go through the windshield and die. The drunk driver is culpable but the car company doesn't get a free pass.
doodlebugging 5y ago
I once worked with a person who had an advanced degree in nuclear physics. They were sharp as a tack, a skilled programmer in multiple languages and on several platforms, and grossly over-qualified for the job they had taken but Congress had just killed their position with the SSC in Texas so they needed to switch gears for a while. Though they were super smart and a rapid typist they were also dyslexic. To make it worse, they would never take responsibility for any mistakes resulting from their failure to double-check the figures entered into the software they needed to run. As a result, we were constantly forced to rerun processes they had worked on because the person never reviewed the parameters they entered into the processing modules before running the jobs. Very simple mistakes like entering 1005 instead of 1500 would cost us a day or more rerunning jobs. It took me weeks to detect the pattern of failures and link them conclusively to this individual since other jobs by other people I supervised also failed but for different reasons. Once I saw the problem and could easily document the issue I took a few moments to sit with them privately to show what I had found and made the recommendation that they should double-check everything they typed before submitting the jobs so that we could eliminate that class of failures. This person was adamant that I had to be wrong and that they were a perfect typist and somehow someone else was responsible. Try as I might I could not get them to adopt a simple review-before-submit plan so, since they were a valuable team member who didn't need to lose their job during a period where good jobs were not easy to find, after consulting with my superiors it became my job to review everything they submitted, correct any input parameter fat-fingers or dyslexic inputs, and resubmit those jobs so that we could eliminate those failures. It was easy enough to do so we made it work and they were never informed I was bird-dogging their work so closely. They remained a valued member of the team until one day they put in their notice. They had been applying for any positions anywhere that would allow them to use their programming skills on problems that a physicist might find challenging. They had been offered and had accepted a position as a programmer with Raytheon working on missile guidance systems. After their exit, we joked that one day we would hear about a fat-finger mistake sending a missile to an unintended location. This person was not connected at all to this event since their work with Raytheon was several years after this event occurred (SSC shuttered in 1993). I have no idea what they worked on while there but I hope that someone noticed the issue and insured that two sets of eyes hit everything they typed. Those were the days.

deleted 5y ago

[deleted]

deleted 5y ago

[deleted]

Mountain_Skies 5y ago

The Patriot missile in the first gulf war was mostly a failure when it came to its stated purpose of intercepting and rendering harmless incoming missiles. It was however a huge propaganda success. Each time it deflected a SCUD, people felt safer, even though often the explosive payload survived and still caused damage on the ground. Given the almost non-existent guidance system on the SCUD, their goal was simply to hit enemy territory and hope to get lucky. The Patriot didn't change that. But it did keep Israel from entering the war and kept the American public believing that the SCUDs were being derogated. Instead of being horrified at the helplessness against SCUDs, people were cheering for the Patriot missile.

ianmf 5y ago
IIRC from my military time, the first generations of the patriot missiles (PAC-1, PAC-2) were designed to explode when in vicinity of the target. The PAC-3 missile was the first to introduce kinetic damage before the payload would explode.
- jandrewrogers 5y ago
  This is correct. The Patriot was an anti-aircraft missile that was retrofitted to kinda sorta be adequate for ballistic missile intercept. It was a proverbial dancing bear -- it was impressive that it worked at all, never mind that it wasn't very good.
qbasic_forever 5y ago
I remember the US gained air superiority and control of the skies pretty quickly in the first gulf war and that was pretty much game over for the SCUD or other threats. They were forced to keep them hidden and moving or else a squad of Apache helicopters would make short work of the batteries.
- goatsi 5y ago
  That was the propaganda published at the time. Postwar analysis showed that the air campaign was totally ineffective at hunting down scud launchers. >Second, there is no technical evidence that a single TEL was actually destroyed during the war, despite the claims of some 100 “kills” by aircrews and special forces. [...] The exact impact of coalition operations against mobile systems is more problematic. Iraqi launch operations never stopped and only diminished somewhat over time, although during the last week of the war launch operations increased in tempo. At best, it can be said that counter-Scud efforts only maintained “pressure” on Iraqi missile operations and that Scud CAP operations apparently were successful at harassing but never halting Iraqi launch operations. https://media.defense.gov/2019/Apr/11/2002115481/-1/-1/0/15CROSSBOW.PDF https://media.defense.gov/2019/Apr/11/2002115481/-1/-1/0/15C...

penisverse 5y ago

I always wonder how much damage a bug can do in a war. Maybe nothing tops an hypothetical catastrophic bug in the Minuteman III ICBM during a first strike/MAD scenario.

13of40 5y ago
The interesting twist here is that the programmed target is calculated to be the place where the most damage can be inflicted. A malfunction involving an actual nuclear explosion is likely to happen over somewhere like the polar ice cap.
andbberger 5y ago
oh a bug in the ICBM warning system would be far worse. if minuteman's are flying we've already lost
- penisverse 5y ago
  And that already happened! https://en.wikipedia.org/wiki/1983_Soviet_nuclear_false_alarm_incident https://en.wikipedia.org/wiki/1983_Soviet_nuclear_false_alar...
elihu 5y ago
A bug in one's cryptography could be really bad, if there's someone like Alan Turing on the other side capable of noticing and exploiting it. Also any automated threat detection system that looks at satellite photos / night vision camera feeds / radar / sonar and misinterprets enemy military equipment for something innocuous or vice versa could be really bad. Especially if weapons are used on the targets on the basis of the system's recommendation.
- gambiting 5y ago
  I mean, the world almost came to an end in 1983 when the Soviet early detection system detected with 100% confidence multiple missile launches from US heading towards Russia. The only reason we are still alive is because one man, officer in the Soviet Air Defence forces, took a personal decision to wait for radar confirmation before informing his superiors(who would have almost certainly ordered a counterstrike before the missiles could arrive). The most terrifying point in all of this is that multiple systems were all coming up with confirmation that the attack is really happening, he had his entire department checking and checking and checking everything over, and it was coming back as 100% positive. I don't think Soviets ever publicly said what the issue actually was, we can only guess. https://en.wikipedia.org/wiki/1983_Soviet_nuclear_false_alarm_incident https://en.wikipedia.org/wiki/1983_Soviet_nuclear_false_alar...

nomilk 5y ago

There's an incredibly interesting video from inside a US command centre of an inbound scud being detected, announced, tracked, and eventually destroyed (presumably by patriot). I tried searching youtube for it, but couldn't spot it. Side-note: search on youtube is a mess. 'For you' and 'Previously watched' collectively took up 16 of the 28 videos displayed on the first page of results, not including '+3 MORE' and '+6 MORE' messages.

ncmncm 5y ago
Anyway the Patriot hit the fuel tank after the warhead had separated, turning the tank from one big projectile into lots of smaller projectiles spreading out to hit lots of targets, and leaving the warhead untouched.

hurl 5y ago

Click-bait title, good article. Not surprised there was a computational error in 1991. Glad there was a failure so more R&D could be done to improve.

gspr 5y ago

> Velocity is a real number that can […] Strange definition of velocity!

sllabres 5y ago

If you like such 'stories' i can recommend "Computer-Related Risks by Peter G. Neumann". A bit older (1994), but still an interesting read.

avnigo 5y ago
Thanks for the recommendation. A lighter read that includes this kind of errors is Humble Pi: A Comedy of Maths Errors by Matt Parker.