4 ms·

I'm so conflicted reading this story. On one hand, yes, there were choices made during the design of the system that directly contributed to this tragedy. And a

I'm so conflicted reading this story. On one hand, yes, there were choices made during the design of the system that directly contributed to this tragedy. And a lot of innocent lives were lost, so saying that's "shit happens, it's an edge case" rings very hollow. On the other hand, this was a very peculiar set of circumstances, very much an edge case. Is it reasonable to expect designers of combat systems to triple check their choices and run more test scenarios to identify and address such edge cases? I'd say yes. However, I think it's unreasonable to expect them to design a perfect system for a highly volatile and chaotic use case such as war.

gpderetta 3y ago
Sorry, I don't see where's the edge case. In a given area there are going to be lots of planes. If there is risk of confusing them and making decisions based on non-reconciled information, it seems a pretty critical flaw.
- Maxion 3y ago
  Hard agree here. There are so many small things there that could be improved. One simple one is identifier re-use, if it is necessary for some reason, then at the very least it shouldn't happen within a specific time frame, so that you may have the same identifier used again as in the scenario.
- quickthrower2 3y ago
  I agree, the described scenario could just be another day at any airport and surrounding airspace (I guess any airport that is dual purpose military and civilian).
rkagerer 3y ago
Reusing identifiers after such a short time was a pretty galactic design cockup. I'm a consultant and if I came across that in a design doc or while analyzing a system to form an understanding in my head of how it works, it would have immediately screamed out to me as asking for this kind of trouble. Operator punched the ID in for (civilian) aircraft A, and unknowingly got the trajectory data for (military) aircraft B. Coding for the 90% common conditions are easy, it's the edge cases where things get hard and true engineering talent shines through. Ignoring them is simply incomplete design. It's not tolerated in other fields of engineering (eg. civil) and it shouldn't be in ours either.
helsinkiandrew 3y ago
Yes - but the implications of reassigning the number immediately to another contact seems something that should have been noticed in the design phase. > Vincennes assigned her the tracking number 4474; Sides assigned her 4131. Aegis unified the contacts under the number 4131. 4474 was then available for re-use, so Aegis assigned it to a US A-6 bomber, which happened to be descending. > But he didn't realize that its tracking number had changed. He thought it was still tracking number 4474,
- tlb 3y ago
  Global commercial flight traffic averages around 100k flights per day. I don't know what fraction is within the radar range of a big ship in a busy area, but maybe 10k? So it's not trivial to avoid reuse within a day while still having 4-digit numbers. Especially when contacts are assigned numbers independently by multiple ships and then reconciled.
  - helsinkiandrew 3y ago
    > I don't know what fraction is within the radar range of a big ship in a busy area maybe 10k? So it's not trivial to avoid reuse within a day while still having 4-digit numbers So in the design phase that should come up as an issue and you would surely use 5 digit numbers
ZephyrBlu 3y ago
This was not an edge case, it was a swiss cheese failure that was just waiting to happen. In a tech company this would correctly be thought of as a systemic failure as opposed to a personal one. There are so many questionable design choices here for a system that is supposed to be used in high-stress situations. A lot of it reads as someone thinking "ooh yeah it would be cool if it did X" instead of "what's the simplest and dumbest possible way to do this".
ughitsaaron 3y ago
Given the stakes of an “edge case” in a war machine, not to mention their cost, it doesn’t seem unreasonable to expect the number of such cases to be zero.
amadeuspagel 3y ago
There was no war.
- ben0x539 3y ago
  Wikipedia says "The attack occurred during the Iran–Iraq War, which had been continuing for nearly eight years." I guess it wasn't supposed to be a war that the US was involved in directly? But they were apparently getting their helicopter shot at and were doing things in Iranian territorial waters, so I guess they weren't just hanging out.
- eastern 3y ago
  Well, apart from the fact that the Iran-Iraq war had been on right there for eight years, there's all this on the referred Wikipedia page, including the fact that the Vincennes was actually in Iranian territorial waters at the time: > The Flight 655 incident occurred a year after the USS Stark incident, during which the Iraqi Air Force attacked the U.S. Navy guided missile frigate USS Stark on 17 May 1987, killing 37 American sailors. > U.S. naval forces had also exchanged gunfire with Iranian gunboats in late 1987, and the guided missile frigate USS Samuel B. Roberts had struck an Iranian sea mine in April 1988. > Two months before the incident, the U.S. had engaged in Operation Praying Mantis, resulting in the sinkings of the Iranian frigate Sahand, the Iranian fast attack craft Joshan, and three Iranian speedboats. > Also, the Iranian frigate Sabalan was crippled, two Iranian platforms were destroyed, and an Iranian fighter was damaged. A total of at least 56 Iranian crew were killed, while the U.S. suffered the loss of only one helicopter, which crashed apparently by accident, killing its two pilots. > On the morning of 3 July 1988, USS Vincennes was passing through the Strait of Hormuz returning from an escort duty. A helicopter deployed from the cruiser reportedly received small arms fire from Iranian patrol vessels as it observed from high altitude. Vincennes moved to engage the Iranian vessels, in the course of which they all violated Omani waters and left after being challenged and ordered to leave by a Royal Navy of Oman warship. > Vincennes then pursued the Iranian gunboats, entering Iranian territorial waters. So yeah, you are right, there was no actual war. But everyone was pretty war-ish
h0l0cube 3y ago
> However, I think it's unreasonable to expect them to design a perfect system for a highly volatile and chaotic use case such as war. When it comes to safety-critical systems, the right engineering choice is to lean towards a 'safe' default. For example, the safe default would be to always slave the cursor: > Once "hooked," the contact would be tracked by Aegis. But critically, unless the operator took the additional step of "slaving" the cursor to that contact, as the contact moved away the cursor would not follow it. And here, don't reassign a tracking number, at least not within in a short timeframe: > Vincennes assigned her the tracking number 4474; Sides assigned her 4131. Aegis unified the contacts under the number 4131. 4474 was then available for re-use, so Aegis assigned it to a US A-6 bomber, which happened to be descending.
- Ferret7446 3y ago
  > the safe default would be to always slave the cursor I don't think so, I imagine that behavior could be frustrating, e.g. if you're cursoring over many contacts. Admittedly I am not an expert either, but that suggestion smells like a classic case of armchair design that would actually cause more problems, because I imagine that the two modes exist for a reason and the designers intentionally chose which default to use, but they didn't anticipate this user error. Thus, I'd suggest that the UI should have made it extremely obvious whether the cursor was slaved and when a contact gets hooked/unhooked under the cursor. If I had to make an analogy, I'd compare it to normal and insert mode in Vi(m). The fact that the default is normal mode actually makes sense even though new users may suggest otherwise, but the real problem is that by default it's hard to tell which mode you are in.
  - h0l0cube 3y ago
    It might help to read the incident further. > The next aircraft taking off on that runway was an Iranian military F-14 fighter. The cursor was only left on the runway for around 90 seconds, but that was long enough for the Vincennes to get an IFF response corresponding to a military fighter. So Flight 655 was reclassified from an unknown contact to a potentially hostile one. The default was that the automated system conflated two completely distinct aircraft. The IFF ("identification friend or foe") for a military aircraft was attributed to a civilian airliner
ben0x539 3y ago
I think a highly volatile and chaotic use case is exactly where I'd expect them to design a perfect, or at least orders of magnitude less susceptible to operator error, system. Of course it's hard for me, a spoiled millennial who got into programming via online games, to imagine what war computers were capable of in 1988, but as described in the thread, this scenario sounds so utterly routine that I am surprised that it basically involved a game of telephone to confirm basic facts about a plane. "A tracked entity gets confused with another tracked entity" or "an entity's status of hostile-or-not gets lost" sounds like exactly the cases that should be impossible to get wrong as a fundamental goal of this kind of operation.
CogitoCogito 3y ago
> On the other hand, this was a very peculiar set of circumstances, very much an edge case. Is it reasonable to expect designers of combat systems to triple check their choices and run more test scenarios to identify and address such edge cases? I'd say yes. However, I think it's unreasonable to expect them to design a perfect system for a highly volatile and chaotic use case such as war. Even if this is your position, it doesn't excuse the Navy's blaming of the crew after it happens. Even if the design issues could be written off as a reasonable mistake, the mistake still lies with the design and not with the crew.
ninkendo 3y ago
It really peeves me to hear the phrase “edge case” used as a defense of incorrect software. As if software should not be expected to deal with edge cases. Edge cases are not rare. If you have a lot of people using your system, or people who use it a long time, hitting an edge case increases in likelihood to the point that it becomes inevitable. It’s a fallacy to think that an edge case being mathematically unlikely implies that it is unlikely to ever happen. See also murphy’s law.

gpderetta 3y ago

Sorry, I don't see where's the edge case. In a given area there are going to be lots of planes. If there is risk of confusing them and making decisions based on non-reconciled information, it seems a pretty critical flaw.

Maxion 3y ago
Hard agree here. There are so many small things there that could be improved. One simple one is identifier re-use, if it is necessary for some reason, then at the very least it shouldn't happen within a specific time frame, so that you may have the same identifier used again as in the scenario.
quickthrower2 3y ago
I agree, the described scenario could just be another day at any airport and surrounding airspace (I guess any airport that is dual purpose military and civilian).

rkagerer 3y ago

Reusing identifiers after such a short time was a pretty galactic design cockup. I'm a consultant and if I came across that in a design doc or while analyzing a system to form an understanding in my head of how it works, it would have immediately screamed out to me as asking for this kind of trouble. Operator punched the ID in for (civilian) aircraft A, and unknowingly got the trajectory data for (military) aircraft B. Coding for the 90% common conditions are easy, it's the edge cases where things get hard and true engineering talent shines through. Ignoring them is simply incomplete design. It's not tolerated in other fields of engineering (eg. civil) and it shouldn't be in ours either.

helsinkiandrew 3y ago

Yes - but the implications of reassigning the number immediately to another contact seems something that should have been noticed in the design phase. > Vincennes assigned her the tracking number 4474; Sides assigned her 4131. Aegis unified the contacts under the number 4131. 4474 was then available for re-use, so Aegis assigned it to a US A-6 bomber, which happened to be descending. > But he didn't realize that its tracking number had changed. He thought it was still tracking number 4474,

tlb 3y ago
Global commercial flight traffic averages around 100k flights per day. I don't know what fraction is within the radar range of a big ship in a busy area, but maybe 10k? So it's not trivial to avoid reuse within a day while still having 4-digit numbers. Especially when contacts are assigned numbers independently by multiple ships and then reconciled.
- helsinkiandrew 3y ago
  > I don't know what fraction is within the radar range of a big ship in a busy area maybe 10k? So it's not trivial to avoid reuse within a day while still having 4-digit numbers So in the design phase that should come up as an issue and you would surely use 5 digit numbers

ZephyrBlu 3y ago

This was not an edge case, it was a swiss cheese failure that was just waiting to happen. In a tech company this would correctly be thought of as a systemic failure as opposed to a personal one. There are so many questionable design choices here for a system that is supposed to be used in high-stress situations. A lot of it reads as someone thinking "ooh yeah it would be cool if it did X" instead of "what's the simplest and dumbest possible way to do this".

ughitsaaron 3y ago

Given the stakes of an “edge case” in a war machine, not to mention their cost, it doesn’t seem unreasonable to expect the number of such cases to be zero.

amadeuspagel 3y ago

There was no war.

ben0x539 3y ago
Wikipedia says "The attack occurred during the Iran–Iraq War, which had been continuing for nearly eight years." I guess it wasn't supposed to be a war that the US was involved in directly? But they were apparently getting their helicopter shot at and were doing things in Iranian territorial waters, so I guess they weren't just hanging out.
eastern 3y ago
Well, apart from the fact that the Iran-Iraq war had been on right there for eight years, there's all this on the referred Wikipedia page, including the fact that the Vincennes was actually in Iranian territorial waters at the time: > The Flight 655 incident occurred a year after the USS Stark incident, during which the Iraqi Air Force attacked the U.S. Navy guided missile frigate USS Stark on 17 May 1987, killing 37 American sailors. > U.S. naval forces had also exchanged gunfire with Iranian gunboats in late 1987, and the guided missile frigate USS Samuel B. Roberts had struck an Iranian sea mine in April 1988. > Two months before the incident, the U.S. had engaged in Operation Praying Mantis, resulting in the sinkings of the Iranian frigate Sahand, the Iranian fast attack craft Joshan, and three Iranian speedboats. > Also, the Iranian frigate Sabalan was crippled, two Iranian platforms were destroyed, and an Iranian fighter was damaged. A total of at least 56 Iranian crew were killed, while the U.S. suffered the loss of only one helicopter, which crashed apparently by accident, killing its two pilots. > On the morning of 3 July 1988, USS Vincennes was passing through the Strait of Hormuz returning from an escort duty. A helicopter deployed from the cruiser reportedly received small arms fire from Iranian patrol vessels as it observed from high altitude. Vincennes moved to engage the Iranian vessels, in the course of which they all violated Omani waters and left after being challenged and ordered to leave by a Royal Navy of Oman warship. > Vincennes then pursued the Iranian gunboats, entering Iranian territorial waters. So yeah, you are right, there was no actual war. But everyone was pretty war-ish

h0l0cube 3y ago

> However, I think it's unreasonable to expect them to design a perfect system for a highly volatile and chaotic use case such as war. When it comes to safety-critical systems, the right engineering choice is to lean towards a 'safe' default. For example, the safe default would be to always slave the cursor: > Once "hooked," the contact would be tracked by Aegis. But critically, unless the operator took the additional step of "slaving" the cursor to that contact, as the contact moved away the cursor would not follow it. And here, don't reassign a tracking number, at least not within in a short timeframe: > Vincennes assigned her the tracking number 4474; Sides assigned her 4131. Aegis unified the contacts under the number 4131. 4474 was then available for re-use, so Aegis assigned it to a US A-6 bomber, which happened to be descending.

Ferret7446 3y ago
> the safe default would be to always slave the cursor I don't think so, I imagine that behavior could be frustrating, e.g. if you're cursoring over many contacts. Admittedly I am not an expert either, but that suggestion smells like a classic case of armchair design that would actually cause more problems, because I imagine that the two modes exist for a reason and the designers intentionally chose which default to use, but they didn't anticipate this user error. Thus, I'd suggest that the UI should have made it extremely obvious whether the cursor was slaved and when a contact gets hooked/unhooked under the cursor. If I had to make an analogy, I'd compare it to normal and insert mode in Vi(m). The fact that the default is normal mode actually makes sense even though new users may suggest otherwise, but the real problem is that by default it's hard to tell which mode you are in.
- h0l0cube 3y ago
  It might help to read the incident further. > The next aircraft taking off on that runway was an Iranian military F-14 fighter. The cursor was only left on the runway for around 90 seconds, but that was long enough for the Vincennes to get an IFF response corresponding to a military fighter. So Flight 655 was reclassified from an unknown contact to a potentially hostile one. The default was that the automated system conflated two completely distinct aircraft. The IFF ("identification friend or foe") for a military aircraft was attributed to a civilian airliner

ben0x539 3y ago

I think a highly volatile and chaotic use case is exactly where I'd expect them to design a perfect, or at least orders of magnitude less susceptible to operator error, system. Of course it's hard for me, a spoiled millennial who got into programming via online games, to imagine what war computers were capable of in 1988, but as described in the thread, this scenario sounds so utterly routine that I am surprised that it basically involved a game of telephone to confirm basic facts about a plane. "A tracked entity gets confused with another tracked entity" or "an entity's status of hostile-or-not gets lost" sounds like exactly the cases that should be impossible to get wrong as a fundamental goal of this kind of operation.

CogitoCogito 3y ago

> On the other hand, this was a very peculiar set of circumstances, very much an edge case. Is it reasonable to expect designers of combat systems to triple check their choices and run more test scenarios to identify and address such edge cases? I'd say yes. However, I think it's unreasonable to expect them to design a perfect system for a highly volatile and chaotic use case such as war. Even if this is your position, it doesn't excuse the Navy's blaming of the crew after it happens. Even if the design issues could be written off as a reasonable mistake, the mistake still lies with the design and not with the crew.

ninkendo 3y ago

It really peeves me to hear the phrase “edge case” used as a defense of incorrect software. As if software should not be expected to deal with edge cases. Edge cases are not rare. If you have a lot of people using your system, or people who use it a long time, hitting an edge case increases in likelihood to the point that it becomes inevitable. It’s a fallacy to think that an edge case being mathematically unlikely implies that it is unlikely to ever happen. See also murphy’s law.