8 ms·

'Dying All Tensed-Up': 30 Years Since the Troubled Secret Mission of STS-27

145 points by pinewurst 8y ago

woliveirajr 8y ago
> Pity they didn’t learn the lesson and maybe Columbia in 2003 could have been saved. But again, mission controllers and engineers ignored the information as “insignificant". That was in the comments. That is the best part of the article. We have a long road to go about learning from things that worked ok by pure luck. But, as usual, we barely learn from mistakes...
- JustSomeNobody 8y ago
  Which should definitely make one worry about our ability to produce self-driving cars that are safe[0]. [0] Yes, safe is relative. Are they safer than humans is one measure. But should we stop there? No. Can we truly even get there? ...
- ubertakter 8y ago
  I don't think any engineers deemed any available information about Columbia as "insignificant", not that there was very much information to go on due to lax execution by camera tracker teams. Management on the other hand was... not effective. I think that's really the crux of your statement. Humans seem to be terrible at working together in large organizations, or at least running them. Organizations seem to learn from mistakes only temporarily, or at all (as you pointed out). Consider the self-driving car group at Uber as a recent example.
  - woliveirajr 8y ago
    Yes, and I agree with you. In my experience, even using some kind of protocol to prioritize which problem should be taken care first, it's easy to miss really important things because someone thought that "time to repair" should give the same score as "severity of consequences", and then we begin to fix things that are "easy and fast", not that are "relevant"
- 8bitsrule 8y ago
  "But, as usual, we barely learn from mistakes... " As in, for example, public warnings by countless actual hackers back in the 80s and 90s about how vulnerable most US systems were. (For which, in return, they were persecuted and prosecuted. With no apologies as said warnings came to deserved fruition.)
radicaldreamer 8y ago
I wonder what kind of payload had to be deployed with the shuttle rather than an unmanned rocket...
- gsnedders 8y ago
  It was the first of the Lacrosse satellites; the remainder were launched on the Titan IV, though the first launch of that wasn't until a year later.
  - deleted 8y ago
    [deleted]
- dasmoth 8y ago
  It was almost certainly a matter of policy (pre-STS-51L, there was still quite a strong push towards the shuttle taking over all US launches) rather than strict necessity. Some flights (STS-41D, for instance) launched civilian comsats which surely can't have had any terribly exotic requirements.
vermontdevil 8y ago
Note the benefit of having a shuttle - the ability to fix the satellite after launching. Too bad the shuttle design (attached to the tank etc) is not ideal.
- java-man 8y ago
  I don't think one needs a shuttle for that.
rdl 8y ago
Launching a (classifies) satellite into a useful-for-spying orbit is one level of secret, although clearly something which could be done with unmanned rockets (as was done later). One of the super secret missions they used to sell the shuttle program was capture (and theft, or tampering and replacement?) of foreign in-orbit satellites. There isn’t any solid evidence this was ever done, but it was a unique capability of something like the shuttle.
- dasmoth 8y ago
  I'm pretty certain any such mission would have leaked by now. At least the fact of it, if not the details. On-orbit servicing of friendly recon sats was considered pretty seriously, though: http://www.thespacereview.com/article/3390/1 http://www.thespacereview.com/article/3390/1
  - mikeash 8y ago
    You’d think the victims would have complained if it had ever happened, too. You can’t hide something like the Shuttle in orbit, so it’ll be clear to anyone who cares to observe.
    - wyldfire 8y ago
      Soviets would not cry that someone took one of their reconnaissance satellites. That would just underscore how far behind they are in space race/star wars.
      - gumby 8y ago
        I don't think the soviets were behind, and in fact they were far ahead in some capabilities (e.g. heavy launch). The program has suffered from neglect since the end of communism though, while the US has made some progress. And it isn't like the US program hasn't suffered from neglect either. The best thing for US space efforts, both orbital and not, has been the end of STS.
        8y ago
JustSomeNobody 8y ago
A couple of thoughts. > Flight controllers were convinced from the grainy images that the damage was not severe and that the crew were mistakenly seeing damage in conditions of poor lighting. I can't help but wonder if they had to say that because there was no real way to either fix the shuttle or retrieve the astronauts. > If the heat shield was damaged, it could spell disaster during the fiery return to Earth and Gibson’s was instructed to use RMS cameras to acquire imagery. Why wasn't this SOP until after we lost another shuttle crew?
- mikeash 8y ago
  Supposedly (I can't find a reliable source), Neil Armstrong was once asked what he would have done with his final hours on the moon if the ascent engine had failed and they had been stranded. His response was that he'd have spent his final hours trying to fix the engine. The point is that there's always something to try, even if it's improbable. Pilots are told to fly as far into the crash as possible, because it never helps to give up, and there might still be something you can do. It's possible that they chose to say that because they thought there was no way to save the astronauts, but they definitely did not have to. And if that's really how it was, they definitely shouldn't have.
- gargravarr 8y ago
  Supposedly NASA was aware of the tile damage to Columbia before re-entry in 2003, but purposefully didn't tell the astronauts because they believed there was indeed no way to fix it; the Shuttle's TPS tiles are right on the bleeding edge of being capable of protecting the craft, just barely thick enough to do their job (thin to save weight) and completely unable to resist impacts. Supposedly the philosophy was such that NASA management would rather the astronauts be completely oblivious to the impending failure during re-entry, carry out a successful mission, and thus experience only a short period of pure horror during re-entry, as opposed to informing them there was nothing to be done and that re-entry would kill them. I have never understood NASA management. It's been responsible for both Shuttle disasters (pushing Challenger to launch despite unsuitable weather). NASA demonstrated what I think is humanity's finest engineering-your-way-out-of-a-problem with Apollo 13, which couldn't benefit from satellite imaging, robot arms or other inventions since, but they seem pretty complacent after that. Source: https://en.wikipedia.org/wiki/Space_Shuttle_Columbia_disaster https://en.wikipedia.org/wiki/Space_Shuttle_Columbia_disaste...
dbcurtis 8y ago
Somewhere in the shuttle design is a lesson about not cramming too much into the MVP. Many years ago, in some high school engineering camp, I remember a university professor who had been involved in the STS design talking about feature creep in the shuttle program in very uncomplimentary terms. According to the prof, the shuttle as originally envisioned would have had about 1/2 the cargo volume, and less throw weight, but in return a higher orbit at engine cut-off, much better serviceability, and a simpler design. In order to get the support of the military, the cargo requirement was pushed to what was just-barely-heroically-possible engineering in order to launch the military satellites desired by the spooks. This decision compromised the program in every way imaginable. The schedule stretched out. The launch-pad weight ballooned, forcing the solid booster design into barely-possible territory. The original goal of "recycle for weekly launches" became a pipe dream. The re-entry heat shield design became very complex. The unit cost ballooned. The cost per launch ballooned. The right answer all along would have been to tell the spooks: "No, damn it, design satellites that fit the payload bay or take a hike." The spooks would have had many years to refine and compact ever-better satellites. We paid a high price for poor program management decisions at the top. The lesson I take out of it: As engineers, we need to think about risk budget at all times. Pushing the envelope on something is probably necessary to beat the competition. Pushing the envelope on everything is poor risk management. As an engineering manager, be very intentional about where risk is allowed. Concentrate your management efforts on de-risking the risky, and delegate the low-risk to clear your mind for managing the unpredictable. I will admit that I have not always done that, and have collected the scars that I deserved.
- trhway 8y ago
  Regarding the feature creep Pentagon Wars come to mind :) Another answer is modularity of design, like Falcon Heavy. Though granted it is much harder for a spaceplane concept and may as well be an ultimate feature creep.
- sgt101 8y ago
  You have to have a lot of power to do that. In most cases you'll just be removed.
- nradov 8y ago
mannykannot 8y ago
"One tile on the shuttle's belly near the nose was completely missing and the underlying metal - a thick mounting plate that helped anchor an antenna - was partially melted. In a slightly different location, the missing tile could have resulted in a catastrophic burn through." https://spaceflightnow.com/shuttle/sts119/090327sts27/ https://spaceflightnow.com/shuttle/sts119/090327sts27/
mannykannot 8y ago
I am wondering why the images of the damage had to be encrypted - what sort of secret about the mission could they reveal? My only guess, which I don't find very convincing, is that it would reveal some modification that had been made to the robot arm, or its camera, for the purpose of the mission.
- hindsightbias 8y ago
  SOP for a Mil mission. Photos could have revealed info on the payload.
- Diederich 8y ago
  I suspect all of the comms were encrypted.
  - mannykannot 8y ago
    I had not thought of that, but there are a couple of quotes that suggest that clear images were technically an option[1]: Shuttle Commander Gibson: "So in other words, the resolution on the encrypted video was that bad that they based a conclusion on it that was in gross error. ... If I had said hey, I think this is important enough for us to break the encryption and send you guys clear video, oh, it would have been pandemonium down there at DOD. But in hindsight, oh man, that's what we should have done." Ascent-Entry Flight Director Wayne Hale: "We were struggling in those days to try to maintain the security classification, so on and so forth... When the crew reported they saw this stuff, we had a long negotiation as I recall with the customer to say well, can we look at the TV? Because we weren't supposed to see any TV from on orbit. (They said) absolutely not. Could we look at the bottom side of the shuttle? That was the agreement, that we could, but we used this special slow-scan TV. And it was grainy." So it looks like a case of "the rules are the rules, period." [1] https://spaceflightnow.com/shuttle/sts119/090327sts27/ https://spaceflightnow.com/shuttle/sts119/090327sts27/
mr_overalls 8y ago
> During their training, they earned the nickname “Swine Flight” from the astronaut office secretaries, and were even given novelty pigs’ snouts, as a result of Gibson’s penchant for making animal-like snorts whenever attractive women were in the vicinity. Gross. I'm glad culture, as well as technology, has evolved since the 80's.
- moron4hire 8y ago
  It has, but let's not confuse that with it being good now. A lot of this sort of thing still goes on at a lot of very high profile places. I've personally seen it happen--and worse!

145 points by pinewurst 8y ago

> Pity they didn’t learn the lesson and maybe Columbia in 2003 could have been saved. But again, mission controllers and engineers ignored the information as “insignificant". That was in the comments. That is the best part of the article. We have a long road to go about learning from things that worked ok by pure luck. But, as usual, we barely learn from mistakes...

JustSomeNobody 8y ago
Which should definitely make one worry about our ability to produce self-driving cars that are safe[0]. [0] Yes, safe is relative. Are they safer than humans is one measure. But should we stop there? No. Can we truly even get there? ...
ubertakter 8y ago
I don't think any engineers deemed any available information about Columbia as "insignificant", not that there was very much information to go on due to lax execution by camera tracker teams. Management on the other hand was... not effective. I think that's really the crux of your statement. Humans seem to be terrible at working together in large organizations, or at least running them. Organizations seem to learn from mistakes only temporarily, or at all (as you pointed out). Consider the self-driving car group at Uber as a recent example.
- woliveirajr 8y ago
  Yes, and I agree with you. In my experience, even using some kind of protocol to prioritize which problem should be taken care first, it's easy to miss really important things because someone thought that "time to repair" should give the same score as "severity of consequences", and then we begin to fix things that are "easy and fast", not that are "relevant"
8bitsrule 8y ago
"But, as usual, we barely learn from mistakes... " As in, for example, public warnings by countless actual hackers back in the 80s and 90s about how vulnerable most US systems were. (For which, in return, they were persecuted and prosecuted. With no apologies as said warnings came to deserved fruition.)

radicaldreamer 8y ago

I wonder what kind of payload had to be deployed with the shuttle rather than an unmanned rocket...

gsnedders 8y ago
It was the first of the Lacrosse satellites; the remainder were launched on the Titan IV, though the first launch of that wasn't until a year later.
- deleted 8y ago
  [deleted]
dasmoth 8y ago
It was almost certainly a matter of policy (pre-STS-51L, there was still quite a strong push towards the shuttle taking over all US launches) rather than strict necessity. Some flights (STS-41D, for instance) launched civilian comsats which surely can't have had any terribly exotic requirements.

vermontdevil 8y ago

Note the benefit of having a shuttle - the ability to fix the satellite after launching. Too bad the shuttle design (attached to the tank etc) is not ideal.

java-man 8y ago
I don't think one needs a shuttle for that.

rdl 8y ago

Launching a (classifies) satellite into a useful-for-spying orbit is one level of secret, although clearly something which could be done with unmanned rockets (as was done later). One of the super secret missions they used to sell the shuttle program was capture (and theft, or tampering and replacement?) of foreign in-orbit satellites. There isn’t any solid evidence this was ever done, but it was a unique capability of something like the shuttle.

dasmoth 8y ago
I'm pretty certain any such mission would have leaked by now. At least the fact of it, if not the details. On-orbit servicing of friendly recon sats was considered pretty seriously, though: http://www.thespacereview.com/article/3390/1 http://www.thespacereview.com/article/3390/1
- mikeash 8y ago
  You’d think the victims would have complained if it had ever happened, too. You can’t hide something like the Shuttle in orbit, so it’ll be clear to anyone who cares to observe.
  - wyldfire 8y ago
    Soviets would not cry that someone took one of their reconnaissance satellites. That would just underscore how far behind they are in space race/star wars.
    - gumby 8y ago
      I don't think the soviets were behind, and in fact they were far ahead in some capabilities (e.g. heavy launch). The program has suffered from neglect since the end of communism though, while the US has made some progress. And it isn't like the US program hasn't suffered from neglect either. The best thing for US space efforts, both orbital and not, has been the end of STS.
      - 8y ago

JustSomeNobody 8y ago

A couple of thoughts. > Flight controllers were convinced from the grainy images that the damage was not severe and that the crew were mistakenly seeing damage in conditions of poor lighting. I can't help but wonder if they had to say that because there was no real way to either fix the shuttle or retrieve the astronauts. > If the heat shield was damaged, it could spell disaster during the fiery return to Earth and Gibson’s was instructed to use RMS cameras to acquire imagery. Why wasn't this SOP until after we lost another shuttle crew?

mikeash 8y ago
Supposedly (I can't find a reliable source), Neil Armstrong was once asked what he would have done with his final hours on the moon if the ascent engine had failed and they had been stranded. His response was that he'd have spent his final hours trying to fix the engine. The point is that there's always something to try, even if it's improbable. Pilots are told to fly as far into the crash as possible, because it never helps to give up, and there might still be something you can do. It's possible that they chose to say that because they thought there was no way to save the astronauts, but they definitely did not have to. And if that's really how it was, they definitely shouldn't have.
gargravarr 8y ago
Supposedly NASA was aware of the tile damage to Columbia before re-entry in 2003, but purposefully didn't tell the astronauts because they believed there was indeed no way to fix it; the Shuttle's TPS tiles are right on the bleeding edge of being capable of protecting the craft, just barely thick enough to do their job (thin to save weight) and completely unable to resist impacts. Supposedly the philosophy was such that NASA management would rather the astronauts be completely oblivious to the impending failure during re-entry, carry out a successful mission, and thus experience only a short period of pure horror during re-entry, as opposed to informing them there was nothing to be done and that re-entry would kill them. I have never understood NASA management. It's been responsible for both Shuttle disasters (pushing Challenger to launch despite unsuitable weather). NASA demonstrated what I think is humanity's finest engineering-your-way-out-of-a-problem with Apollo 13, which couldn't benefit from satellite imaging, robot arms or other inventions since, but they seem pretty complacent after that. Source: https://en.wikipedia.org/wiki/Space_Shuttle_Columbia_disaster https://en.wikipedia.org/wiki/Space_Shuttle_Columbia_disaste...

dbcurtis 8y ago

Somewhere in the shuttle design is a lesson about not cramming too much into the MVP. Many years ago, in some high school engineering camp, I remember a university professor who had been involved in the STS design talking about feature creep in the shuttle program in very uncomplimentary terms. According to the prof, the shuttle as originally envisioned would have had about 1/2 the cargo volume, and less throw weight, but in return a higher orbit at engine cut-off, much better serviceability, and a simpler design. In order to get the support of the military, the cargo requirement was pushed to what was just-barely-heroically-possible engineering in order to launch the military satellites desired by the spooks. This decision compromised the program in every way imaginable. The schedule stretched out. The launch-pad weight ballooned, forcing the solid booster design into barely-possible territory. The original goal of "recycle for weekly launches" became a pipe dream. The re-entry heat shield design became very complex. The unit cost ballooned. The cost per launch ballooned. The right answer all along would have been to tell the spooks: "No, damn it, design satellites that fit the payload bay or take a hike." The spooks would have had many years to refine and compact ever-better satellites. We paid a high price for poor program management decisions at the top. The lesson I take out of it: As engineers, we need to think about risk budget at all times. Pushing the envelope on something is probably necessary to beat the competition. Pushing the envelope on everything is poor risk management. As an engineering manager, be very intentional about where risk is allowed. Concentrate your management efforts on de-risking the risky, and delegate the low-risk to clear your mind for managing the unpredictable. I will admit that I have not always done that, and have collected the scars that I deserved.

trhway 8y ago
Regarding the feature creep Pentagon Wars come to mind :) Another answer is modularity of design, like Falcon Heavy. Though granted it is much harder for a spaceplane concept and may as well be an ultimate feature creep.
sgt101 8y ago
You have to have a lot of power to do that. In most cases you'll just be removed.
nradov 8y ago

mannykannot 8y ago

"One tile on the shuttle's belly near the nose was completely missing and the underlying metal - a thick mounting plate that helped anchor an antenna - was partially melted. In a slightly different location, the missing tile could have resulted in a catastrophic burn through." https://spaceflightnow.com/shuttle/sts119/090327sts27/ https://spaceflightnow.com/shuttle/sts119/090327sts27/

mannykannot 8y ago

I am wondering why the images of the damage had to be encrypted - what sort of secret about the mission could they reveal? My only guess, which I don't find very convincing, is that it would reveal some modification that had been made to the robot arm, or its camera, for the purpose of the mission.

hindsightbias 8y ago
SOP for a Mil mission. Photos could have revealed info on the payload.
Diederich 8y ago
I suspect all of the comms were encrypted.
- mannykannot 8y ago
  I had not thought of that, but there are a couple of quotes that suggest that clear images were technically an option[1]: Shuttle Commander Gibson: "So in other words, the resolution on the encrypted video was that bad that they based a conclusion on it that was in gross error. ... If I had said hey, I think this is important enough for us to break the encryption and send you guys clear video, oh, it would have been pandemonium down there at DOD. But in hindsight, oh man, that's what we should have done." Ascent-Entry Flight Director Wayne Hale: "We were struggling in those days to try to maintain the security classification, so on and so forth... When the crew reported they saw this stuff, we had a long negotiation as I recall with the customer to say well, can we look at the TV? Because we weren't supposed to see any TV from on orbit. (They said) absolutely not. Could we look at the bottom side of the shuttle? That was the agreement, that we could, but we used this special slow-scan TV. And it was grainy." So it looks like a case of "the rules are the rules, period." [1] https://spaceflightnow.com/shuttle/sts119/090327sts27/ https://spaceflightnow.com/shuttle/sts119/090327sts27/

mr_overalls 8y ago

> During their training, they earned the nickname “Swine Flight” from the astronaut office secretaries, and were even given novelty pigs’ snouts, as a result of Gibson’s penchant for making animal-like snorts whenever attractive women were in the vicinity. Gross. I'm glad culture, as well as technology, has evolved since the 80's.

moron4hire 8y ago
It has, but let's not confuse that with it being good now. A lot of this sort of thing still goes on at a lot of very high profile places. I've personally seen it happen--and worse!