5 ms·

> According to Russia Today, the person responsible for the hole has been identified. That was pretty scary sentence for me. Short story, I know the guy who ma

> According to Russia Today, the person responsible for the hole has been identified. That was pretty scary sentence for me. Short story, I know the guy who made the mistake on the mirror for the Hubble Telescope. https://www.nasa.gov/content/hubbles-mirror-flaw https://www.nasa.gov/content/hubbles-mirror-flaw He was a great guy and was the President for the Trouts Unlimited when I was a kid. That one thing defined him for so long. Was pretty sad story to me. He did one mistake but the quality control department were never troubled like he was even though they had it right in front of them for years.

stareatgoats 8y ago
Yea, accidents and mistakes should obviously be seen as learning opportunities, and the ones making them should really be treated as heroes for demonstrating how things can go wrong. But when millions (billions?) of dollars and/or lives are put at risk you better have a good explanation, that's for sure. Putting glue in an accidental hole of a space vehicle or similar and hoping no-one discovers would probably not help your case further (if it were to be discovered ...). The guy who miscalibrated the machinery that made the Hubble telescope (1/50th of a hairs breadth error) may have had a good explanation for all I know- care to share the details?
- CWuestefeld 8y ago
  My grandfather worked on the HST mirror. It's both what he was most proud of, and the most ashamed of, at the same time. He talked about this not really as a single person's fault (except in terms of a senior manager's failure), but as an organizational problem. He pointed out two things: 1. In particular, the organization was poorly designed, with QA being too close (in org chart terms) to production. If there's one finger to point, it's at the guy who was the common manager of both departments. And I think this is something that the engineering profession has taken to heart since then. 2. People at the time thought of things with somewhat different priorities because the context changed. At the time the mirror was being built, it was understood that the space shuttle would make it relatively simpler to fix serious problems (well, as "simpler" as any space missions could be). As it turned out, the space shuttle was less effective in that role than its initial requirements led HST management to believe. This probably led to a more relaxed attitude about the ultimate result of any problems.
  - jacquesm 8y ago
    > In particular, the organization was poorly designed, with QA being too close (in org chart terms) to production. I see this with great regularity in my practice and it is really strange that this lesson has to be learned all over time and again. Latest incarnation: make developers responsible for their own QA. That's about as logical as asking the butcher to grade their own meat.
    - dmix 8y ago
      > QA being too close (in org chart terms) to production. Can you explain in a bit more detail what this means in practice? I haven't worked with a QA team before and I'm curious. Does it just mean the QA checks happen well after the specific work was being done, making it harder to go back and fix things (due to mental context switching, tooling, etc)? Or just too many last-minute fixes?
      - karambahh 8y ago
        My positive experiences with QA were always when the QA team was strictly separated from the production team. Ideally, no friendship whatsoever. Doing QA is hard. Bouncing issues & QA reports between teams is hard. Bouncing issues between people that know each others too well or have a common manager leads to all sorts of issues. In this specific instance, it could be as simple as the manager trying to balance things between teams and thus letting things slip past the due date or with sub-par quality. My worst experience was when a married couple worked together on a project I was involved with: one on the sw team and the other in QA. Working together is hard, working on "antagonistic" teams is hell
    - DanBC 8y ago
      > Latest incarnation: make developers responsible for their own QA. That's about as logical as asking the butcher to grade their own meat. No, you're wrong. Self certification works when it's done properly. As with any QA system it fails if it's done poorly, but any QA will fail when done poorly, and self certification is less likely to fail. I built safety critical equipment for coal mining, and some aerospace test equipment, and saw a variety of QA systems from very good (MTBF 14 years) and piss poor (product broken when leaving the factory).
- village-idiot 8y ago
  IIRC a calibration rod had some paint missing from one end, which affected measurements for grinding.
  - baldfat 8y ago
    Well I heard that it was actually installed upside down. Well at least that was what the local rumors were.
CommieBobDole 8y ago
I always wondered how this happened, given that that the HST was far from a "one-off" product - would love to know if this was an error that affected all of the KH-11 mirrors up to that point (and went undetected), or if there was something about the Hubble mirror that was different? Unfortunately, that information is probably still classified.
- kurthr 8y ago
  The focal length of the Hubble was certainly different, but maybe other things too... I'd be surprised if this wasn't a copy/paste error, but maybe it really was some huge miscalculation that nobody caught. There were a lot of KHs (Keyhole) flying long before Hubble so they didn't have unfixed issues and their test fixtures were all set up (and frankly, probably passed Hubble). https://en.wikipedia.org/wiki/KH-11_Kennen https://en.wikipedia.org/wiki/KH-11_Kennen
  - baldfat 8y ago
    The mirror was too flat in the middle and there was a slight wave. It was a mechanical issue when the polished the mirror starting in 1979 for over a year. It wasn't launched till 1990.
- baldfat 8y ago
  They state that the error in the mirror if you enlarged to to the size of the earth would not be any higher than 6 inches. http://www.scienceclarified.com/scitech/Telescopes/Hubble.html http://www.scienceclarified.com/scitech/Telescopes/Hubble.ht... is a great article on it. Since this all happened in my area I was always interested. The fix of the mirror is one of the greatest feats I have ever read. They weren't sure how much could be fixed. It ended up being better than anyone could ever expect.
skj 8y ago
"An unnamed source told RIA Novosti the person responsible for drilling the hole filled it with glue instead of reporting the mistake." A mistake is one thing. Mistakes are lessons and it's good to have people who have some solid lessons in their backgrounds. Trying to patch it with some glue without telling anyone is a character flaw and the person needs to go, so everyone else gets the lesson.
- amelius 8y ago
  True, but apparently still a quality control should be in place to find such problems.
  - stcredzero 8y ago
    It's precisely this kind of bodging which caused great loss of life with the de Haviland Comet. Installing the radio antenna opening with punch rivets instead of drilling out the holes ended up killing two planeloads of people. https://en.wikipedia.org/wiki/De_Havilland_Comet https://en.wikipedia.org/wiki/De_Havilland_Comet
- grawprog 8y ago
  Ya. That's how things operate at my work and it works well. As much as it sucks mistakes happen, things get damaged, sometimes things worth lots of money. If feels really shitty to have to go tell someone you made a mistake, that a bunch of time got wasted, we're going to have to redo work, that you just destroyed a couple thousand dollars worth of things. It really sucks... But... it sucks a whole lot less than when someone tries to cover up or hide a mistake. It always ends up coming out in the end, it always makes things worse than they could have been in the end and everyone ends up even more pissed off than they would have been. It's not worth it. Actually, my parents forced me to learn this lesson young. Me and two other kids did something fairly terrible when I was in elementary school. My parents took me and made me personally apologize to the person who detrimented from our actions. It wasn't the most pleasant of experiences but in the end t was the right thing to do and it taught me, if you fuck up, the best thing you can do is admit it, try and make it right and learn from it. I've been trying to teach the new guy at my work this. He started off worried about making mistakes, would beat himself up and try to cover them up. It's been a few months of talking to him and telling him to focus on learning and not worrying about mistakes. If they happen, I get him to work through what went wrong before anything too bad happens and i've found his attitude has changed quite a bit since he first started. He doesn't worry so much and he's been learning a lot more quickly. When something does happen, instead of panicking he just comes to tell me and i'll help him work though figuring it out. We're all human, mistakes happen. They just need to be dealt with in a reasonable way and if you make a mistake admit it and if you can't figure out how to fix the problem there's no shame in asking for help.

stareatgoats 8y ago

Yea, accidents and mistakes should obviously be seen as learning opportunities, and the ones making them should really be treated as heroes for demonstrating how things can go wrong. But when millions (billions?) of dollars and/or lives are put at risk you better have a good explanation, that's for sure. Putting glue in an accidental hole of a space vehicle or similar and hoping no-one discovers would probably not help your case further (if it were to be discovered ...). The guy who miscalibrated the machinery that made the Hubble telescope (1/50th of a hairs breadth error) may have had a good explanation for all I know- care to share the details?

CWuestefeld 8y ago
My grandfather worked on the HST mirror. It's both what he was most proud of, and the most ashamed of, at the same time. He talked about this not really as a single person's fault (except in terms of a senior manager's failure), but as an organizational problem. He pointed out two things: 1. In particular, the organization was poorly designed, with QA being too close (in org chart terms) to production. If there's one finger to point, it's at the guy who was the common manager of both departments. And I think this is something that the engineering profession has taken to heart since then. 2. People at the time thought of things with somewhat different priorities because the context changed. At the time the mirror was being built, it was understood that the space shuttle would make it relatively simpler to fix serious problems (well, as "simpler" as any space missions could be). As it turned out, the space shuttle was less effective in that role than its initial requirements led HST management to believe. This probably led to a more relaxed attitude about the ultimate result of any problems.
- jacquesm 8y ago
  > In particular, the organization was poorly designed, with QA being too close (in org chart terms) to production. I see this with great regularity in my practice and it is really strange that this lesson has to be learned all over time and again. Latest incarnation: make developers responsible for their own QA. That's about as logical as asking the butcher to grade their own meat.
  - dmix 8y ago
    > QA being too close (in org chart terms) to production. Can you explain in a bit more detail what this means in practice? I haven't worked with a QA team before and I'm curious. Does it just mean the QA checks happen well after the specific work was being done, making it harder to go back and fix things (due to mental context switching, tooling, etc)? Or just too many last-minute fixes?
    - karambahh 8y ago
      My positive experiences with QA were always when the QA team was strictly separated from the production team. Ideally, no friendship whatsoever. Doing QA is hard. Bouncing issues & QA reports between teams is hard. Bouncing issues between people that know each others too well or have a common manager leads to all sorts of issues. In this specific instance, it could be as simple as the manager trying to balance things between teams and thus letting things slip past the due date or with sub-par quality. My worst experience was when a married couple worked together on a project I was involved with: one on the sw team and the other in QA. Working together is hard, working on "antagonistic" teams is hell
  - DanBC 8y ago
    > Latest incarnation: make developers responsible for their own QA. That's about as logical as asking the butcher to grade their own meat. No, you're wrong. Self certification works when it's done properly. As with any QA system it fails if it's done poorly, but any QA will fail when done poorly, and self certification is less likely to fail. I built safety critical equipment for coal mining, and some aerospace test equipment, and saw a variety of QA systems from very good (MTBF 14 years) and piss poor (product broken when leaving the factory).
village-idiot 8y ago
IIRC a calibration rod had some paint missing from one end, which affected measurements for grinding.
- baldfat 8y ago
  Well I heard that it was actually installed upside down. Well at least that was what the local rumors were.

CommieBobDole 8y ago

I always wondered how this happened, given that that the HST was far from a "one-off" product - would love to know if this was an error that affected all of the KH-11 mirrors up to that point (and went undetected), or if there was something about the Hubble mirror that was different? Unfortunately, that information is probably still classified.

kurthr 8y ago
The focal length of the Hubble was certainly different, but maybe other things too... I'd be surprised if this wasn't a copy/paste error, but maybe it really was some huge miscalculation that nobody caught. There were a lot of KHs (Keyhole) flying long before Hubble so they didn't have unfixed issues and their test fixtures were all set up (and frankly, probably passed Hubble). https://en.wikipedia.org/wiki/KH-11_Kennen https://en.wikipedia.org/wiki/KH-11_Kennen
- baldfat 8y ago
  The mirror was too flat in the middle and there was a slight wave. It was a mechanical issue when the polished the mirror starting in 1979 for over a year. It wasn't launched till 1990.
baldfat 8y ago
They state that the error in the mirror if you enlarged to to the size of the earth would not be any higher than 6 inches. http://www.scienceclarified.com/scitech/Telescopes/Hubble.html http://www.scienceclarified.com/scitech/Telescopes/Hubble.ht... is a great article on it. Since this all happened in my area I was always interested. The fix of the mirror is one of the greatest feats I have ever read. They weren't sure how much could be fixed. It ended up being better than anyone could ever expect.

skj 8y ago

"An unnamed source told RIA Novosti the person responsible for drilling the hole filled it with glue instead of reporting the mistake." A mistake is one thing. Mistakes are lessons and it's good to have people who have some solid lessons in their backgrounds. Trying to patch it with some glue without telling anyone is a character flaw and the person needs to go, so everyone else gets the lesson.

amelius 8y ago
True, but apparently still a quality control should be in place to find such problems.
- stcredzero 8y ago
  It's precisely this kind of bodging which caused great loss of life with the de Haviland Comet. Installing the radio antenna opening with punch rivets instead of drilling out the holes ended up killing two planeloads of people. https://en.wikipedia.org/wiki/De_Havilland_Comet https://en.wikipedia.org/wiki/De_Havilland_Comet
grawprog 8y ago
Ya. That's how things operate at my work and it works well. As much as it sucks mistakes happen, things get damaged, sometimes things worth lots of money. If feels really shitty to have to go tell someone you made a mistake, that a bunch of time got wasted, we're going to have to redo work, that you just destroyed a couple thousand dollars worth of things. It really sucks... But... it sucks a whole lot less than when someone tries to cover up or hide a mistake. It always ends up coming out in the end, it always makes things worse than they could have been in the end and everyone ends up even more pissed off than they would have been. It's not worth it. Actually, my parents forced me to learn this lesson young. Me and two other kids did something fairly terrible when I was in elementary school. My parents took me and made me personally apologize to the person who detrimented from our actions. It wasn't the most pleasant of experiences but in the end t was the right thing to do and it taught me, if you fuck up, the best thing you can do is admit it, try and make it right and learn from it. I've been trying to teach the new guy at my work this. He started off worried about making mistakes, would beat himself up and try to cover them up. It's been a few months of talking to him and telling him to focus on learning and not worrying about mistakes. If they happen, I get him to work through what went wrong before anything too bad happens and i've found his attitude has changed quite a bit since he first started. He doesn't worry so much and he's been learning a lot more quickly. When something does happen, instead of panicking he just comes to tell me and i'll help him work though figuring it out. We're all human, mistakes happen. They just need to be dealt with in a reasonable way and if you make a mistake admit it and if you can't figure out how to fix the problem there's no shame in asking for help.