7 ms·

Did we throw away everything we learned in the apollo program? They won’t be starting from scratch on the design.

NikolaeVarius 7y ago
A good chunk of Apollo knowledge isn't very useful for various reasons. 1) The tribal knowledge to even implement Apollo era technology is pretty much gone 2) Apollo flight hardware doesn't exist, probably couldn't even be manufactured without extremely expensive tooling and build up to recreate outdated hardware. 3) Lots of Apollo era designs don't exist on paper. The F1 engine doesn't really even have a standard design, since every iteration was changed to be "better" by throwing shit against the wall and seeing what sticked.
- mandelbrotwurst 7y ago
  Would you mind elaborating on that first point?
  - Aloha 7y ago
    The people who did it are dead or dying, the tribal knowledge wasn't written down.
    - azernik 7y ago
      From my Jewish perspective, I think of it as oral tradition vs written tradition (like the pre-Mishnaic distinction between the Written Law and the Oral Law).
      - geogra4 7y ago
        They need a space flight talmud
        azernik 7y ago
        In general, I see good commit messages as a Talmudic codification of a company's oral knowledge - complete with back-and-forth conversations, contradictory points of view and distinct characters. Still better than having to go talk to the people involved, some of whom may no longer be in the organization. (Specifically with commit messages as Gemara and wikis as the much more organized and impersonal Mishnah.)
        IIAOPSW 7y ago
        Or, hear me out, let's add the space flight knowledge to the existing talmud.
      - ngcc_hk 7y ago
        It is still surprise to me the switch to woman based Jewish identity and the importance of oral tradition in your culture. Also shocked when Dead Sea Scroll come out, I only knew then the Jewish bible oldest is only 1000 years old.
        flatfilefan 7y ago
        There’s a theory that all written artifacts are at most 1000 years old. Christian dates (in Roman) got M (1000) infront of all dates and Jewish an V (5000). So we got 2000 years and 6000 claimed history. Having written the above it appeared to me this could be an ironic illustration of tribal knowledge vs technical documentation issue. Knowing when something happened and reading about it are two separate things.
    - teh_infallible 7y ago
      Isn’t it odd that the most important political and military mission of a generation relied on “tribal information which wasn’t written down?”
      - NikolaeVarius 7y ago
        Not really? Especially in an era where you had to literally write things down versus throwing in a link to a slack message onto a Jira ticket somewhere (Which no one will ever read anyway since text search in Jira sucks) Also its worth emphasizing that a lot of the actual physics regarding like rocket combustion was not understood so the design process had elements of "make a rocket engine that satisfies some generic constraints and we can figure out the black magic parts such as POGO". Not that useful to write down.
      - ardy42 7y ago
        > Isn’t it odd that the most important political and military mission of a generation relied on “tribal information which wasn’t written down?” No. Figuring out how to accurately describe something in writing is hard, and then there are all the details you have to leave out to make the document readable, some of which may be more important than you realized. Every project relies on tribal knowledge, and much of that knowledge will be lost unless is followed up by an equally massive project by a horde of technical writers to nail everything down.
        pbhjpbhj 7y ago
        Technical documents don't have to be readable, they have to be comprehensible (and preferably quite comprehensive too).
        TeMPOraL 7y ago
        They have to be readable enough for the reader to be able to recreate what's described from those documents, and you can only test for that by... trying to use the documents to recreate the work. Which is rarely attempted, especially in large, expensive projects. (It's the same thing as the "you only have a backup if you actually restored it at least once" adage.)
    - 2dollars27cents 7y ago
      I'm sure this is a valid point, but I think it is overemphasized. Much of that tribal knowledge would have been very esoteric and focused on technologies that are no longer critical. for example, fabricating magnetic core memory. I'm sure some of the required skill is gone, but we don't need to use that memory technology anymore.
      - sigstoat 7y ago
        having been involved in spacecraft operations, it has nothing to do with how the memory is wired up. there would've been a huge amount of knowledge about how to operate apollo spacecraft in general. there is no way it is preserved. and this thread is beneath a comment suggesting that we don't start from scratch, and reuse apollo designs. as you point out, we can make better stuff now. the only value to the apollo designs would be if we had operational experience to go with them.
      - fsloth 7y ago
        I think the computer aspects are not the most difficult part of the tech stack when dealing with space flight.
        sandworm101 7y ago
        Tell that to Boeing. Thier last flight didn't make it to the space station because of two computer errors. Had that happened over the moon, people likely would have died.
    - mandelbrotwurst 7y ago
      Hm, it seems I should have been more specific. I am aware of the mechanism, I'm hoping someone might have even a bit of insight into what that tribal knowledge is/was.
      - NikolaeVarius 7y ago
        https://www.history.com/news/moon-landing-technology-inventions-computers-heat-shield-rovers https://www.history.com/news/moon-landing-technology-inventi... Stuff like > Finally, the parachutes were folded and packed by hand. During the Apollo missions in the 1960s and early 1970s, only three people in the country were trained, and then licensed by the Federal Aviation Administration, to fold Apollo parachutes—Norma Cretal, Buzz Corey and Jimmy Calunga —and they handled all 11 Apollo missions. Their skills were considered so essential that NASA forbade them from ever riding in the same car together. The agency couldn’t afford to chance that all three would be injured in a single accident. I also doubt that the seamstresses techniques to sew those parachutes are something easily written down
    - throwawayhhakdl 7y ago
      I don’t think that’s the right way to phrase it. Complicated things are probably written down somewhere, but it’s just not practical to assimilate the knowledge without having been involved in its creation. The loss of this kind of thinking isn’t so much a failure of people to write documentation, but a natural consequence of things being hard.
  - gravypod 7y ago
    I think this is a common problem experienced when documenting the fabrication needs of scientific equipment. I used to work for a lab that had a physicist who worked a lot with magnetometers. A magnetometer is literally just a coil of wire around a solid core. How complex could that be to document? Well he mentioned that there was one alloy that they used for pretty much all of the high precision fluxgate magnetometers were constructed from one alloy. The person/team/company that had made this alloy had went belly up and while they published the contents for the alloy they did not publish the exact steps for how they made this alloy. Because of this some research teams had stockpiled some of this material and it had apparently gave them a bit of an edge. We were "lucky" that my coworker knew a guy who knew a guy who knew a guy in Canada who also had some of this who could build us a magnetometer for our use case as he had some. This is the best article I could find about this: https://www.geosci-instrum-method-data-syst.net/8/227/2019/gi-8-227-2019.pdf https://www.geosci-instrum-method-data-syst.net/8/227/2019/g... This happens because it's almost impossible to write a paper that includes every single detail needed to reproduce your work and fit within any journal's page count limits. For metallurgy it's like trying to write down your family's double chocolate chip cookie recipe except your chocolate chips are atoms, your oven's temperature varies +-500C, and all of your ingredients can spoil when mixed in the wrong order, cooled at the wrong rate, heated at the wrong rate, aren't mixed with trace elements of other ingredients that came in your ingredients, etc. I have no experience with this but I've heard this also applies to chemistry glass blowers: https://www.nmr.mgh.harvard.edu/news/20150127/nearly-lost-art-scientific-glassblowing https://www.nmr.mgh.harvard.edu/news/20150127/nearly-lost-ar...
    - deleted 7y ago
      [deleted]
    - skriticos2 7y ago
      Metallurgy is fascinating! This reminds me of the semi-secret one crystal cooling process for airplane turbine segments. The entire thing comes out as one crystal and you need very precise temperature control not just as a whole or in various spots but in various gradients of the oven to get the thing come out properly. It would take anyone forever to replicate this from scratch, even though I can easily explain what the expected end result should look like.
  - mattrp 7y ago
    A little tangential but the SLS itself is an attempt to reuse technology off the shuttle... and yet, hundreds of billions later and nothing... the whole concept of technology repurposing is fine on paper... but for some reason NASA can’t even seem to do that right either. It’s a colossal mess...
    - Cthulhu_ 7y ago
      Sounds like software; trying to build on top of 40+ year old software vs building something new.
    - deleted 7y ago
      [deleted]
- ChickeNES 7y ago
  > 3) Lots of Apollo era designs don't exist on paper. The F1 engine doesn't really even have a standard design, since every iteration was changed to be "better" by throwing shit against the wall and seeing what sticked. I believe the F-1 specifically does have full CAD models now after a reverse engineering effort, though in general your point still stands. They attempted to restart J-2 production for Ares, but ran into so many complications that the J-2X was basically a clean-sheet design
azernik 7y ago
In addition to technology loss mentioned in other replies, reproducing Apollo designs without major redesigns is not a viable option for simple cost reasons: no NASA program in the foreseeable future will have anything near Apollo's budget, and so any new moon lander will have to be much cheaper. Given the advances in control systems, simulation, and materials I think this is very doable, but it's work.
- harryh 7y ago
  NASA's budget during the Apollo era peacked in 1966 around 4.5 billion. That's about 35 billion in today's dollars. That's a fair bit more than the actual NASA budget of 23.5 billion but it's not all that far off. The money is there. The challenges are mostly, as other's have noted, large changes in safety requirements and general execution challenges for large scale projects in today's America.
  - NikolaeVarius 7y ago
    During Apollo a majority of that budget was dedicated to Apollo. Modern NASA maintains the ISS, multiple observatories, satellites, the DSN, and dozens of similar projects, budget alone does not paint a clear picture
    - harryh 7y ago
      Ya that's true. Apollo was 60-70% of NASA's budget for close to a decade. There's a lot more going on now. If anyone is curios about the details, there's a good summary here: https://www.nasa.gov/sites/default/files/atoms/files/fy2020_summary_budget_brief.pdf https://www.nasa.gov/sites/default/files/atoms/files/fy2020_...
WalterBright 7y ago
I have a friend who ordered the mechanical drawings for a P-51, intending to make P-51 assemblies. In investigating this, he talked to some P-51 experts who told him that the drawings were nearly useless. The real specification of the airplane was the tools and jigs created to make and assemble the parts. Those are long gone. The mechanical drawings were made after the fact to satisfy some DoD requirement.
- netman21 7y ago
  This. I used to work in automotive engineering. I marveled at the lack of security around drawings for mechanical components. Ford, GM, and Chrysler all used the same design shops to create blueprints. When I went to get the latest drawings they would be sitting out on a big table in the lobby. The rolled up blueprint on top of the 9"mag tape that contained the CAD files. They were clearly marked with the car's model year and the part name and number. Anyone could grab it off the table. Only later in my career did I understand that the drawing for a component, while necessary, was more of a record, not an actual representation of the final component. The final part was a product of a tool that had been crafted to produce it. Die barbers had polished a stamping tool to get a wrinkle out. Someone in manufacturing had changed the melt time for a casting. The slump temperature for a windshield glass had been adjusted. A spray pattern for body paint was manually plugged into a robot arm and adjusted until the desired results were obtained.
  - jsymolon 7y ago
    chuckle, Classic case of the documentation doesn't match the "code".
- pjc50 7y ago
  Yes. The UK encountered a variant of this with its project to extend the life of the Nimrod recon aircraft, which were built on airframes that had essentially been designed in the 1950s. The aircraft all being individual meant that a conventional waterfall project to build new parts and retrofit them encountered huge cost overruns. Someone described it as more of an "aircraft reenactment society" than a viable aircraft for the 21st century. One caught fire in the air and the project was eventually abandoned.
ngcc_hk 7y ago
Organisation memory perhaps is a better word though not tribal / cultural memory. You can’t write everything done and for an ongoing concern you always have people around. In fact one of the problem is these Organization memory is sometimes in the way and you do not know why or even that exist. But here is about the absence, as it is not an on-going concern.
- cs02rm0 7y ago
  Organisation would imply to me that the knowledge was held by the organisation as a whole though. Tribal implies small groups of individuals, which is probably closer to the truth.
Cthulhu_ 7y ago
Some things will have to be rebuilt though. I read a story the other day (probably from HN as well) about rebuilding an older model rocket engine, because they still had the plans. Turns out a lot of that depended on the skill of the engineers, welders, machinists etc at the time, which could compensate for small changes on the fly. It didn't work the same with modern manufacturing technologies and materials. Mind you I do believe the modern stuff is better, they can do the same or better performance with less material and waste.

A good chunk of Apollo knowledge isn't very useful for various reasons. 1) The tribal knowledge to even implement Apollo era technology is pretty much gone 2) Apollo flight hardware doesn't exist, probably couldn't even be manufactured without extremely expensive tooling and build up to recreate outdated hardware. 3) Lots of Apollo era designs don't exist on paper. The F1 engine doesn't really even have a standard design, since every iteration was changed to be "better" by throwing shit against the wall and seeing what sticked.

mandelbrotwurst 7y ago
Would you mind elaborating on that first point?
- Aloha 7y ago
  The people who did it are dead or dying, the tribal knowledge wasn't written down.
  - azernik 7y ago
    From my Jewish perspective, I think of it as oral tradition vs written tradition (like the pre-Mishnaic distinction between the Written Law and the Oral Law).
    - geogra4 7y ago
      They need a space flight talmud
      - azernik 7y ago
        In general, I see good commit messages as a Talmudic codification of a company's oral knowledge - complete with back-and-forth conversations, contradictory points of view and distinct characters. Still better than having to go talk to the people involved, some of whom may no longer be in the organization. (Specifically with commit messages as Gemara and wikis as the much more organized and impersonal Mishnah.)
      - IIAOPSW 7y ago
        Or, hear me out, let's add the space flight knowledge to the existing talmud.
    - ngcc_hk 7y ago
      It is still surprise to me the switch to woman based Jewish identity and the importance of oral tradition in your culture. Also shocked when Dead Sea Scroll come out, I only knew then the Jewish bible oldest is only 1000 years old.
      - flatfilefan 7y ago
        There’s a theory that all written artifacts are at most 1000 years old. Christian dates (in Roman) got M (1000) infront of all dates and Jewish an V (5000). So we got 2000 years and 6000 claimed history. Having written the above it appeared to me this could be an ironic illustration of tribal knowledge vs technical documentation issue. Knowing when something happened and reading about it are two separate things.
  - teh_infallible 7y ago
    Isn’t it odd that the most important political and military mission of a generation relied on “tribal information which wasn’t written down?”
    - NikolaeVarius 7y ago
      Not really? Especially in an era where you had to literally write things down versus throwing in a link to a slack message onto a Jira ticket somewhere (Which no one will ever read anyway since text search in Jira sucks) Also its worth emphasizing that a lot of the actual physics regarding like rocket combustion was not understood so the design process had elements of "make a rocket engine that satisfies some generic constraints and we can figure out the black magic parts such as POGO". Not that useful to write down.
    - ardy42 7y ago
      > Isn’t it odd that the most important political and military mission of a generation relied on “tribal information which wasn’t written down?” No. Figuring out how to accurately describe something in writing is hard, and then there are all the details you have to leave out to make the document readable, some of which may be more important than you realized. Every project relies on tribal knowledge, and much of that knowledge will be lost unless is followed up by an equally massive project by a horde of technical writers to nail everything down.
      - pbhjpbhj 7y ago
        Technical documents don't have to be readable, they have to be comprehensible (and preferably quite comprehensive too).
        TeMPOraL 7y ago
        They have to be readable enough for the reader to be able to recreate what's described from those documents, and you can only test for that by... trying to use the documents to recreate the work. Which is rarely attempted, especially in large, expensive projects. (It's the same thing as the "you only have a backup if you actually restored it at least once" adage.)
  - 2dollars27cents 7y ago
    I'm sure this is a valid point, but I think it is overemphasized. Much of that tribal knowledge would have been very esoteric and focused on technologies that are no longer critical. for example, fabricating magnetic core memory. I'm sure some of the required skill is gone, but we don't need to use that memory technology anymore.
    - sigstoat 7y ago
      having been involved in spacecraft operations, it has nothing to do with how the memory is wired up. there would've been a huge amount of knowledge about how to operate apollo spacecraft in general. there is no way it is preserved. and this thread is beneath a comment suggesting that we don't start from scratch, and reuse apollo designs. as you point out, we can make better stuff now. the only value to the apollo designs would be if we had operational experience to go with them.
    - fsloth 7y ago
      I think the computer aspects are not the most difficult part of the tech stack when dealing with space flight.
      - sandworm101 7y ago
        Tell that to Boeing. Thier last flight didn't make it to the space station because of two computer errors. Had that happened over the moon, people likely would have died.
  - mandelbrotwurst 7y ago
    Hm, it seems I should have been more specific. I am aware of the mechanism, I'm hoping someone might have even a bit of insight into what that tribal knowledge is/was.
    - NikolaeVarius 7y ago
      https://www.history.com/news/moon-landing-technology-inventions-computers-heat-shield-rovers https://www.history.com/news/moon-landing-technology-inventi... Stuff like > Finally, the parachutes were folded and packed by hand. During the Apollo missions in the 1960s and early 1970s, only three people in the country were trained, and then licensed by the Federal Aviation Administration, to fold Apollo parachutes—Norma Cretal, Buzz Corey and Jimmy Calunga —and they handled all 11 Apollo missions. Their skills were considered so essential that NASA forbade them from ever riding in the same car together. The agency couldn’t afford to chance that all three would be injured in a single accident. I also doubt that the seamstresses techniques to sew those parachutes are something easily written down
  - throwawayhhakdl 7y ago
    I don’t think that’s the right way to phrase it. Complicated things are probably written down somewhere, but it’s just not practical to assimilate the knowledge without having been involved in its creation. The loss of this kind of thinking isn’t so much a failure of people to write documentation, but a natural consequence of things being hard.
- gravypod 7y ago
  I think this is a common problem experienced when documenting the fabrication needs of scientific equipment. I used to work for a lab that had a physicist who worked a lot with magnetometers. A magnetometer is literally just a coil of wire around a solid core. How complex could that be to document? Well he mentioned that there was one alloy that they used for pretty much all of the high precision fluxgate magnetometers were constructed from one alloy. The person/team/company that had made this alloy had went belly up and while they published the contents for the alloy they did not publish the exact steps for how they made this alloy. Because of this some research teams had stockpiled some of this material and it had apparently gave them a bit of an edge. We were "lucky" that my coworker knew a guy who knew a guy who knew a guy in Canada who also had some of this who could build us a magnetometer for our use case as he had some. This is the best article I could find about this: https://www.geosci-instrum-method-data-syst.net/8/227/2019/gi-8-227-2019.pdf https://www.geosci-instrum-method-data-syst.net/8/227/2019/g... This happens because it's almost impossible to write a paper that includes every single detail needed to reproduce your work and fit within any journal's page count limits. For metallurgy it's like trying to write down your family's double chocolate chip cookie recipe except your chocolate chips are atoms, your oven's temperature varies +-500C, and all of your ingredients can spoil when mixed in the wrong order, cooled at the wrong rate, heated at the wrong rate, aren't mixed with trace elements of other ingredients that came in your ingredients, etc. I have no experience with this but I've heard this also applies to chemistry glass blowers: https://www.nmr.mgh.harvard.edu/news/20150127/nearly-lost-art-scientific-glassblowing https://www.nmr.mgh.harvard.edu/news/20150127/nearly-lost-ar...
  - deleted 7y ago
    [deleted]
  - skriticos2 7y ago
    Metallurgy is fascinating! This reminds me of the semi-secret one crystal cooling process for airplane turbine segments. The entire thing comes out as one crystal and you need very precise temperature control not just as a whole or in various spots but in various gradients of the oven to get the thing come out properly. It would take anyone forever to replicate this from scratch, even though I can easily explain what the expected end result should look like.
- mattrp 7y ago
  A little tangential but the SLS itself is an attempt to reuse technology off the shuttle... and yet, hundreds of billions later and nothing... the whole concept of technology repurposing is fine on paper... but for some reason NASA can’t even seem to do that right either. It’s a colossal mess...
  - Cthulhu_ 7y ago
    Sounds like software; trying to build on top of 40+ year old software vs building something new.
  - deleted 7y ago
    [deleted]
ChickeNES 7y ago
> 3) Lots of Apollo era designs don't exist on paper. The F1 engine doesn't really even have a standard design, since every iteration was changed to be "better" by throwing shit against the wall and seeing what sticked. I believe the F-1 specifically does have full CAD models now after a reverse engineering effort, though in general your point still stands. They attempted to restart J-2 production for Ares, but ran into so many complications that the J-2X was basically a clean-sheet design

azernik 7y ago

In addition to technology loss mentioned in other replies, reproducing Apollo designs without major redesigns is not a viable option for simple cost reasons: no NASA program in the foreseeable future will have anything near Apollo's budget, and so any new moon lander will have to be much cheaper. Given the advances in control systems, simulation, and materials I think this is very doable, but it's work.

harryh 7y ago
NASA's budget during the Apollo era peacked in 1966 around 4.5 billion. That's about 35 billion in today's dollars. That's a fair bit more than the actual NASA budget of 23.5 billion but it's not all that far off. The money is there. The challenges are mostly, as other's have noted, large changes in safety requirements and general execution challenges for large scale projects in today's America.
- NikolaeVarius 7y ago
  During Apollo a majority of that budget was dedicated to Apollo. Modern NASA maintains the ISS, multiple observatories, satellites, the DSN, and dozens of similar projects, budget alone does not paint a clear picture
  - harryh 7y ago
    Ya that's true. Apollo was 60-70% of NASA's budget for close to a decade. There's a lot more going on now. If anyone is curios about the details, there's a good summary here: https://www.nasa.gov/sites/default/files/atoms/files/fy2020_summary_budget_brief.pdf https://www.nasa.gov/sites/default/files/atoms/files/fy2020_...

WalterBright 7y ago

I have a friend who ordered the mechanical drawings for a P-51, intending to make P-51 assemblies. In investigating this, he talked to some P-51 experts who told him that the drawings were nearly useless. The real specification of the airplane was the tools and jigs created to make and assemble the parts. Those are long gone. The mechanical drawings were made after the fact to satisfy some DoD requirement.

netman21 7y ago
This. I used to work in automotive engineering. I marveled at the lack of security around drawings for mechanical components. Ford, GM, and Chrysler all used the same design shops to create blueprints. When I went to get the latest drawings they would be sitting out on a big table in the lobby. The rolled up blueprint on top of the 9"mag tape that contained the CAD files. They were clearly marked with the car's model year and the part name and number. Anyone could grab it off the table. Only later in my career did I understand that the drawing for a component, while necessary, was more of a record, not an actual representation of the final component. The final part was a product of a tool that had been crafted to produce it. Die barbers had polished a stamping tool to get a wrinkle out. Someone in manufacturing had changed the melt time for a casting. The slump temperature for a windshield glass had been adjusted. A spray pattern for body paint was manually plugged into a robot arm and adjusted until the desired results were obtained.
- jsymolon 7y ago
  chuckle, Classic case of the documentation doesn't match the "code".
pjc50 7y ago
Yes. The UK encountered a variant of this with its project to extend the life of the Nimrod recon aircraft, which were built on airframes that had essentially been designed in the 1950s. The aircraft all being individual meant that a conventional waterfall project to build new parts and retrofit them encountered huge cost overruns. Someone described it as more of an "aircraft reenactment society" than a viable aircraft for the 21st century. One caught fire in the air and the project was eventually abandoned.

ngcc_hk 7y ago

Organisation memory perhaps is a better word though not tribal / cultural memory. You can’t write everything done and for an ongoing concern you always have people around. In fact one of the problem is these Organization memory is sometimes in the way and you do not know why or even that exist. But here is about the absence, as it is not an on-going concern.

cs02rm0 7y ago
Organisation would imply to me that the knowledge was held by the organisation as a whole though. Tribal implies small groups of individuals, which is probably closer to the truth.

Cthulhu_ 7y ago

Some things will have to be rebuilt though. I read a story the other day (probably from HN as well) about rebuilding an older model rocket engine, because they still had the plans. Turns out a lot of that depended on the skill of the engineers, welders, machinists etc at the time, which could compensate for small changes on the fly. It didn't work the same with modern manufacturing technologies and materials. Mind you I do believe the modern stuff is better, they can do the same or better performance with less material and waste.