7 ms·

From a process design/management view I'm not sure you're right. In fact I think you're probably wrong, but there is definitely room for disagreement. To overs

by gpm 7y ago

From a process design/management view I'm not sure you're right. In fact I think you're probably wrong, but there is definitely room for disagreement. To oversimplify things massively, "how careful do we want to be to avoid testing failures" is a parameter that SpaceX management gets to control. At the extreme careful end everything costs billions of dollars because you're paralyzed analyzing things and double/triple/quadruple/quintuple checking things. At the not at all careful end of things you keep building "complete rockets", pointing them at mars without any testing, and blowing them up. Obviously neither is rational. They've settled on some parameters that are basically "don't worry that much about test failures, but keep the failures really cheap". They seem to be doing pretty well by doing that. They're experimenting with manufacturing techniques, hiring and training a workforce, building facilities. The prototypes that they are building keep failing, but that looks to be a relatively minor cost considering that the current primary goal is (according to them) to build out the manufacturing processes and make the design easy to manufacture. Being slightly more careful would undoubtedly reduce the numbers of prototype failures, but would it actually be worth the cost of slowing other things down? Remember that the main cost of this program to SpaceX is engineering salaries, the faster it goes the cheaper it is. So of course this test didn't go as planned, of course it would be better if it had worked, but would it have really been worth it to management to reduce the probability of this test failing? Like I said, maybe they aren't being careful enough, but I don't think we have any real evidence for that right now and I personally doubt it.

bumby 7y ago
I think this is where NASA went wrong in many respects. They got so risk adverse, they built a bureaucracy of checks and oversight that moves slow. That’s one of thew reasons the industry needs fresh blood like SpaceX
- mcbutterbunz 7y ago
  Two reasons and probably more: 1. The risk is different when there are human lives on the line. 2. NASA is publicly funded and if they want to keep receiving funding, they can’t be blowing up the public’s money.
  - ClumsyPilot 7y ago
    I feel NASA's approach is right for manned spaceflight, but is not sensible for robotic missions - metal is cheap, design time is expensive. I feel the public appearance is not a hard - nosed calculation, but Is instead a pr requirement
    - greglindahl 7y ago
      NASA has different risk categories for manned spaceflight and robotic missions, depending on cost.
    - sbuttgereit 7y ago
      I actually disagree with you. For one, consider that Starship is ultimately intended for manned spaceflight... yet SpaceX is still trying things and blowing stuff up (or imploding or...) From my armchair perspective I think the risk adversity criticism is justified. NASA should take much more risk even in human spaceflight... in the early stages of platform development. First see if you can build the damn thing and build it repeatedly. Later, after that period, reduce risk and prove out and refine things to the human spaceflight standard. If you jump immediately to, "my God people are suppose to fly on that thing!" well, you're probably going to pay the Russians a lot of money for a fairly long time. I do agree with you that the political management becomes a real issue. Politicians and bureaucrats are naturally disposed to a certain cravenness about such issues once they get to significant positions of power (senior execs of many companies do, too). But I think a charismatic leader that's out in front of the messaging could pull it off.
      - kiba 7y ago
        If they actually test and launch the damn thing, we would be further along with a safer rocket and don't have to pay the Russians.
      - ClumsyPilot 7y ago
        Well, ironically and counter to your point, Russians had the same basic design now for nearly 60 years!
- kiba 7y ago
  They got really risk adverse and their vehicles aren't that reliable and safe at the end anyway.
- akiselev 7y ago
  A JPL project manager teaching a course I took blamed it on the public's "mathematical illiteracy." He explained that there are diminishing returns on mission success rates so NASA often had the choice of paying X per mission attempt each with an 80% success rate or 2X for a single launch with 95% success rate. 80% sounds bad but the chance of two launches failing is 4%, which means a 96% chance overall success rate. If the first launch succeeds, they can usually save between 20-40% of the total budget (the remaining amount is overhead to build backups and keep scientists and engineers employed in case the first launch fails). However, since NASA is no longer doing world shattering stuff like landing on the moon, the human echo chamber amplifies each failure and drowns out the successes. Since NASA has to justify many appropriations on a per project basis, failures lead to projects getting canceled and NASA doing less actual science, even though their budget rarely shrinks - with more and more money instead allocated to inefficiencies like maintaining unused real estate/hardware and spreading out manufacturing across all districts in order to curry favor to get more projects going. That's just for robotic/satellite missions. It's ten times worse when a manned mission ends fatally.
  - jeffwass 7y ago
    > since NASA is no longer doing world shattering stuff like landing on the moon NASA has its Artemis program which is doing exactly this, returning to the Moon with a goal of a longer-term lunar base. And they just held a period of application (it closed a few days ago) for the next generation of Artemis astronauts. Artemis : https://www.nasa.gov/specials/artemis/ https://www.nasa.gov/specials/artemis/
    - nickik 7y ago
      Yeah they said they will return to the Moon or Mars (depending on the president) since Bush 1. That is almost 40 years. So another new moon plan. Another Shuttle derived launcher. Not surprising that nobody is getting very exited with Artemis. The SLS and Orion can only be described as dead-weights on NASA since they are so incredibly expensive. SLS has been in development for over 10 years at more then 2 billion per year and it will not launch until 2022 hopefully.
    - 7y ago
deleted 7y ago
[deleted]
Ari_Ugwu 7y ago
This does feel like Musk's software background shining through: * Fail Fast. * Red. Green. Refactor.
- kiba 7y ago
  This is more of his automative background shining through. That's where the whole machine building the machine idea comes from.
beamatronic 7y ago
The value of knowledge they obtain from a failure far outweighs the “cheap tests” as you put it.
pavelrub 7y ago
It's unclear to me what you are basing this on. How do you know that the "how careful do we want to be to avoid testing failures" parameter was exactly where SpaceX wanted it to be in this experiment? Or was at an optimal point for SpaceX in general? If you don't know this - then your reply doesn't really mean anything, and can be be used to justify any mistake whatsoever.
- gpm 7y ago
  I definitely don't know that the parameter is where SpaceX wants it to be, or even more importantly where SpaceX should want it to be given the information available to SpaceX. That's why there is room for disagreement. I do have some hints that it's probably more or less where SpaceX wants it to be, some hints that they have chosen this parameter rationally, and some faith that they aren't stupid. For instance, Elon often discusses future tests failing as acceptable, they've declined to put in place more stringent QA efforts, they're still mostly building the hardware exposed to the elements, Elon has discussed that currently their approach is building a lot of starships to iterate on the manufacturing methods and to help them make an easy to manufacture design - he has signaled fairly clearly via his tweets that actually flying the "ships" is secondary, and so on. My prior is that the parameter is as likely to be set as too high as too low. The testing failure at this stage in development is weak evidence that it is too low, the rest of the information I've just talked about is more convincing evidence that it is an appropriate value or too high, hence I think the post I responded to is "probably wrong". An example of where I think the level of caution was probably too low (or there were other issues, but let's pretend for a second that this is the only knob management has, as I said it is an over simplification) - the recent CST 100 orbital flight test failure. That test was intended to demonstrate they were ready to fly humans, instead they suffered a series of serious failures, failing that sort of test is pretty strong evidence that they weren't careful enough. They've indicated this type of failure was unexpected and that it will cost the company slightly under half a billion dollars, which is also pretty strong evidence that they weren't careful enough. Moreover I know that they neglected to use procedures such as end-to-end software simulation that would have reduced this risk, and that they were in fact contractually required to perform. Meanwhile I don't have any real evidence that they were careful enough with regards to that test. So starting with the same prior as above, I quickly fall to the "they probably weren't careful enough side". > Or was at an optimal point for SpaceX in general The optimal point is definitely not a global value for all of SpaceX. Human spaceflight programs have obviously set it very high instead of very low. As they progress from experimenting with manufacturing to flying payloads it obviously gets turned up. Etc.

by gpm 7y ago

bumby 7y ago

I think this is where NASA went wrong in many respects. They got so risk adverse, they built a bureaucracy of checks and oversight that moves slow. That’s one of thew reasons the industry needs fresh blood like SpaceX

mcbutterbunz 7y ago
Two reasons and probably more: 1. The risk is different when there are human lives on the line. 2. NASA is publicly funded and if they want to keep receiving funding, they can’t be blowing up the public’s money.
- ClumsyPilot 7y ago
  I feel NASA's approach is right for manned spaceflight, but is not sensible for robotic missions - metal is cheap, design time is expensive. I feel the public appearance is not a hard - nosed calculation, but Is instead a pr requirement
  - greglindahl 7y ago
    NASA has different risk categories for manned spaceflight and robotic missions, depending on cost.
  - sbuttgereit 7y ago
    I actually disagree with you. For one, consider that Starship is ultimately intended for manned spaceflight... yet SpaceX is still trying things and blowing stuff up (or imploding or...) From my armchair perspective I think the risk adversity criticism is justified. NASA should take much more risk even in human spaceflight... in the early stages of platform development. First see if you can build the damn thing and build it repeatedly. Later, after that period, reduce risk and prove out and refine things to the human spaceflight standard. If you jump immediately to, "my God people are suppose to fly on that thing!" well, you're probably going to pay the Russians a lot of money for a fairly long time. I do agree with you that the political management becomes a real issue. Politicians and bureaucrats are naturally disposed to a certain cravenness about such issues once they get to significant positions of power (senior execs of many companies do, too). But I think a charismatic leader that's out in front of the messaging could pull it off.
    - kiba 7y ago
      If they actually test and launch the damn thing, we would be further along with a safer rocket and don't have to pay the Russians.
    - ClumsyPilot 7y ago
      Well, ironically and counter to your point, Russians had the same basic design now for nearly 60 years!
kiba 7y ago
They got really risk adverse and their vehicles aren't that reliable and safe at the end anyway.
akiselev 7y ago
A JPL project manager teaching a course I took blamed it on the public's "mathematical illiteracy." He explained that there are diminishing returns on mission success rates so NASA often had the choice of paying X per mission attempt each with an 80% success rate or 2X for a single launch with 95% success rate. 80% sounds bad but the chance of two launches failing is 4%, which means a 96% chance overall success rate. If the first launch succeeds, they can usually save between 20-40% of the total budget (the remaining amount is overhead to build backups and keep scientists and engineers employed in case the first launch fails). However, since NASA is no longer doing world shattering stuff like landing on the moon, the human echo chamber amplifies each failure and drowns out the successes. Since NASA has to justify many appropriations on a per project basis, failures lead to projects getting canceled and NASA doing less actual science, even though their budget rarely shrinks - with more and more money instead allocated to inefficiencies like maintaining unused real estate/hardware and spreading out manufacturing across all districts in order to curry favor to get more projects going. That's just for robotic/satellite missions. It's ten times worse when a manned mission ends fatally.
- jeffwass 7y ago
  > since NASA is no longer doing world shattering stuff like landing on the moon NASA has its Artemis program which is doing exactly this, returning to the Moon with a goal of a longer-term lunar base. And they just held a period of application (it closed a few days ago) for the next generation of Artemis astronauts. Artemis : https://www.nasa.gov/specials/artemis/ https://www.nasa.gov/specials/artemis/
  - nickik 7y ago
    Yeah they said they will return to the Moon or Mars (depending on the president) since Bush 1. That is almost 40 years. So another new moon plan. Another Shuttle derived launcher. Not surprising that nobody is getting very exited with Artemis. The SLS and Orion can only be described as dead-weights on NASA since they are so incredibly expensive. SLS has been in development for over 10 years at more then 2 billion per year and it will not launch until 2022 hopefully.
  - 7y ago

deleted 7y ago

[deleted]

Ari_Ugwu 7y ago

This does feel like Musk's software background shining through: * Fail Fast. * Red. Green. Refactor.

kiba 7y ago
This is more of his automative background shining through. That's where the whole machine building the machine idea comes from.

beamatronic 7y ago

The value of knowledge they obtain from a failure far outweighs the “cheap tests” as you put it.

pavelrub 7y ago

It's unclear to me what you are basing this on. How do you know that the "how careful do we want to be to avoid testing failures" parameter was exactly where SpaceX wanted it to be in this experiment? Or was at an optimal point for SpaceX in general? If you don't know this - then your reply doesn't really mean anything, and can be be used to justify any mistake whatsoever.

gpm 7y ago
I definitely don't know that the parameter is where SpaceX wants it to be, or even more importantly where SpaceX should want it to be given the information available to SpaceX. That's why there is room for disagreement. I do have some hints that it's probably more or less where SpaceX wants it to be, some hints that they have chosen this parameter rationally, and some faith that they aren't stupid. For instance, Elon often discusses future tests failing as acceptable, they've declined to put in place more stringent QA efforts, they're still mostly building the hardware exposed to the elements, Elon has discussed that currently their approach is building a lot of starships to iterate on the manufacturing methods and to help them make an easy to manufacture design - he has signaled fairly clearly via his tweets that actually flying the "ships" is secondary, and so on. My prior is that the parameter is as likely to be set as too high as too low. The testing failure at this stage in development is weak evidence that it is too low, the rest of the information I've just talked about is more convincing evidence that it is an appropriate value or too high, hence I think the post I responded to is "probably wrong". An example of where I think the level of caution was probably too low (or there were other issues, but let's pretend for a second that this is the only knob management has, as I said it is an over simplification) - the recent CST 100 orbital flight test failure. That test was intended to demonstrate they were ready to fly humans, instead they suffered a series of serious failures, failing that sort of test is pretty strong evidence that they weren't careful enough. They've indicated this type of failure was unexpected and that it will cost the company slightly under half a billion dollars, which is also pretty strong evidence that they weren't careful enough. Moreover I know that they neglected to use procedures such as end-to-end software simulation that would have reduced this risk, and that they were in fact contractually required to perform. Meanwhile I don't have any real evidence that they were careful enough with regards to that test. So starting with the same prior as above, I quickly fall to the "they probably weren't careful enough side". > Or was at an optimal point for SpaceX in general The optimal point is definitely not a global value for all of SpaceX. Human spaceflight programs have obviously set it very high instead of very low. As they progress from experimenting with manufacturing to flying payloads it obviously gets turned up. Etc.