5 ms·

I was just about to write something similar - the reason nasa projects fail is because they refuse to allow for iteration. Every flight from the first flight m

by mattrp 7y ago

I was just about to write something similar - the reason nasa projects fail is because they refuse to allow for iteration. Every flight from the first flight must be flawless. Look at Spacex ... they iterate just fine for their own private fleet but the minute they start work with nasa they grind to a a slow pace.. boeing, bless their poor hearts, grew up on the nasa pork so they aren’t structured to iterate if they wanted to... like a declawed cat who has never been outdoors pretending to be fierce, they thrive on the impression that what they are doing is space exploration. And now when it’s starting to become real... oops...

SketchySeaBeast 7y ago
> the reason nasa projects fail is because they refuse to allow for iteration Could this be a relic of the era when we needed people in the spacecraft? As far as I'm aware, SpaceX has never had manned flights - a catastrophic explosion is momentary bad press, but they can continue on. Every time someone died in NASA it was a real tragedy. Plus every NASA launchpad explosion is more obviously "wasted" tax payer money.
- jtdev 7y ago
  I think NASA had unmanned test flights even in the early days.(https://en.wikipedia.org/wiki/List_of_Apollo_missions#Unmanned_test_missions https://en.wikipedia.org/wiki/List_of_Apollo_missions#Unmann...). I think everyone would agree that manned flights should be as thoroughly tested as possible. Although SpaceX has never had manned flights, they do dock with the ISS, a failure of which could have very real consequences including possible loss of life. The belief that an iterative, agile approach is too "fast and loose" for complex engineering projects has been throughly debunked, but for some reason continues to pervade.
  - bumby 7y ago
    Your comment comes across as if it's a forgone conclusion that agile development is adequate but it seems like there is still plenty of debate on this in terms of safety-critical systems. "The quality control mechanisms supported by current agile processes (e.g., informal reviews, pair-programming) have not been proven to be adequate to assure users that the product is safe. In fact there is some doubt these techniques alone will be sufficient. Formal specification, rigorous test coverage, and other formal analysis and evaluation techniques included in software engineering approaches provide better, but also more expensive, mechanisms to tackle the development of safety- or business- critical software" [1] SpaceX has shown quality control issues on their hardware processes in the past, which, from the outside looking in seems like they should have been caught.[2] I wonder if the fact that they are a relatively young organization is why these processes were lacking to begin with. It seems to be the nature of the learning process that as the 'unknown unknowns' are uncovered, processes will become more robust (and possibly cumbersome) by extension. The real question is whether they should have been unknown to begin with. I also wonder if they will start to look less like the agile upstart as they create more mature processes. [1] https://arxiv.org/ftp/arxiv/papers/1409/1409.6600.pdf https://arxiv.org/ftp/arxiv/papers/1409/1409.6600.pdf [2] https://www.wsj.com/articles/structural-failure-likely-cause-of-rocket-explosion-spacex-chief-says-1437426821 https://www.wsj.com/articles/structural-failure-likely-cause...
    - jtdev 7y ago
      Reference 1 is nearly 20 years old, much has changed in those 20 years. Reference 2 is non sequitur; nobody is suggesting that there are zero errors/failures in agile, simply that Boeing may benefit from taking a more agile approach. Here's how iteration works: https://www.youtube.com/watch?v=bvim4rsNHkQ https://www.youtube.com/watch?v=bvim4rsNHkQ
      - bumby 7y ago
        I agree that the attitude regarding agile development has changed, but the point from Ref. 1 was that the topic isn't nearly as settled yet as your comment might lead one to believe. There is still considerable debate within the aerospace industry on how and when agile development is appropriate. I would be legitimately interested if you have relevant references to successful applications of agile methodologies to safety-critical human rated space software development. Sorry if I wasn't clear enough on the point of Ref. 2. The reason that was brought up was the issue found was related to supplier quality. The fact that these checks were not in place is a bit surprising from a quality perspective and SpaceX indicated they will include additional quality oversight in the future.[1] I.e., through failures, they are adding additional processes that move them away from being an agile organization and closer towards the bureaucratic processes they are often contrasted against. "SpaceX will implement additional hardware quality audits throughout the vehicle to further ensure all parts received perform as expected per their certification documentation."[1] [1]https://www.spacex.com/news/2015/07/20/crs-7-investigation-update https://www.spacex.com/news/2015/07/20/crs-7-investigation-u...
        jtdev 7y ago
        "At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly." - The Agile Manifesto
        bumby 7y ago
        I'm familiar with the agile approach, but what you quoted isn't unique to agile; it can be found in any quality program that implements continuous improvement. It also slightly begs the question by using the agile manifesto as a reference to justify agile development. What I am hoping to find is more concrete examples of successful application of agile processes to safety-critical software. For example, how can user stories be used in lieu of a traditional software requirements specification and still effectively capture all the necessary safety requirements (including those the customer may be unaware of).
- deleted 7y ago
  [deleted]
- mattrp 7y ago
  Oh it’s definitely because of this... that’s why I think Spacex was right to pursue a path where each flight gets considerably cheaper with reuse and to prove the platform with cargo rather than pursue space tourism. Now they have a flight proven business and can work on the hard stuff. Each time a broken timer doesn’t work for spacex, as long as the rocket is recovered, they basically lose the cost of the capsule and the fuel. Every time Boeing fails, it’s what? $100 million (I don’t have a source for that)... spacex could have iterated already and have a man in space but nasa is worried about protecting Boeing’s ego that they are slowing spacex’s cadence... maybe that’s not fair... I get that nasa also is not an agile org so it needs time to absorb and analyze findings too. Regardless, at this point, I think it’s clear that nasa is propping up a failed program at Boeing and the sooner they deal with it and SLS, the sooner we can get on to actual space exploration.
  - rst 7y ago
    The cost of a new-built capsule is likely more than the cost of the rocket, for both companies. Though Boeing may not be out that money in this case -- the CST-100 is supposed to be reusable, so there's at least the prospect of relaunching the same capsule if this one lands safely (and they will try to get it back on the ground even if ISS rendezvous is no longer possible). As for SpaceX's cadence, one of those NASA-mandated tests, which was thought to be low-risk beforehand, did blow up the capsule. That sort of thing makes it a little awkward to be critical of the test requirements afterward. And while I am also very much a fan of the "cheap iterations" approach up to a point, it's less appropriate once people are on the craft and their lives are at stake.
    - mattrp 7y ago
      My understanding is once the capsules splash down that’s it for the capsule as a passenger ferry. Does Boeing get to reuse theirs? I lost track of the final resolution on the stand test but if I recall I don’t think that was intended to be low risk..they shook it until something broke if I recall. But even so wouldn’t you count that as an iteration?
      - 7y ago
LeifCarrotson 7y ago
From 1958, when McDonnell (now Boeing) built the Mercury spacecraft, through 1969, when Apollo 11 (first stage and lunar rover built by Boeing, 3rd stage by McDonnel) landed on the Moon, Boeing rapidly iterated and was an 'outdoor cat'. NASA moved pretty quickly back then, too. They've just gotten comfortable indoors, growing fat, lazy, and old in the decades since then.
- bumby 7y ago
  I know your comment is the typical explanation, but to play devil's advocate, it may just be that as organizations mature they move away from "innovation" and towards "maintenance". There's a perspective that that may not exactly be a bad thing. [1] I've heard people say SpaceX looks very much like NASA during the Apollo days, including the average age being in the mid-20s in the control rooms. Contrast that with today where NASA's average age is north of 50. What I think will be interesting is if SpaceX maintains this over the coming decades or if they will become more bureaucratic as the organization ages. http://freakonomics.com/podcast/in-praise-of-maintenance/ http://freakonomics.com/podcast/in-praise-of-maintenance/
davidmr 7y ago
> I was just about to write something similar - the reason nasa projects fail is because they refuse to allow for iteration. If that's why they fail, do they regularly succeed in spite of it or also because of it?
- mattrp 7y ago
  That’s fair criticism... can I qualify that I meant new development? Curious, what do you see as succeeding?
  - davidmr 7y ago
    I mean, they regularly build robots, strap them on huge rockets, shoot them 300,000,000 miles away to mars, land them without a scratch, and drive them around for years. That’s awesome.

by mattrp 7y ago

SketchySeaBeast 7y ago

> the reason nasa projects fail is because they refuse to allow for iteration Could this be a relic of the era when we needed people in the spacecraft? As far as I'm aware, SpaceX has never had manned flights - a catastrophic explosion is momentary bad press, but they can continue on. Every time someone died in NASA it was a real tragedy. Plus every NASA launchpad explosion is more obviously "wasted" tax payer money.

jtdev 7y ago
I think NASA had unmanned test flights even in the early days.(https://en.wikipedia.org/wiki/List_of_Apollo_missions#Unmanned_test_missions https://en.wikipedia.org/wiki/List_of_Apollo_missions#Unmann...). I think everyone would agree that manned flights should be as thoroughly tested as possible. Although SpaceX has never had manned flights, they do dock with the ISS, a failure of which could have very real consequences including possible loss of life. The belief that an iterative, agile approach is too "fast and loose" for complex engineering projects has been throughly debunked, but for some reason continues to pervade.
- bumby 7y ago
  Your comment comes across as if it's a forgone conclusion that agile development is adequate but it seems like there is still plenty of debate on this in terms of safety-critical systems. "The quality control mechanisms supported by current agile processes (e.g., informal reviews, pair-programming) have not been proven to be adequate to assure users that the product is safe. In fact there is some doubt these techniques alone will be sufficient. Formal specification, rigorous test coverage, and other formal analysis and evaluation techniques included in software engineering approaches provide better, but also more expensive, mechanisms to tackle the development of safety- or business- critical software" [1] SpaceX has shown quality control issues on their hardware processes in the past, which, from the outside looking in seems like they should have been caught.[2] I wonder if the fact that they are a relatively young organization is why these processes were lacking to begin with. It seems to be the nature of the learning process that as the 'unknown unknowns' are uncovered, processes will become more robust (and possibly cumbersome) by extension. The real question is whether they should have been unknown to begin with. I also wonder if they will start to look less like the agile upstart as they create more mature processes. [1] https://arxiv.org/ftp/arxiv/papers/1409/1409.6600.pdf https://arxiv.org/ftp/arxiv/papers/1409/1409.6600.pdf [2] https://www.wsj.com/articles/structural-failure-likely-cause-of-rocket-explosion-spacex-chief-says-1437426821 https://www.wsj.com/articles/structural-failure-likely-cause...
  - jtdev 7y ago
    Reference 1 is nearly 20 years old, much has changed in those 20 years. Reference 2 is non sequitur; nobody is suggesting that there are zero errors/failures in agile, simply that Boeing may benefit from taking a more agile approach. Here's how iteration works: https://www.youtube.com/watch?v=bvim4rsNHkQ https://www.youtube.com/watch?v=bvim4rsNHkQ
    - bumby 7y ago
      I agree that the attitude regarding agile development has changed, but the point from Ref. 1 was that the topic isn't nearly as settled yet as your comment might lead one to believe. There is still considerable debate within the aerospace industry on how and when agile development is appropriate. I would be legitimately interested if you have relevant references to successful applications of agile methodologies to safety-critical human rated space software development. Sorry if I wasn't clear enough on the point of Ref. 2. The reason that was brought up was the issue found was related to supplier quality. The fact that these checks were not in place is a bit surprising from a quality perspective and SpaceX indicated they will include additional quality oversight in the future.[1] I.e., through failures, they are adding additional processes that move them away from being an agile organization and closer towards the bureaucratic processes they are often contrasted against. "SpaceX will implement additional hardware quality audits throughout the vehicle to further ensure all parts received perform as expected per their certification documentation."[1] [1]https://www.spacex.com/news/2015/07/20/crs-7-investigation-update https://www.spacex.com/news/2015/07/20/crs-7-investigation-u...
      - jtdev 7y ago
        "At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly." - The Agile Manifesto
        bumby 7y ago
        I'm familiar with the agile approach, but what you quoted isn't unique to agile; it can be found in any quality program that implements continuous improvement. It also slightly begs the question by using the agile manifesto as a reference to justify agile development. What I am hoping to find is more concrete examples of successful application of agile processes to safety-critical software. For example, how can user stories be used in lieu of a traditional software requirements specification and still effectively capture all the necessary safety requirements (including those the customer may be unaware of).
deleted 7y ago
[deleted]
mattrp 7y ago
Oh it’s definitely because of this... that’s why I think Spacex was right to pursue a path where each flight gets considerably cheaper with reuse and to prove the platform with cargo rather than pursue space tourism. Now they have a flight proven business and can work on the hard stuff. Each time a broken timer doesn’t work for spacex, as long as the rocket is recovered, they basically lose the cost of the capsule and the fuel. Every time Boeing fails, it’s what? $100 million (I don’t have a source for that)... spacex could have iterated already and have a man in space but nasa is worried about protecting Boeing’s ego that they are slowing spacex’s cadence... maybe that’s not fair... I get that nasa also is not an agile org so it needs time to absorb and analyze findings too. Regardless, at this point, I think it’s clear that nasa is propping up a failed program at Boeing and the sooner they deal with it and SLS, the sooner we can get on to actual space exploration.
- rst 7y ago
  The cost of a new-built capsule is likely more than the cost of the rocket, for both companies. Though Boeing may not be out that money in this case -- the CST-100 is supposed to be reusable, so there's at least the prospect of relaunching the same capsule if this one lands safely (and they will try to get it back on the ground even if ISS rendezvous is no longer possible). As for SpaceX's cadence, one of those NASA-mandated tests, which was thought to be low-risk beforehand, did blow up the capsule. That sort of thing makes it a little awkward to be critical of the test requirements afterward. And while I am also very much a fan of the "cheap iterations" approach up to a point, it's less appropriate once people are on the craft and their lives are at stake.
  - mattrp 7y ago
    My understanding is once the capsules splash down that’s it for the capsule as a passenger ferry. Does Boeing get to reuse theirs? I lost track of the final resolution on the stand test but if I recall I don’t think that was intended to be low risk..they shook it until something broke if I recall. But even so wouldn’t you count that as an iteration?
    - 7y ago

LeifCarrotson 7y ago

From 1958, when McDonnell (now Boeing) built the Mercury spacecraft, through 1969, when Apollo 11 (first stage and lunar rover built by Boeing, 3rd stage by McDonnel) landed on the Moon, Boeing rapidly iterated and was an 'outdoor cat'. NASA moved pretty quickly back then, too. They've just gotten comfortable indoors, growing fat, lazy, and old in the decades since then.

bumby 7y ago
I know your comment is the typical explanation, but to play devil's advocate, it may just be that as organizations mature they move away from "innovation" and towards "maintenance". There's a perspective that that may not exactly be a bad thing. [1] I've heard people say SpaceX looks very much like NASA during the Apollo days, including the average age being in the mid-20s in the control rooms. Contrast that with today where NASA's average age is north of 50. What I think will be interesting is if SpaceX maintains this over the coming decades or if they will become more bureaucratic as the organization ages. http://freakonomics.com/podcast/in-praise-of-maintenance/ http://freakonomics.com/podcast/in-praise-of-maintenance/

davidmr 7y ago

> I was just about to write something similar - the reason nasa projects fail is because they refuse to allow for iteration. If that's why they fail, do they regularly succeed in spite of it or also because of it?

mattrp 7y ago
That’s fair criticism... can I qualify that I meant new development? Curious, what do you see as succeeding?
- davidmr 7y ago
  I mean, they regularly build robots, strap them on huge rockets, shoot them 300,000,000 miles away to mars, land them without a scratch, and drive them around for years. That’s awesome.