4 ms·

While I think NASA is the most incredible organization in human history, be wary of simple underpromising-and-overdelivering: If everything they build lasts lo

While I think NASA is the most incredible organization in human history, be wary of simple underpromising-and-overdelivering: If everything they build lasts longer than expected, not only are they giving bad estimates (which perverts allocation of resources), they are way too smart and experienced to be doing it unintentionally. Now if Ford Motor underpromised on your car, telling you it would last 2 years, you wouldn't buy it in the first place - you'd never find out that it really lasts 10 years. But I suspect NASA gets away with it because everyone is blown away by even the underpromised result - helicopters on Mars, holy sh-! - and also nobody has experience with how long Martian helicopters typically fly.

tracerbulletx 3y ago
Or the estimates are based around guaranteeing a high statistical chance they hit the estimate which is perfectly fine to do.
- wolverine876 3y ago
  Yes, same thing - and I think we can assume that NASA has people smart enough to know it's the same thing.
- Buttons840 3y ago
  Yeah, I've often wished organizations I've worked for would define what an "estimate" is. Do they want an estimate I think there's a 50% chance we'll hit, or do they want an estimate I think there's a 99% chance we'll hit? In my experience they want the 50% estimate and expect us to hit that estimate 99% of the time.
bryanlarsen 3y ago
A 99% chance of lasting 5 flights is likely the same build as a 50% chance of lasting 72. OTOH there is approximately 0% chance a Ford is going to last 10 years without any maintenance or repairs.
gizmo686 3y ago
They never said the helicopter would break down after 5 flights. They said it was designed for 5 flights. Shipping it to Mars is expensive and slow, so they build in a lot of margin to increase the odds that they meet the requirements. Take a much more common example: expiration dates. A product going bad before its expiration date is an issue. Potential a massive one that could kill people. A product that remains in good condition after its expiration date is fine. As a result, expiration dates almost always occur far before the product actually expires.
- wolverine876 3y ago
  > They never said the helicopter would break down after 5 flights. They said it was designed for 5 flights. That is salesperson talk. NASA has been doing public communication half a century too, and they know what people will take from it.
  - ygjb 3y ago
    Meh. I don't think your position makes sense. Your assessment of the estimates is basically MBA talk. They designed the helicopter for 5 flights. It is very likely the case that the requirements that a rotary wing aircraft can operate for 5 flights with minimal operator intervention, and only between missions, resulted in an helicopter that had a longer lifespan. Could they have done it cheaper? Maybe, probably, but most likely with a much higher risk of failure, and given the time, distance, and cost, and the organization doing the work, I am pretty confident that the team made the right choice.
    - aurareturn 3y ago
      They designed the helicopter for at least 5 flights. It's just PR. Classic underpromising and overdelivering. NASA is a an entity that seeks public support and funding. Saying they over delivered by 16x is a good way to get the public, such as people in this thread, to think they're doing great.
      - 3y ago
sitharus 3y ago
It’s more like Ford sell you a car that’s guaranteed to have no breakdowns or need oil changes for the first two years. It might run a lot longer, but you they only promise two years so you plan to use it for that long. If it broke in those two years you’d be very upset, but if it does last longer that’s a bonus.
- mcmoor 3y ago
  I think it's more like guarantee. If it breaks down in less than 2 years, you get your money back. Still, no one buy a car to only use it for its guarantee period.
  - userbinator 3y ago
    Still, no one buy a car to only use it for its guarantee period. Unlike with software, where they call that "support" and try to scare you into their next version immediately after that ends. Unfortunately, the automakers are slowly picking up those practices from software.
  - sitharus 3y ago
    But we’re not talking about literal cars. We’re talking about on-off speciality devices designed to perform a task where there is no possibility of repair or service. They’re overbuilt to provide the best chance that the main mission is accomplished. Once that’s done they can keep going until it breaks.
duxup 3y ago
>be wary of simple underpromising-and-overdelivering Is that actually the case when it comes to the helicopter in question?
- glenstein 3y ago
  Also, why is "under promising and over delivering" a bad thing? There's a missing step in the argument where it would be necessary to explain why that is bad.
dylan604 3y ago
OR, it's design a thing to work for the mission that had been designed/approved/funded. Any science that can continue after the missions actual experiments have concluded is just bonus. Would you want the thing to be designed at any lower caliber just because it was only needing to do 5 short durations? Would it actually be any cheaper? Doubtful. So, we keep having gear designed that shows us exactly what could happen for longevity while accomplishing the actual mission goals for the same budget
EasyMark 3y ago
NASA isn't a business. And while they do have to keep budget in mind, thank fsm they aren't ruled by quarterly profits minded bean counters.
boringg 3y ago
If only the rest of the world hit their targets in the same way. The world would be much more reliable place.
jakobson14 3y ago
Ford knows how to build cars after building them for 100+ years. There's nobody who has ever tried to fly a helicopter on mars before. NASA built in lots of margin to help ensure they met their objective, especially considering that it's really fucking hard to get a new helicopter to mars if it breaks.
mturmon 3y ago
You are proposing an interesting thought experiment or hypothetical (“to what extent was the design team expecting a 10x longer lifetime?”). That’s worth pondering. Furthermore, you are answering it with a “they must have known!” I just don’t see how you can know this. (For context, I work at JPL and interact regularly with some of the Perseverance team, but nobody from the helicopter team, which was very small.) Some context. The helicopter was originally quite controversial because it had no clear science rationale (mission science requirements did not imply a need for a helicopter). My understanding is that it was put on as a tech demo and (paraphrasing) cool thing. The lab director at the time was a proponent. Of course it was a huge success. Because of its precarious status, it was not designed to fly a lot. Planning flights takes a lot of resources, coordination with the rover takes resources, etc. Bandwidth, time, and people. My point is that the “plan” was kept small partly because these ancillary costs had to be kept down. This would hold even if the hardware could last forever. The tech demo must not get in the way of the prime science mission, which had already been planned. The use of COTS components is another factor. The effects of radiation on microprocessors (in the Mars-surface or deep space environment) are not really well known, as a practical matter. There is just not much engineering experience with how bad it can get. So the longevity of the Snapdragon was (AFAIK) highly uncertain, not to mention whatever other COTS components were on it. Hope this context is helpful.
- wolverine876 3y ago
  Hey - thank you! I really appreciate hearing from someone who actually knows what they are talking about. I know nothing about the helicopter in particular but as I pointed out, NASA seems to very frequently underestimates longevity and performance (not a crime at all and maybe politically wise); I can't remember when they've overestimated. Forget my impressions in prior posts - how and why does that actually happen? > tech demo ... precarious ... not much engineering experience Also, while I've taken the opportunity of spare resources to do experiments (though not usually on Mars), I'm careful about timing and appearances - I don't want to embarass my team or partners with a failure that's visible to outsiders who wouldn't understand it was an experiment, and would think we just failed at our jobs. Everything NASA does seems to be on a very public stage, interpreted by endless outsiders (like on HN :) ). The whole world watched the helicopter's first flight. How can someone approve the helicopter, and promotion of its flight, and risk the world saying, for the next decade, 'that crazy helicopter embarassment shows we need to rein in NASA', blah blah blah? NASA's reputation is very important to humanity. Thanks for sharing what you know. You guys do amazing things; people are so jaded that they don't realize how incredible it is.
sowbug 3y ago
As the expression goes, "Any idiot can build a bridge that stands, but it takes an engineer to build a bridge that barely stands."
fsckboy 3y ago
>which perverts allocation of resources when they get the funding for the mission is when they make the lowball estimates, but with the funding they build equipment that is much more durable. It's hard to see where any perversion takes place
- glenstein 3y ago
  Exactly, that's actually getting more bang for the buck.
- wolverine876 3y ago
  Resources are allocated based on return on investment. If estimates are inaccurate, so are ROI projections. It's not just this project, but others too: How do you compare ROI of multiple projects all with inaccurate estimates?
  - fsckboy 3y ago
    sorry, no, you are ignoring the direction of the errors.

tracerbulletx 3y ago

Or the estimates are based around guaranteeing a high statistical chance they hit the estimate which is perfectly fine to do.

wolverine876 3y ago
Yes, same thing - and I think we can assume that NASA has people smart enough to know it's the same thing.
Buttons840 3y ago
Yeah, I've often wished organizations I've worked for would define what an "estimate" is. Do they want an estimate I think there's a 50% chance we'll hit, or do they want an estimate I think there's a 99% chance we'll hit? In my experience they want the 50% estimate and expect us to hit that estimate 99% of the time.

bryanlarsen 3y ago

A 99% chance of lasting 5 flights is likely the same build as a 50% chance of lasting 72. OTOH there is approximately 0% chance a Ford is going to last 10 years without any maintenance or repairs.

gizmo686 3y ago

They never said the helicopter would break down after 5 flights. They said it was designed for 5 flights. Shipping it to Mars is expensive and slow, so they build in a lot of margin to increase the odds that they meet the requirements. Take a much more common example: expiration dates. A product going bad before its expiration date is an issue. Potential a massive one that could kill people. A product that remains in good condition after its expiration date is fine. As a result, expiration dates almost always occur far before the product actually expires.

wolverine876 3y ago
> They never said the helicopter would break down after 5 flights. They said it was designed for 5 flights. That is salesperson talk. NASA has been doing public communication half a century too, and they know what people will take from it.
- ygjb 3y ago
  Meh. I don't think your position makes sense. Your assessment of the estimates is basically MBA talk. They designed the helicopter for 5 flights. It is very likely the case that the requirements that a rotary wing aircraft can operate for 5 flights with minimal operator intervention, and only between missions, resulted in an helicopter that had a longer lifespan. Could they have done it cheaper? Maybe, probably, but most likely with a much higher risk of failure, and given the time, distance, and cost, and the organization doing the work, I am pretty confident that the team made the right choice.
  - aurareturn 3y ago
    They designed the helicopter for at least 5 flights. It's just PR. Classic underpromising and overdelivering. NASA is a an entity that seeks public support and funding. Saying they over delivered by 16x is a good way to get the public, such as people in this thread, to think they're doing great.
    - 3y ago

sitharus 3y ago

It’s more like Ford sell you a car that’s guaranteed to have no breakdowns or need oil changes for the first two years. It might run a lot longer, but you they only promise two years so you plan to use it for that long. If it broke in those two years you’d be very upset, but if it does last longer that’s a bonus.

mcmoor 3y ago
I think it's more like guarantee. If it breaks down in less than 2 years, you get your money back. Still, no one buy a car to only use it for its guarantee period.
- userbinator 3y ago
  Still, no one buy a car to only use it for its guarantee period. Unlike with software, where they call that "support" and try to scare you into their next version immediately after that ends. Unfortunately, the automakers are slowly picking up those practices from software.
- sitharus 3y ago
  But we’re not talking about literal cars. We’re talking about on-off speciality devices designed to perform a task where there is no possibility of repair or service. They’re overbuilt to provide the best chance that the main mission is accomplished. Once that’s done they can keep going until it breaks.

duxup 3y ago

>be wary of simple underpromising-and-overdelivering Is that actually the case when it comes to the helicopter in question?

glenstein 3y ago
Also, why is "under promising and over delivering" a bad thing? There's a missing step in the argument where it would be necessary to explain why that is bad.

dylan604 3y ago

OR, it's design a thing to work for the mission that had been designed/approved/funded. Any science that can continue after the missions actual experiments have concluded is just bonus. Would you want the thing to be designed at any lower caliber just because it was only needing to do 5 short durations? Would it actually be any cheaper? Doubtful. So, we keep having gear designed that shows us exactly what could happen for longevity while accomplishing the actual mission goals for the same budget

EasyMark 3y ago

NASA isn't a business. And while they do have to keep budget in mind, thank fsm they aren't ruled by quarterly profits minded bean counters.

boringg 3y ago

If only the rest of the world hit their targets in the same way. The world would be much more reliable place.

jakobson14 3y ago

Ford knows how to build cars after building them for 100+ years. There's nobody who has ever tried to fly a helicopter on mars before. NASA built in lots of margin to help ensure they met their objective, especially considering that it's really fucking hard to get a new helicopter to mars if it breaks.

mturmon 3y ago

You are proposing an interesting thought experiment or hypothetical (“to what extent was the design team expecting a 10x longer lifetime?”). That’s worth pondering. Furthermore, you are answering it with a “they must have known!” I just don’t see how you can know this. (For context, I work at JPL and interact regularly with some of the Perseverance team, but nobody from the helicopter team, which was very small.) Some context. The helicopter was originally quite controversial because it had no clear science rationale (mission science requirements did not imply a need for a helicopter). My understanding is that it was put on as a tech demo and (paraphrasing) cool thing. The lab director at the time was a proponent. Of course it was a huge success. Because of its precarious status, it was not designed to fly a lot. Planning flights takes a lot of resources, coordination with the rover takes resources, etc. Bandwidth, time, and people. My point is that the “plan” was kept small partly because these ancillary costs had to be kept down. This would hold even if the hardware could last forever. The tech demo must not get in the way of the prime science mission, which had already been planned. The use of COTS components is another factor. The effects of radiation on microprocessors (in the Mars-surface or deep space environment) are not really well known, as a practical matter. There is just not much engineering experience with how bad it can get. So the longevity of the Snapdragon was (AFAIK) highly uncertain, not to mention whatever other COTS components were on it. Hope this context is helpful.

wolverine876 3y ago
Hey - thank you! I really appreciate hearing from someone who actually knows what they are talking about. I know nothing about the helicopter in particular but as I pointed out, NASA seems to very frequently underestimates longevity and performance (not a crime at all and maybe politically wise); I can't remember when they've overestimated. Forget my impressions in prior posts - how and why does that actually happen? > tech demo ... precarious ... not much engineering experience Also, while I've taken the opportunity of spare resources to do experiments (though not usually on Mars), I'm careful about timing and appearances - I don't want to embarass my team or partners with a failure that's visible to outsiders who wouldn't understand it was an experiment, and would think we just failed at our jobs. Everything NASA does seems to be on a very public stage, interpreted by endless outsiders (like on HN :) ). The whole world watched the helicopter's first flight. How can someone approve the helicopter, and promotion of its flight, and risk the world saying, for the next decade, 'that crazy helicopter embarassment shows we need to rein in NASA', blah blah blah? NASA's reputation is very important to humanity. Thanks for sharing what you know. You guys do amazing things; people are so jaded that they don't realize how incredible it is.

sowbug 3y ago

As the expression goes, "Any idiot can build a bridge that stands, but it takes an engineer to build a bridge that barely stands."

fsckboy 3y ago

>which perverts allocation of resources when they get the funding for the mission is when they make the lowball estimates, but with the funding they build equipment that is much more durable. It's hard to see where any perversion takes place

glenstein 3y ago
Exactly, that's actually getting more bang for the buck.
wolverine876 3y ago
Resources are allocated based on return on investment. If estimates are inaccurate, so are ROI projections. It's not just this project, but others too: How do you compare ROI of multiple projects all with inaccurate estimates?
- fsckboy 3y ago
  sorry, no, you are ignoring the direction of the errors.