3 ms·

We've seen one counterexample recently: NASA Perseverance Rover. They emphasized that this was their first time of running everything in the real environment a

by euske 6y ago

We've seen one counterexample recently: NASA Perseverance Rover. They emphasized that this was their first time of running everything in the real environment although I'm 99.999984% certain that they've tested it in the closest setup as possible. I'm also certain that there must have been a few glitches here and there, but the overall system worked. There's a process of mitigating and containing the faults and failures, and NASA is known for that. I don't disagree with the principle of OP, but just repeating it in a dogmatic way doesn't make things better. I want to see practical examples and ways to fill the gap (between the dev and prod).

asd4 6y ago
Most hardware engineering is done ahead of time without a full production style environment. This is because the cost of iterating is much too high. You can't build a bridge every time you want to try a new cable or bolt. It forces designers to make models and assumptions about their systems and, inherently, puts downward pressure on complexity. It also forces them to truly understand the principles behind what they are building. The fact that Perseverance and other Mars rovers have been successful is amazing and took an incredible amount of work to accomplish. These are complicated systems that were vetted using models and simulations without ever having been run "in production". This comes at a high cost. Critical software is never tested in production or run "in system" before it is deployed. Airplanes, banks, medical systems all require extensive validation through testing on models and simulations. You can't test your changes for the first time on a live aircraft or living tissue. Costs reflect that. The truth is, a lot of software is not critical. You can get away with hacking / trial and error type development and never fully understand the system you are helping build. Frankly there is a lot of money to be made providing brand new services that are unreliable or quirky or ephemeral because those services never existed before. My point is that how you test and validate your software product depends on your application. Sometimes the costs don't make sense to "run everything" and sometimes its physically impossible. I agree that you should always advocate for the highest fidelity testing your business case can afford, but be prepared to settle for less than everything and rely on your engineering skills to buy down risks in the gaps.

by euske 6y ago

asd4 6y ago

Most hardware engineering is done ahead of time without a full production style environment. This is because the cost of iterating is much too high. You can't build a bridge every time you want to try a new cable or bolt. It forces designers to make models and assumptions about their systems and, inherently, puts downward pressure on complexity. It also forces them to truly understand the principles behind what they are building. The fact that Perseverance and other Mars rovers have been successful is amazing and took an incredible amount of work to accomplish. These are complicated systems that were vetted using models and simulations without ever having been run "in production". This comes at a high cost. Critical software is never tested in production or run "in system" before it is deployed. Airplanes, banks, medical systems all require extensive validation through testing on models and simulations. You can't test your changes for the first time on a live aircraft or living tissue. Costs reflect that. The truth is, a lot of software is not critical. You can get away with hacking / trial and error type development and never fully understand the system you are helping build. Frankly there is a lot of money to be made providing brand new services that are unreliable or quirky or ephemeral because those services never existed before. My point is that how you test and validate your software product depends on your application. Sometimes the costs don't make sense to "run everything" and sometimes its physically impossible. I agree that you should always advocate for the highest fidelity testing your business case can afford, but be prepared to settle for less than everything and rely on your engineering skills to buy down risks in the gaps.