4 ms·

That section reads like a Bad Advice column for software testing in general. "Ehh, don't bother with end to end testing -- testing each part in isolation will b

by twh270 3y ago

That section reads like a Bad Advice column for software testing in general. "Ehh, don't bother with end to end testing -- testing each part in isolation will be fine." Maybe fine for an e-commerce site or game, where the stakes are measured mainly in lost revenue/customers and a team of engineers burning the midnight oil to sort the problem. Not so fine for spaceflight.

boringg 3y ago
When lax QA testing / engineering practices / boost the share price strategies from move fast break stuff e-commerce/social media practices seep into real world systems companies.
- Zach_the_Lizard 3y ago
  From an outsider's perspective, it seems "move fast and break things" worked well enough for SpaceX. Boeing et al seem to be following the "move slow and hide problems so we don't fix anything" mantra.
  - bumby 3y ago
    >"move fast and break things" worked well enough for SpaceX. Counterpoint: SpaceX had to re-learn well-known practices that are so commonplace in aerospace it's shocking they weren't being conducted [1]. One example is more complete material testing on critical components as part of supplier quality control. When they lost a rocket due to this process gap, their solution was to layer on those common QC practices. IMO these are risks that, over time, may turn SpaceX into the dinosaurs they are competing with. (reference to Chesterson's fence is probably apt.) "Moving fast and breaking things" may be fine, but to the GP's point, it has to be a risk-based decision. We probably shouldn't be aiming to move fast when lives are at risk. [1] https://spacenews.com/falcon-9-failure-linked-to-upper-stage-tank-strut/ https://spacenews.com/falcon-9-failure-linked-to-upper-stage...
im_down_w_otp 3y ago
You're going to be very sad to discover the sentiments around software-intensive systems testing in a lot of places & industries that you'd rightfully assume should know better and do better. I had a safety critical embedded engineer who had worked on everything from weapons systems to automobiles first not have any idea what property-based testing was and then proceed to tell me at length how it was pointless because unit testing is all you need as long as you keep making your units smaller and smaller. At the time I worked in FinTech and was aghast at what I was hearing as they worked on self-driving. If I had to pin down what made me shift my career path into "cyber-physical systems", it was probably that conversation. Since making the transition and later founding a company dedicated to testing in that domain, I've heard dozens and dozens of sentiments even more concerning.
- deleted 3y ago
  [deleted]
- JohnFen 3y ago
  > and then proceed to tell me at length how it was pointless because unit testing is all you need as long as you keep making your units smaller and smaller. I've encountered devs that think this nonsense before, but always assumed that this attitude couldn't be held with systems that are actually critical. I'm a bit less naive now.
  - shermantanktop 3y ago
    I thought that too, early on. Sort of a perfect-bricks-make-perfect-walls idea. And when problems emerge at the integration stage, it’s often traceable to a gap that a lower-level test could have caught. So the mentality is a bit self-reinforcing.
    - im_down_w_otp 3y ago
      Exactly. All the products we make are focused at the integration and system level because it's crystal clear that's where enormous amounts of issues arise and unbounded numbers of 11th hour fire drills occur. The various industries who've had multiple product cycles with lots of software are starting to figure this out, but it took awhile. Despite how obvious it is with a little outside observer perspective, the "perfect bricks" way of thinking is pervasive because that's how the builders and supply chains are organized.
deleted 3y ago
[deleted]

by twh270 3y ago

boringg 3y ago

When lax QA testing / engineering practices / boost the share price strategies from move fast break stuff e-commerce/social media practices seep into real world systems companies.

Zach_the_Lizard 3y ago
From an outsider's perspective, it seems "move fast and break things" worked well enough for SpaceX. Boeing et al seem to be following the "move slow and hide problems so we don't fix anything" mantra.
- bumby 3y ago
  >"move fast and break things" worked well enough for SpaceX. Counterpoint: SpaceX had to re-learn well-known practices that are so commonplace in aerospace it's shocking they weren't being conducted [1]. One example is more complete material testing on critical components as part of supplier quality control. When they lost a rocket due to this process gap, their solution was to layer on those common QC practices. IMO these are risks that, over time, may turn SpaceX into the dinosaurs they are competing with. (reference to Chesterson's fence is probably apt.) "Moving fast and breaking things" may be fine, but to the GP's point, it has to be a risk-based decision. We probably shouldn't be aiming to move fast when lives are at risk. [1] https://spacenews.com/falcon-9-failure-linked-to-upper-stage-tank-strut/ https://spacenews.com/falcon-9-failure-linked-to-upper-stage...

im_down_w_otp 3y ago

You're going to be very sad to discover the sentiments around software-intensive systems testing in a lot of places & industries that you'd rightfully assume should know better and do better. I had a safety critical embedded engineer who had worked on everything from weapons systems to automobiles first not have any idea what property-based testing was and then proceed to tell me at length how it was pointless because unit testing is all you need as long as you keep making your units smaller and smaller. At the time I worked in FinTech and was aghast at what I was hearing as they worked on self-driving. If I had to pin down what made me shift my career path into "cyber-physical systems", it was probably that conversation. Since making the transition and later founding a company dedicated to testing in that domain, I've heard dozens and dozens of sentiments even more concerning.

deleted 3y ago
[deleted]
JohnFen 3y ago
> and then proceed to tell me at length how it was pointless because unit testing is all you need as long as you keep making your units smaller and smaller. I've encountered devs that think this nonsense before, but always assumed that this attitude couldn't be held with systems that are actually critical. I'm a bit less naive now.
- shermantanktop 3y ago
  I thought that too, early on. Sort of a perfect-bricks-make-perfect-walls idea. And when problems emerge at the integration stage, it’s often traceable to a gap that a lower-level test could have caught. So the mentality is a bit self-reinforcing.
  - im_down_w_otp 3y ago
    Exactly. All the products we make are focused at the integration and system level because it's crystal clear that's where enormous amounts of issues arise and unbounded numbers of 11th hour fire drills occur. The various industries who've had multiple product cycles with lots of software are starting to figure this out, but it took awhile. Despite how obvious it is with a little outside observer perspective, the "perfect bricks" way of thinking is pervasive because that's how the builders and supply chains are organized.

deleted 3y ago

[deleted]