4 ms·

I work in HW industry, but doing SW. If I deploy a bug despite "unit" tests, then it will probably be caught by 2nd tier tests 1 day later, which will already

I work in HW industry, but doing SW. If I deploy a bug despite "unit" tests, then it will probably be caught by 2nd tier tests 1 day later, which will already be more expensive since I'll need to read report, context switch from other thing, fix it, redeploy, wait for CI, wait for 2nd tier tests, etc. If it will not be caught by 2nd tier, then it probably will be caught by validation engineers, which not only costs their time, but they also have to contact me / fill some bug, I need to get familiar with it, context switch again, fix it, redeploy, wait for tests #1 and #2 and wait for their confirmation If it will be released to actual customers then there's reputational dmg (can be huge depending on impact), possibly some lost sales / customers, probably a few weeks before it goes back to me thru customer->???->validation->me So, while 100x more expensive is some extreme case, then usually the cost is very significantly higher the later you find the bug. But what about HW bugs? I think those can be really, really expensive. Imagine catching hardware bug which affects some computation that MUST be fixed at hardware level. Catching it before release and catching it after selling 100m copies is tens of bilions of dollars difference for top GPUs and CPUs. Why do you think that those companies are willing to build same product twice so they test eachother? It significantly increases the cost, but reduces the risk. Think about CrowdStrike's incident: catching that bug at dev level would cost them shitton times less than what actually happened :P Their stock dropped by like 45%

Dylan16807 1y ago
> Think about CrowdStrike's incident: catching that bug at dev level would cost them shitton times less than what actually happened :P > Their stock dropped by like 45% A program that can block booting is definitely on the high side of software bug risk. But even in that case, the post-drop stock was still trading higher than a year before and they got back to the same level in six months.
- hobs 1y ago
  And in most corporate mindsets that's six months of growth off the table, six months of fighting just to get back to normal - that's a huge real cost and a huge opportunity cost.
  - Dylan16807 1y ago
    There was still plenty of growth to today. I doubt the stock is much lower than it would have been if that event never happened. Mild real cost, mild opportunity cost.
monksy 1y ago
> So, while 100x more expensive is some extreme case, then usually the cost is very significantly higher the later you find the bug. When you said the reputational damage, lost sales/customers.. you're already in 100x territory. It may not be exactly there but it's close or exceeding it. Yet we'll still get the undisciplined engineers whining that they have to write a small and isolated unit test.
- deleted 1y ago
  [deleted]
- cwsx 1y ago
  > Yet we'll still get the undisciplined engineers whining that they have to write a small and isolated unit test. Oh I wish that were true for my experiences - much more commonly it's a project manager that doesn't understand the value of tests....
aaron695 1y ago
[dead]
mixmastamyk 1y ago
Yes, am reminded of the Pentium FDIV bug: https://en.wikipedia.org/wiki/Pentium_FDIV_bug https://en.wikipedia.org/wiki/Pentium_FDIV_bug $475 million charge, in 1990's dollars. Every additional person who gets involved who didn't author the bug needs to research and communicate the problem. 100x sounds small after just a few layers of indirection.

Dylan16807 1y ago

> Think about CrowdStrike's incident: catching that bug at dev level would cost them shitton times less than what actually happened :P > Their stock dropped by like 45% A program that can block booting is definitely on the high side of software bug risk. But even in that case, the post-drop stock was still trading higher than a year before and they got back to the same level in six months.

hobs 1y ago
And in most corporate mindsets that's six months of growth off the table, six months of fighting just to get back to normal - that's a huge real cost and a huge opportunity cost.
- Dylan16807 1y ago
  There was still plenty of growth to today. I doubt the stock is much lower than it would have been if that event never happened. Mild real cost, mild opportunity cost.

monksy 1y ago

> So, while 100x more expensive is some extreme case, then usually the cost is very significantly higher the later you find the bug. When you said the reputational damage, lost sales/customers.. you're already in 100x territory. It may not be exactly there but it's close or exceeding it. Yet we'll still get the undisciplined engineers whining that they have to write a small and isolated unit test.

deleted 1y ago
[deleted]
cwsx 1y ago
> Yet we'll still get the undisciplined engineers whining that they have to write a small and isolated unit test. Oh I wish that were true for my experiences - much more commonly it's a project manager that doesn't understand the value of tests....

aaron695 1y ago

[dead]

mixmastamyk 1y ago

Yes, am reminded of the Pentium FDIV bug: https://en.wikipedia.org/wiki/Pentium_FDIV_bug https://en.wikipedia.org/wiki/Pentium_FDIV_bug $475 million charge, in 1990's dollars. Every additional person who gets involved who didn't author the bug needs to research and communicate the problem. 100x sounds small after just a few layers of indirection.