7 ms·

> British soldiers in World War I were equipped with a Brodie helmet, a steel hat designed to protect its wearer from overhead blasts and shrapnel while conduct

> British soldiers in World War I were equipped with a Brodie helmet, a steel hat designed to protect its wearer from overhead blasts and shrapnel while conducting trench warfare. After its deployment, field hospitals saw an uptick in soldiers with severe head injuries. > Because of the rise in injuries, British command considered going back to the drawing board with the helmet’s design. Fortunately, a statistician pointed out that the dramatic rise in hospital cases was because people were surviving injuries that previously would have killed them—before the introduction of steel the British Army used felt or leather as headwear material. I've seen this same story except with warplanes during the War. Story goes that an allied air force tried to improve the percentage of planes that would return from a bombing raid, so they inspected returning planes, found the places where they had holes in them and added extra layers of steel to those areas for the next bomb run; but this had no improvement on the percentage of planes that returned. That is until a "statistician" realized that all of the places on the plane that they found holes in were actually parts of the plane that could get hit and survive and return. They then started adding extra steel to the parts of the planes they found no holes or damage assuming that if those parts were hit the plane would get shot down and not return. After this change they started seeing a dramatic increase in the amount of planes returning from a raid. Does anyone know the true source of these stories? Regardless, it really opened my eyes to how changing your perspective on the cause of the problem can help find the best solution, and to this day I still think of this story when solving a problem.

mwcremer 5y ago
You are thinking of Abraham Wald: https://en.wikipedia.org/wiki/Abraham_Wald https://en.wikipedia.org/wiki/Abraham_Wald
- Sniffnoy 5y ago
  Note that the Wald story may not have actually happened: http://www.ams.org/publicoutreach/feature-column/fc-2016-06 http://www.ams.org/publicoutreach/feature-column/fc-2016-06 (But yes, Abraham Wald is the statistician that the story is about.)
  - swyx 5y ago
    yes and here's some excellent metaresearch on the "measles plane" https://counting.substack.com/p/its-that-ds-meme-plane-with-dots https://counting.substack.com/p/its-that-ds-meme-plane-with-...
  - ArnoVW 5y ago
    Aaaaaactually... in the postscript he sort of walks back everything. But since the article has mathematical and educational merit on its own, I guess he maintained it. "My indignation at how the internet dealt with Wald's work was overblown. Stephen Stigler (son of George, and a statistician at the University of Chicago) called my attention to a note by W. Allen Wallis himself in which he mentions Wald's work explicitly in connection with survivorship bias. Wallis' original article in the Journal of the American Statistical Association was followed by two very brief comments and then by a further 'rejoinder' of a bit more than one page. Towards the end of it he says, "The military was inclined to provide protection for those parts that on returning planes showed the most hits. Wald assumed, on good evidence, that hits in combat were uniformly distributed over the planes. It follows that hits on the more vulnerable parts were less likely to be found on returning planes than hits on the less vulnerable parts, since planes receiving hits on the more vulnerable parts were less likely to return to provide data. From these premises, he devised methods for estimating vulnerability of various parts." Amazing article btw. Thanks. This is what makes HN so unique.
  - deleted 5y ago
    [deleted]
- animal_spirits 5y ago
  Yes, thank you. Now I know where this came from!
telesilla 5y ago
We apply this kind of thinking at work: early on we focused all our energy fixing user problems or adding features that were requested. We realised it is actually a small number of users who volunteer to report to us what's wrong or missing (outside of regular crash/feature tracking) - they are the ones that care enough to make an effort, and it's a small percentage. So we started campaigning to engage users we never heard from, to understand what problems they had that we weren't solving, to get them energised enough to report to us. It's been very successful, engagement has gone up significantly.
- selestify 5y ago
  Did those users actually report different issues altogether from the original ones, or did they just report more of the same kinds of issues?
  - telesilla 5y ago
    Yes there is a large set of venn diagrams we weren't connecting that it gives us pretty easy access to greater coverage. The harder part becomes messaging and marketing as we target more use cases.
- mkl 5y ago
  What mechanisms did you use to get users providing feedback? Popups asking or something?
  - telesilla 5y ago
    Good old email segmenting. It's become a real powerhouse for us. Figure out what groups of people you have, talk with a few to figure out what they want, build it and target those who match. We're not a startup however so this is more of a scale up strategy or if you already have access to a large number of users who trust you and want just send your emails to spam.
- vkk8 5y ago
  This makes sense. The sort of users that leave any kind of feedback or even hang around long enough for you to gather statistics are those who think the product is pretty good, but there are just these small issues that they hope will get fixed. The users who think your product is complete crap try it, stop using it very early and never leave feedback or statistics for you to analyze.
deleted 5y ago
[deleted]
treeman79 5y ago
I always feel these stories are nonsense, they no one can be that stupid. Then I think about the people I’ve dealt with over past 20 years. And I fully believe these scenarios.
fortran77 5y ago
That plane with the red Measels spots has become a joke meme on twitter. Here's the Wikipedia page on it to start you off on your deep dive: https://en.wikipedia.org/wiki/Survivorship_bias https://en.wikipedia.org/wiki/Survivorship_bias
Aperocky 5y ago
> That is until a "statistician" realized that all of the places on the plane that they found holes in were actually parts of the plane that could get hit and survive and return. They then started adding extra steel to the parts of the planes they found no holes or damage assuming that if those parts were hit the plane would get shot down and not return. After this change they started seeing a dramatic increase in the amount of planes returning from a raid. This is highly likely to be grade A BS because it would be quite obvious to literally everyone flying an aircraft. It only make sense because the reader are not given a full minute to think about this.
carapace 5y ago
Taking the train over the Rockies they tell a story about Westinghouse[1] inverting the logic on the pneumatic brakes. When first introduced they failed open, meaning the train could move. This was convenient on the flats, but if the train was stopped on a grade and the pressure failed... down the hill you go. So Westinghouse made them fail closed: no pressure, no rolling down the hill. IIRC it saved lives. [1] https://en.wikipedia.org/wiki/George_Westinghouse https://en.wikipedia.org/wiki/George_Westinghouse
branko_d 5y ago
https://en.wikipedia.org/wiki/Survivorship_bias#In_the_military https://en.wikipedia.org/wiki/Survivorship_bias#In_the_milit...
awinter-py 5y ago
old tech blog about improving youtube page weight -- it paradoxically worsened their long tail stats once they sliced it geographically, they found that the increase was all in places like siberia where previously the site was unusable https://blog.chriszacharias.com/page-weight-matters https://blog.chriszacharias.com/page-weight-matters

mwcremer 5y ago

You are thinking of Abraham Wald: https://en.wikipedia.org/wiki/Abraham_Wald https://en.wikipedia.org/wiki/Abraham_Wald

Sniffnoy 5y ago
Note that the Wald story may not have actually happened: http://www.ams.org/publicoutreach/feature-column/fc-2016-06 http://www.ams.org/publicoutreach/feature-column/fc-2016-06 (But yes, Abraham Wald is the statistician that the story is about.)
- swyx 5y ago
  yes and here's some excellent metaresearch on the "measles plane" https://counting.substack.com/p/its-that-ds-meme-plane-with-dots https://counting.substack.com/p/its-that-ds-meme-plane-with-...
- ArnoVW 5y ago
  Aaaaaactually... in the postscript he sort of walks back everything. But since the article has mathematical and educational merit on its own, I guess he maintained it. "My indignation at how the internet dealt with Wald's work was overblown. Stephen Stigler (son of George, and a statistician at the University of Chicago) called my attention to a note by W. Allen Wallis himself in which he mentions Wald's work explicitly in connection with survivorship bias. Wallis' original article in the Journal of the American Statistical Association was followed by two very brief comments and then by a further 'rejoinder' of a bit more than one page. Towards the end of it he says, "The military was inclined to provide protection for those parts that on returning planes showed the most hits. Wald assumed, on good evidence, that hits in combat were uniformly distributed over the planes. It follows that hits on the more vulnerable parts were less likely to be found on returning planes than hits on the less vulnerable parts, since planes receiving hits on the more vulnerable parts were less likely to return to provide data. From these premises, he devised methods for estimating vulnerability of various parts." Amazing article btw. Thanks. This is what makes HN so unique.
- deleted 5y ago
  [deleted]
animal_spirits 5y ago
Yes, thank you. Now I know where this came from!

telesilla 5y ago

We apply this kind of thinking at work: early on we focused all our energy fixing user problems or adding features that were requested. We realised it is actually a small number of users who volunteer to report to us what's wrong or missing (outside of regular crash/feature tracking) - they are the ones that care enough to make an effort, and it's a small percentage. So we started campaigning to engage users we never heard from, to understand what problems they had that we weren't solving, to get them energised enough to report to us. It's been very successful, engagement has gone up significantly.

selestify 5y ago
Did those users actually report different issues altogether from the original ones, or did they just report more of the same kinds of issues?
- telesilla 5y ago
  Yes there is a large set of venn diagrams we weren't connecting that it gives us pretty easy access to greater coverage. The harder part becomes messaging and marketing as we target more use cases.
mkl 5y ago
What mechanisms did you use to get users providing feedback? Popups asking or something?
- telesilla 5y ago
  Good old email segmenting. It's become a real powerhouse for us. Figure out what groups of people you have, talk with a few to figure out what they want, build it and target those who match. We're not a startup however so this is more of a scale up strategy or if you already have access to a large number of users who trust you and want just send your emails to spam.
vkk8 5y ago
This makes sense. The sort of users that leave any kind of feedback or even hang around long enough for you to gather statistics are those who think the product is pretty good, but there are just these small issues that they hope will get fixed. The users who think your product is complete crap try it, stop using it very early and never leave feedback or statistics for you to analyze.

deleted 5y ago

[deleted]

treeman79 5y ago

I always feel these stories are nonsense, they no one can be that stupid. Then I think about the people I’ve dealt with over past 20 years. And I fully believe these scenarios.

fortran77 5y ago

That plane with the red Measels spots has become a joke meme on twitter. Here's the Wikipedia page on it to start you off on your deep dive: https://en.wikipedia.org/wiki/Survivorship_bias https://en.wikipedia.org/wiki/Survivorship_bias

Aperocky 5y ago

> That is until a "statistician" realized that all of the places on the plane that they found holes in were actually parts of the plane that could get hit and survive and return. They then started adding extra steel to the parts of the planes they found no holes or damage assuming that if those parts were hit the plane would get shot down and not return. After this change they started seeing a dramatic increase in the amount of planes returning from a raid. This is highly likely to be grade A BS because it would be quite obvious to literally everyone flying an aircraft. It only make sense because the reader are not given a full minute to think about this.

carapace 5y ago

Taking the train over the Rockies they tell a story about Westinghouse[1] inverting the logic on the pneumatic brakes. When first introduced they failed open, meaning the train could move. This was convenient on the flats, but if the train was stopped on a grade and the pressure failed... down the hill you go. So Westinghouse made them fail closed: no pressure, no rolling down the hill. IIRC it saved lives. [1] https://en.wikipedia.org/wiki/George_Westinghouse https://en.wikipedia.org/wiki/George_Westinghouse

branko_d 5y ago

https://en.wikipedia.org/wiki/Survivorship_bias#In_the_military https://en.wikipedia.org/wiki/Survivorship_bias#In_the_milit...

awinter-py 5y ago

old tech blog about improving youtube page weight -- it paradoxically worsened their long tail stats once they sliced it geographically, they found that the increase was all in places like siberia where previously the site was unusable https://blog.chriszacharias.com/page-weight-matters https://blog.chriszacharias.com/page-weight-matters