5 ms·

Seaborn is a visualization library. No statistical tests should have been done with seaborn as an intermediate processing step. I guess they used some of the co

Seaborn is a visualization library. No statistical tests should have been done with seaborn as an intermediate processing step. I guess they used some of the convenience functions as part of the data analysis. Seaborn is a final step tool, not a data analysis tool. That's an embarrassing lesson to learn post-publication.

sitkack 3y ago
Take a look at the linked chart in my other comment. Visualization is absolutely a driver during research, it isn’t just an embarrassing revelation. Charts killed the Challenger crew.
- ayhanfuat 3y ago
  It can be a driver. But then you do a deep dive. You create frequency tables, you create crosstabs, you calculate summary statistics, you do inferential statistics. There is no excuse for not catching this pre-publication.
  - sitkack 3y ago
    That is not the point I was making.
- sillysaurusx 3y ago
  > Charts killed the Challenger crew. https://www.tiktok.com/t/ZT8oG7ym6/ https://www.tiktok.com/t/ZT8oG7ym6/ This is one of my favorite TikToks of all time, and you’ll see why. It goes into detail about how charts killed the Challenger crew. But the storytelling is second to none.
  - mannykannot 3y ago
    I composed this comment concurrently with yours, so I'm moving it here as a response... This is going off-topic, but Tufte's attempt to cast the problem as fundamentally one of poor data presentation is rather self-servingly tendentious, IMHO, in a way that unfairly attributes a degree of culpability to the engineers who tried to stop the launch. The excellent video you link to, taken as a whole, supports this view, I believe.
    - sillysaurusx 3y ago
      That’s an interesting point (and quite on topic; communication is key in STEM, and this ties in with that). Hypothetically, what would be the most fair argument in that situation? It’s quite remarkable that a line engineer was convinced the rocket was going to explode, even to the extent of hopping in his car with his daughter and frying to stop the launch after his company gave the go-ahead. Data presentation seems like one of the few things that could have convinced upper management that there was a serious problem. One thing I don’t understand (possibly unrelated to your point): if there were very few launches in cold temperature in general, how could he have convinced himself that there was going to be a disaster due to the weather? If I were in his shoes, I might’ve talked myself out of it by saying "well, I suppose it’s true we don’t have much data about cold temperature launches; how certain am I that the cold weather problems till now weren’t a fluke or a non-issue?"
      - raziel2701 3y ago
        He knew how O-rings behaved, they fail to make a seal at those low temperatures. You don't need to have multiple failed launches to reach the conclusion he reached.
        sillysaurusx 3y ago
        You do, though. Because although there’s a chance of catastrophic failure, the typical case is that the launch goes fine, and then they notice some unexpected degradation of the O-ring after the shuttle comes back. That’s how they were measuring chance of O-ring difficulties; a shuttle had never exploded, but there had been observable signs that things could have gone badly. In other words, in most mechanical systems, a certain amount of wear and tear is acceptable. It’s only at extremes (way too cold) that it becomes a disaster. Convincing yourself that you’re certain there will be a disaster this time is a level of scientific and engineering confidence that’s hard to fathom. In that situation I would have done everything possible to alert management that there was a high chance of an issue, but would I have grabbed my daughter and drove to stop the launch because I was 100% certain it would blow? Probably not.
      - 3y ago
- deleted 3y ago
  [deleted]
Aloisius 3y ago
The bulk of the problem was caused by erroneous metadata. The bug in Seaborn simply meant that the histograms that could have alerted them that something was wrong with their analysis, didn't.

sitkack 3y ago

Take a look at the linked chart in my other comment. Visualization is absolutely a driver during research, it isn’t just an embarrassing revelation. Charts killed the Challenger crew.

ayhanfuat 3y ago
It can be a driver. But then you do a deep dive. You create frequency tables, you create crosstabs, you calculate summary statistics, you do inferential statistics. There is no excuse for not catching this pre-publication.
- sitkack 3y ago
  That is not the point I was making.
sillysaurusx 3y ago
> Charts killed the Challenger crew. https://www.tiktok.com/t/ZT8oG7ym6/ https://www.tiktok.com/t/ZT8oG7ym6/ This is one of my favorite TikToks of all time, and you’ll see why. It goes into detail about how charts killed the Challenger crew. But the storytelling is second to none.
- mannykannot 3y ago
  I composed this comment concurrently with yours, so I'm moving it here as a response... This is going off-topic, but Tufte's attempt to cast the problem as fundamentally one of poor data presentation is rather self-servingly tendentious, IMHO, in a way that unfairly attributes a degree of culpability to the engineers who tried to stop the launch. The excellent video you link to, taken as a whole, supports this view, I believe.
  - sillysaurusx 3y ago
    That’s an interesting point (and quite on topic; communication is key in STEM, and this ties in with that). Hypothetically, what would be the most fair argument in that situation? It’s quite remarkable that a line engineer was convinced the rocket was going to explode, even to the extent of hopping in his car with his daughter and frying to stop the launch after his company gave the go-ahead. Data presentation seems like one of the few things that could have convinced upper management that there was a serious problem. One thing I don’t understand (possibly unrelated to your point): if there were very few launches in cold temperature in general, how could he have convinced himself that there was going to be a disaster due to the weather? If I were in his shoes, I might’ve talked myself out of it by saying "well, I suppose it’s true we don’t have much data about cold temperature launches; how certain am I that the cold weather problems till now weren’t a fluke or a non-issue?"
    - raziel2701 3y ago
      He knew how O-rings behaved, they fail to make a seal at those low temperatures. You don't need to have multiple failed launches to reach the conclusion he reached.
      - sillysaurusx 3y ago
        You do, though. Because although there’s a chance of catastrophic failure, the typical case is that the launch goes fine, and then they notice some unexpected degradation of the O-ring after the shuttle comes back. That’s how they were measuring chance of O-ring difficulties; a shuttle had never exploded, but there had been observable signs that things could have gone badly. In other words, in most mechanical systems, a certain amount of wear and tear is acceptable. It’s only at extremes (way too cold) that it becomes a disaster. Convincing yourself that you’re certain there will be a disaster this time is a level of scientific and engineering confidence that’s hard to fathom. In that situation I would have done everything possible to alert management that there was a high chance of an issue, but would I have grabbed my daughter and drove to stop the launch because I was 100% certain it would blow? Probably not.
    - 3y ago
deleted 3y ago
[deleted]

Aloisius 3y ago

The bulk of the problem was caused by erroneous metadata. The bug in Seaborn simply meant that the histograms that could have alerted them that something was wrong with their analysis, didn't.