3 ms·

I think part of this issue will be resolved as technology advances - eventually, it will become common practice to release all experimental data publicly, which

by keefe 16y ago

I think part of this issue will be resolved as technology advances - eventually, it will become common practice to release all experimental data publicly, which will reduce the trust requirement. After all, the heart of science is independently confirming experimental results.

MichaelSalib 16y ago
I very much doubt that this is true. Raw data by itself is almost never definitive. Meaningful conclusions often require that data be manipulated in fairly extreme ways. Each one of those manipulations provides fertile ground for denialist critique. To give you one example with which I'm familiar, several research groups have attempted to measure the number of fatalities associated with the Iraq War. One of the standard techniques for doing so involves cluster sampling, i.e., picking a small number of locations and conducting multiple interviews near each cluster site. One research group doing this noticed that the mortality rate from their Fallujah cluster was much much much higher than any of the other clusters. So they purposefully excluded it from their analysis although they kept it as part of their raw data and explained why they removed it. As a result, their total death toll was much lower than it would have been. Denialists immediately criticized them for manipulating data so as to inflate the death toll. How did they do that? Very simply: if you omit the outlier, then the spread of the probability distribution shrinks dramatically. If you include the outlier, then the width of the distribution expands so far out that it encompasses zero excess deaths. Now in either case, the distribution was screaming that there were many excess deaths, but these denialists claimed that scientists were reducing the average fatality count just so they could exclude the minute possibility that the war decreased death rates. Obviously, there is no way these scientists could win: if they include the outlier, then they're deliberately inflating the death toll, but if they exclude the outlier, then they're manipulating data to obscure the possibility that the war saved lives. If the scientists had just quietly dropped their Falujah data, this controversy never would have happened. In this case, releasing data just gave denialists fodor for crazy attacks.
- MichaelSalib 16y ago
  Downvoters: I'm curious, what is wrong with this comment?
  - keefe 16y ago
    I often wonder this myself, something seems to make people more inclined to disagree than agree.
    - ars 16y ago
      It's not people - it's person. There are a lot more readers than modders.
  - ars 16y ago
    I wouldn't downmod you for that, but man I don't agree with that comment. They are fitting data to expectations, and worse you are telling them to conceal the fact that they do so, just so no one can argue with them. That is pretty much as far away as you can possibly get from true science. What they should do is put massive error bars on the numbers and say "We can't." We can't tell you the true number, it's not possible to calculate it.
    - MichaelSalib 16y ago
      They are fitting data to expectations Eh? This is a war zone that has seen very intense fighting. It is not unreasonable to assume that data from this one site is unreliable for any of a million different reasons, especially given just how far it was from the other sites. Do you generally insist that scientists must never eliminate any outlier data points, no matter how suspect? you are telling them to conceal the fact that they do so I'm not telling them to conceal anything. I think they did the correct thing: they published all their data, and they disclosed exactly what data points they removed and why they did so. My point is that because they published their data and disclosed what they did and why they did it, they faced criticisms that never would have seen the light of day otherwise. And to me that suggests that it is unreasonable to believe that deniers are going to become less prevalent in a world where more data is published openly. What they should do is put massive error bars on the numbers and say "We can't." We can't tell you the true number, it's not possible to calculate it. I don't understand this at all. "Massive" doesn't mean anything. They calculated appropriate confidence intervals and included them in their analysis. What exactly do you think they did wrong?
tlack 16y ago
I agree that technology, in general, aides the cause of public trust in science. In some cases, such as the case of ClimateGate, the data itself -- the specific sample values -- were manipulated before their interpretations of it were even written. How do you see technology aiding in this case?

by keefe 16y ago

MichaelSalib 16y ago

I very much doubt that this is true. Raw data by itself is almost never definitive. Meaningful conclusions often require that data be manipulated in fairly extreme ways. Each one of those manipulations provides fertile ground for denialist critique. To give you one example with which I'm familiar, several research groups have attempted to measure the number of fatalities associated with the Iraq War. One of the standard techniques for doing so involves cluster sampling, i.e., picking a small number of locations and conducting multiple interviews near each cluster site. One research group doing this noticed that the mortality rate from their Fallujah cluster was much much much higher than any of the other clusters. So they purposefully excluded it from their analysis although they kept it as part of their raw data and explained why they removed it. As a result, their total death toll was much lower than it would have been. Denialists immediately criticized them for manipulating data so as to inflate the death toll. How did they do that? Very simply: if you omit the outlier, then the spread of the probability distribution shrinks dramatically. If you include the outlier, then the width of the distribution expands so far out that it encompasses zero excess deaths. Now in either case, the distribution was screaming that there were many excess deaths, but these denialists claimed that scientists were reducing the average fatality count just so they could exclude the minute possibility that the war decreased death rates. Obviously, there is no way these scientists could win: if they include the outlier, then they're deliberately inflating the death toll, but if they exclude the outlier, then they're manipulating data to obscure the possibility that the war saved lives. If the scientists had just quietly dropped their Falujah data, this controversy never would have happened. In this case, releasing data just gave denialists fodor for crazy attacks.

MichaelSalib 16y ago
Downvoters: I'm curious, what is wrong with this comment?
- keefe 16y ago
  I often wonder this myself, something seems to make people more inclined to disagree than agree.
  - ars 16y ago
    It's not people - it's person. There are a lot more readers than modders.
- ars 16y ago
  I wouldn't downmod you for that, but man I don't agree with that comment. They are fitting data to expectations, and worse you are telling them to conceal the fact that they do so, just so no one can argue with them. That is pretty much as far away as you can possibly get from true science. What they should do is put massive error bars on the numbers and say "We can't." We can't tell you the true number, it's not possible to calculate it.
  - MichaelSalib 16y ago
    They are fitting data to expectations Eh? This is a war zone that has seen very intense fighting. It is not unreasonable to assume that data from this one site is unreliable for any of a million different reasons, especially given just how far it was from the other sites. Do you generally insist that scientists must never eliminate any outlier data points, no matter how suspect? you are telling them to conceal the fact that they do so I'm not telling them to conceal anything. I think they did the correct thing: they published all their data, and they disclosed exactly what data points they removed and why they did so. My point is that because they published their data and disclosed what they did and why they did it, they faced criticisms that never would have seen the light of day otherwise. And to me that suggests that it is unreasonable to believe that deniers are going to become less prevalent in a world where more data is published openly. What they should do is put massive error bars on the numbers and say "We can't." We can't tell you the true number, it's not possible to calculate it. I don't understand this at all. "Massive" doesn't mean anything. They calculated appropriate confidence intervals and included them in their analysis. What exactly do you think they did wrong?

tlack 16y ago

I agree that technology, in general, aides the cause of public trust in science. In some cases, such as the case of ClimateGate, the data itself -- the specific sample values -- were manipulated before their interpretations of it were even written. How do you see technology aiding in this case?