4 ms·

The very experiences that Feynman recounts have lead to a practice of doing "blind" experiments in particle physics and other places. You simply do your analys

The very experiences that Feynman recounts have lead to a practice of doing "blind" experiments in particle physics and other places. You simply do your analysis with some extra, unknown, factor added and once you think you done the best you can, the blinding is removed and you check to see what you measured.

foldr 7y ago
Sure, but who knows if this has been beneficial to scientific progress or not? It's not like we can wind back time and do a comparative experiment. edit To expand on this a bit, as it might sound flippant. People often talk about "bias" as if it's something that obviously ought to be eliminated. But in fact you can't get anywhere without biases. We only have the time and resources to explore a tiny portion of the hypothesis space in most domains.
- SiempreViernes 7y ago
  Now you are basically saying "what if overtraining is good?"
  - foldr 7y ago
    Overtraining is bad by definition, like overcooking. But "don't overtrain" is about as useful a maxim as "don't overcook". Naive bayes has higher bias than logistic regression. Is that a good or a bad thing? Depends. (I don't actually see any analogy with overtraining.)
- tempguy9999 7y ago
  > People often talk about "bias" as if it's something that obviously ought to be eliminated Yeah, bias means our model of reality is distorted; one doesn't correspond to the other as well as it could. An example of bias, from https://en.wikipedia.org/wiki/Space_Shuttle_Challenger_disaster#Richard_Feynman https://en.wikipedia.org/wiki/Space_Shuttle_Challenger_disas... (appropriately enough) <<<In the appendix, he argued that the estimates of reliability offered by NASA management were wildly unrealistic, differing as much as a thousandfold from the estimates of working engineers. "For a successful technology," he concluded, "reality must take precedence over public relations, for nature cannot be fooled." >>> That is bias, and it killed people and didn't do the US any favours. If by bias you mean something else, your post needed to be clearer.
  - foldr 7y ago
    I think that's an odd definition of bias. Bias is just a prior inclination to believe P rather than not P. P may or may not be true. In any case, that's the kind of bias that's relevant to this discussion.
    - tempguy9999 7y ago
      Okay, decent answer, upvoted. I think what I said still applies, if P = "shuttle is safe 9,999 launches in 10,000" then not(P) turns out to be true. I think that's bias by both our definitions.
AlexCoventry 7y ago
That's really interesting. How are the blinding factors constructed and applied?

foldr 7y ago

Sure, but who knows if this has been beneficial to scientific progress or not? It's not like we can wind back time and do a comparative experiment. edit To expand on this a bit, as it might sound flippant. People often talk about "bias" as if it's something that obviously ought to be eliminated. But in fact you can't get anywhere without biases. We only have the time and resources to explore a tiny portion of the hypothesis space in most domains.

SiempreViernes 7y ago
Now you are basically saying "what if overtraining is good?"
- foldr 7y ago
  Overtraining is bad by definition, like overcooking. But "don't overtrain" is about as useful a maxim as "don't overcook". Naive bayes has higher bias than logistic regression. Is that a good or a bad thing? Depends. (I don't actually see any analogy with overtraining.)
tempguy9999 7y ago
> People often talk about "bias" as if it's something that obviously ought to be eliminated Yeah, bias means our model of reality is distorted; one doesn't correspond to the other as well as it could. An example of bias, from https://en.wikipedia.org/wiki/Space_Shuttle_Challenger_disaster#Richard_Feynman https://en.wikipedia.org/wiki/Space_Shuttle_Challenger_disas... (appropriately enough) <<<In the appendix, he argued that the estimates of reliability offered by NASA management were wildly unrealistic, differing as much as a thousandfold from the estimates of working engineers. "For a successful technology," he concluded, "reality must take precedence over public relations, for nature cannot be fooled." >>> That is bias, and it killed people and didn't do the US any favours. If by bias you mean something else, your post needed to be clearer.
- foldr 7y ago
  I think that's an odd definition of bias. Bias is just a prior inclination to believe P rather than not P. P may or may not be true. In any case, that's the kind of bias that's relevant to this discussion.
  - tempguy9999 7y ago
    Okay, decent answer, upvoted. I think what I said still applies, if P = "shuttle is safe 9,999 launches in 10,000" then not(P) turns out to be true. I think that's bias by both our definitions.

AlexCoventry 7y ago

That's really interesting. How are the blinding factors constructed and applied?