4 ms·

Great points. It's one thing to say that a scientific theory should be falsifiable (as a necessary but not sufficient condition), but the Popperians lose me whe

Great points. It's one thing to say that a scientific theory should be falsifiable (as a necessary but not sufficient condition), but the Popperians lose me when they claim either (1) that the only valid scientific study is an attempt at falsification, (2) that corroboration (or in Popperian terms, a failed attempt at falsification) has no significance. Both of these tropes come up in this "response", where Ionides et al. use them to throw some pretty serious shade at the ASA: > This approach to the logic of scientific progress, that data can serve to falsify scientific hypotheses but not to demonstrate their truth, was developed by Popper (1959) and has broad acceptance within the scientific community. In the words of Popper (1963), “It is easy to obtain confirmations, or verifications, for nearly every theory,” while, “Every genuine test of a theory is an attempt to falsify it, or to refute it. Testability is falsifiability.” The ASA’s statement appears to be contradicting the scientific method described by Einstein and Popper. However, (1) completely ignores discovery science (something I have ranted about here previously), and implies fully accepting Popper's claim to have solved the centuries-old "problem of induction" [1]. Meanwhile, (2) seems to pop up pretty frequently even though I'm not sure Popper himself would have agreed, considering [2]: > It is easy, [Popper] argues, to obtain evidence in favour of virtually any theory, and he consequently holds that such ‘corroboration’, as he terms it, should count scientifically only if it is the positive result of a genuinely ‘risky’ prediction Ionides et al. conveniently left out the second half of that context. Hmm, if only there were some sort of way to quantify that "riskiness". Like some sort of theorem... maybe of the form P(H|E) = P(E|H)*P(H)/P(E) [1] https://en.wikipedia.org/wiki/Problem_of_induction https://en.wikipedia.org/wiki/Problem_of_induction [2] https://plato.stanford.edu/entries/popper/ https://plato.stanford.edu/entries/popper/

claudiawerner 8y ago
That's a good point, I haven't actually spent a lot of time on Popper, though I was amused to find that under some interpretation of his argument, the most extreme varieties of obvious pseudoscience fall under science. In particular I wasn't aware the article even addressed Popper, in particular what's in footnote ii: >Strictly speaking any method can be made to falsify claims with the addition of a falsification rule. However, the rule must still be shown to be reliable. At the very least, I think it's shaky to try and apply the same standard regardless of the actual method of the science - since some fields take different approaches to others. That's not to say that all approaches are valid, but that one idea of science in one field shouldn't necessarily need to hold up in other one. Some sciences obviously greatly benefit (or did in the past) from even a simple statement of falsifiability, but to others it seems like a total mismatch for the problem domain. I hadn't considered Bayes' rule to work that way (my only exposure to statistics is an intro course on information theory as part of an EE degree) but if I'm right, this opens up a discussion as to how risky is risky enough - is astrology still in the picture then?
- cbkeller 8y ago
  Without having looked into it, my impression is that the popularity of astrology (and psychics, etc.) is down to the fact that they tend to make broad, vague predictions with a relatively high P(E) that people (emotionally) tend to underestimate. For instance "something bad will happen to you this month" -> "OMG, I stubbed my toe the other day, they were totally right". If we're accurate at assigning P(E), however, then we'll probably end up with a low P(H|E). The prior P(H) is also arguably trouble. What's a good prior likelihood for the hypothesis that magic sky gods affect your fortune? I don't know, but I'd probably have to start somewhere pretty low.
- roenxi 8y ago
  > I was amused to find that under some interpretation of his argument, the most extreme varieties of obvious pseudoscience fall under science You can't define science as 'the right answer' because that is unknowable (notwithstanding that there are a lot of things where if we havn't figured out the truth then the universe is set up to radically deceive us). And if you accept that unknowability as a premise, then pseudoscience is a perfectly valid science. Something can be completely wrong and still be established in a highly scientific way. Just because something is provably wrong doesn't make it unscientific. It just makes people who believe it provably wrong. They are likely to be stupid, I admit.

claudiawerner 8y ago

That's a good point, I haven't actually spent a lot of time on Popper, though I was amused to find that under some interpretation of his argument, the most extreme varieties of obvious pseudoscience fall under science. In particular I wasn't aware the article even addressed Popper, in particular what's in footnote ii: >Strictly speaking any method can be made to falsify claims with the addition of a falsification rule. However, the rule must still be shown to be reliable. At the very least, I think it's shaky to try and apply the same standard regardless of the actual method of the science - since some fields take different approaches to others. That's not to say that all approaches are valid, but that one idea of science in one field shouldn't necessarily need to hold up in other one. Some sciences obviously greatly benefit (or did in the past) from even a simple statement of falsifiability, but to others it seems like a total mismatch for the problem domain. I hadn't considered Bayes' rule to work that way (my only exposure to statistics is an intro course on information theory as part of an EE degree) but if I'm right, this opens up a discussion as to how risky is risky enough - is astrology still in the picture then?

cbkeller 8y ago
Without having looked into it, my impression is that the popularity of astrology (and psychics, etc.) is down to the fact that they tend to make broad, vague predictions with a relatively high P(E) that people (emotionally) tend to underestimate. For instance "something bad will happen to you this month" -> "OMG, I stubbed my toe the other day, they were totally right". If we're accurate at assigning P(E), however, then we'll probably end up with a low P(H|E). The prior P(H) is also arguably trouble. What's a good prior likelihood for the hypothesis that magic sky gods affect your fortune? I don't know, but I'd probably have to start somewhere pretty low.
roenxi 8y ago
> I was amused to find that under some interpretation of his argument, the most extreme varieties of obvious pseudoscience fall under science You can't define science as 'the right answer' because that is unknowable (notwithstanding that there are a lot of things where if we havn't figured out the truth then the universe is set up to radically deceive us). And if you accept that unknowability as a premise, then pseudoscience is a perfectly valid science. Something can be completely wrong and still be established in a highly scientific way. Just because something is provably wrong doesn't make it unscientific. It just makes people who believe it provably wrong. They are likely to be stupid, I admit.