3 ms·

I like the article and the proposals. I think "no evidence" is problematic and I think Bayesian reasoning is fundamental to science reporting. Maybe it doesn't

I like the article and the proposals. I think "no evidence" is problematic and I think Bayesian reasoning is fundamental to science reporting. Maybe it doesn't even go far enough though, sometimes you need domain expertise. Consider: ● Pesticide A was evaluated by the FDA and was declared safe in USA. ● Pesticide A was evaluated and banned in Europe. ● Pesticide A's molecule is similar to pesticide B, which is declared unsafe in USA. ● Of the last 10,000 pesticides to be declared safe by the USA, it actually turns out that Y% of them end up later being declared unsafe or only safer at a lesser dose. ● Pesticide A has only been evaluated by giving a 10,000x dose and looking for effects within a 1-week period. It has never been evaluated by giving a natural dose and looking for long term effects, nor effects on pregnant mothers. ● State that uses Pesticide A (but also many other products) has a 10x elevation of very rare cancers This is a very realistic picture of the type of datapoints you might start with. --- Or consider a case like this: ● Fighter pilot says he saw an alien spacecraft accelerate 100x the speed of a rocket with no noise, or visible propellant ● 50 other people say they saw strange things in incidents too ● There's good reason to think that faster-than-light-travel violates causality ● Scientists haven't found any radiowaves that indicate intelligent life anywhere they have looked I think the point is: I don't see how you get good answers without a bunch of really good bayesianists in the FDA or in Science

hnfong 3y ago
I totally agree, but even good bayesianists still can't deal with the problem that people want concrete answers, and they don't accept "it depends on your priors"... :-/
- BobaFloutist 3y ago
  Isn't it part of the job of the good Bayesianists to establish their own reasonable set of priors? It seems silly for a layman to ask a statistician if there's any there there and have the statistician say "Well that depends, what do you think?"
  - hnfong 3y ago
    The whole point of priors is that everyone can have different ones. If the priors don't affect the results too much (due to a huge amount of evidence), then you can just get the same results using frequentist methods.
- zug_zug 3y ago
  People wanting concrete answers is a good force. Authorities saying "We don't know" is also a good force. Combine these two good forces and you end up with "Let's do some more research" which is what we need more of. "Is autism actually going up in the US? We don't actually know, and instead of bayesian hedging, let's construct some studies that definitively answer that."
  - hnfong 3y ago
    In my comment I originally considered writing "people want concrete answers, now"... I totally agree that for important issues more research is always needed when one needs to resort to Bayesian analysis, but sometimes for policy reasons somebody needs to make a decision quick and they can't wait 2 more years for indisputable evidence. COVID is a good example, governments can't just say let's wait for two years and see what happens... Even for the autism example, even if authorities only find a substantial but not definite likelihood that autism is going up, they are supposed to prepare for policies that might help alleviate the problem, instead of waiting it to become a definite problem and then fixing it after the fact. For example, let's say chemical X is deemed to have 80% chance of causing increased autism, it's hard to argue we should wait until it's 99% before restricting the use of X. Of course if it turns out upon more research the likelihood is closer to 0% the policy can be reversed.

hnfong 3y ago

I totally agree, but even good bayesianists still can't deal with the problem that people want concrete answers, and they don't accept "it depends on your priors"... :-/

BobaFloutist 3y ago
Isn't it part of the job of the good Bayesianists to establish their own reasonable set of priors? It seems silly for a layman to ask a statistician if there's any there there and have the statistician say "Well that depends, what do you think?"
- hnfong 3y ago
  The whole point of priors is that everyone can have different ones. If the priors don't affect the results too much (due to a huge amount of evidence), then you can just get the same results using frequentist methods.
zug_zug 3y ago
People wanting concrete answers is a good force. Authorities saying "We don't know" is also a good force. Combine these two good forces and you end up with "Let's do some more research" which is what we need more of. "Is autism actually going up in the US? We don't actually know, and instead of bayesian hedging, let's construct some studies that definitively answer that."
- hnfong 3y ago
  In my comment I originally considered writing "people want concrete answers, now"... I totally agree that for important issues more research is always needed when one needs to resort to Bayesian analysis, but sometimes for policy reasons somebody needs to make a decision quick and they can't wait 2 more years for indisputable evidence. COVID is a good example, governments can't just say let's wait for two years and see what happens... Even for the autism example, even if authorities only find a substantial but not definite likelihood that autism is going up, they are supposed to prepare for policies that might help alleviate the problem, instead of waiting it to become a definite problem and then fixing it after the fact. For example, let's say chemical X is deemed to have 80% chance of causing increased autism, it's hard to argue we should wait until it's 99% before restricting the use of X. Of course if it turns out upon more research the likelihood is closer to 0% the policy can be reversed.