4 ms·

in my experience academics are the worst at understanding uncertainty. They refuse to believe anything that doesn't pass their beloved t-test, believing the hol

• jamesblonde 3y ago
  You are conflating the typical threshold for drugs to pass random placebo controlled trials with all of academia. That demonstrates a lack of understanding and places you firmly in group 2.
  • caddemon 3y ago
    It's not just drug trials, it's a huge proportion of biology and psychology research at least. Additionally, most research in biology is expensive to get off the ground, and in order to get funding it absolutely is important to project confidence - there's an entire game surrounding grant writing that sounds a lot more like "group 2" than "group 1". Group 1 is just a theoretical ideal, not the reality of academic science. While that may be somewhat field dependent it's definitely not restricted to a small niche either.
  • tommiegannert 3y ago
    My wife's course exercises in statistics for health studies says "if the p-value is below 0.05, the relationship is statistically significant." Sure, it's a medical institution (Karolinska institutet in Sweden), but the programme she's in is about workplace health management, not even close to drugs.
    • jamesblonde 3y ago
      Two data points do not a generalization make. I work in engineering sciences, and we have no such threshold.
• detourdog 3y ago
  The problem I have with experts is that they are confident that they understand what someone else is articulating. One problem is that experts use their own jargon and often confuse one use of normal words with their jargon. The second step is dismissing rather than inquiring.
• advael 3y ago
  This actually strikes me as an example of a political decision creating a stickiness to academic formal standards (as others point out this has become a policy for various organizations' approval of findings). Since the "replication crisis" that happened in the last decade, a lot of people's explanations for this have centered around the use of the fisherian significance test and how gameable it can be, as well as pressures toward publishing positive results. I think there's some merit to solutions involving changing those standards and a lot of STEM people have advocated for them. However, things like drug trials have politically-determined standards that need to be upheld, and some of those standards have become well-known by the general public, so they can't really be bugfixed quickly
  • jvans 3y ago
    There is no single or set of standards that can prevent you from having to think critically about a result. Rarely are results so overwhelmingly positive that it's obvious what the result means. In those cases it doesn't matter what method you use anyway. The rest of the time, you should use many methods, be wary of their assumptions and think very hard about what the data is telling you. Realize that you are biased in favor of a positive result and play devil's advocate with yourself. This is very very hard work, and most people don't want to do it, especially when their careers improve when they look the other way
    • advael 3y ago
      I view this as obvious but I guess that doesn't go without saying. The two cultures I've characterized actually share a commonly held strong belief that there exist metrics or heuristics that can replace the need for fluid critical thinking. Obviously it's not everyone in either culture who thinks this way, but it's a dangerous form of automation for both