6 ms·

In grad school, I had conversations with a couple different conversations with fellow PhD students in the life sciences. The question I asked was not about p-va

In grad school, I had conversations with a couple different conversations with fellow PhD students in the life sciences. The question I asked was not about p-values / hypothesis testing but about the (lack of use) of something else technical (I don’t recall the specific question). They all responded, “we haven’t done math since our intro to calculus class and we don’t know how to code.” I feel like that is really at the heart of the problem. It is difficult to create the kind of bespoke scientific model required and use it to perform statistical analysis when you lack the mathematical and computational background. If you don’t know what you’re doing, it’s very easy to misapply existing statistical tests, let alone understand why you might care about something like the bias and variance of your estimator. It’s an understandable problem — if you are a biologist, you want to think about biology, not statistics and most biology work doesn’t require math so it appeals to the scientifically-minded that dislike math and programming. I’m curious if people here who studied those subjects think this intuition tracks, or if I am way off base. If it is part of the problem, I don’t know what the solution is beyond requiring more mathematical and computational literacy, which likely comes at the cost of discipline-specific material.

evrimoztamur 3y ago
I have a background in economics and finance with a stronger-than-average focus in statistics (compared to my peers in these fields), and I agree with you that statistical literacy (both in terms of application/interpretation but also an understanding of the mathematical foundations) are lacking in not only life sciences but really any field besides statistics. I am certain I must have done some bad-stats with the research I worked on too, because as the article also points out, it's hard to do it right and be certain you have a correct model with appropriate tests (even with so many eyes on it!) Increasing the literacy requirements is an option, but not realistically implementable. I think providing guidance and implementation support by actual practicing statisticians being involved in the data analysis sections of published research as a prerequisite is more realistic. A process in which experiments are conducted and data is collected, and the statisticians are involved in the interpretation in collaboration with the researchers is expensive in time and money, but I guess will produce much better quality analyses. Who wants to put this one to test?
- tnecniv 3y ago
  I agree that collaborating with dedicated statisticians is more feasible. I think part of the challenge might be ingraining in researchers that “stats is both hard and important so you need to take it seriously or find / hire someone who does.” If they don’t appreciate the problem, they won’t seek a better solution.
- j16sdiz 3y ago
  Have been there. They blame the statistician getting in their way the paper published.
Fomite 3y ago
So I'm an epidemiologist and/or a disease ecologist depending on what hat I'm currently wearing, so I think I count as someone who can address your question. And I'm going to give you the epidemiologists favorite answer: "It depends." Having a math and computation background certainly helps. There's a conference I go to every year that I refer to as a hypothesis generating conference - I go and listen to talks, and write down ways to address problems primarily with computational models that the presenters (mostly clinicians) think are intractable. But I've also seen people with very strong math and computation backgrounds run completely aground, either by being in love with a method more than a question, or just utterly lacking subject matter expertise. I think there's two more cultural things that are plaguing science, that don't necessarily get fixed by "take more calculus": 1) A desire for there to be rote guidelines. "In case of X, I do Y." A lot of the push recently (see the "Use confidence intervals, not p-values" or preregistration) are about making better rote guidelines, but they're still rote. As an example, the JAMA journals prohibit the use of causal language unless you're reporting an RCT. Which is mostly fine. Except, on occasion, when you get a mathematical model accepted into a JAMA journal. A model that is inherently causal, but doesn't fit "the list", and you end up writing the equivalent of "Two plus two is associated with an outcome of four." 2) Publishing. But not in the normal way people bring up publishing. There's a desire for papers to be a complete story. To be done. The experimental sciences sort of leaked into everything, so there's a notion that all studies have a beginning, middle, and end that closes the book on whatever small bit of a project you're working on, versus something closer to living progress on a question.
- tnecniv 3y ago
  Thanks! Yeah I should clarify that these friends I were talking to do the kind of bio / neuroscience where they spend most of their time in a lab running a physical experiment so their skills are focused around that kind of wet work and they interact very little with mathematical folks based on what they implied. Also, yes, in any field you have people that miss the science for the method. Sometimes that makes sense if their research is on the method and not the application, but the best papers are strong in both. On point 1, I feel like any kind of statistical cookbook that says “if x do y” will lead to the same problem. Any such recipe will not be able to capture the nuances of problems many scientists face. I’m not sure what the solution is other than some vague “we need better modeling tools” statement that is as wishful as asking people to learn stats better. On point 2, I also agree that’s an issue. In my field, the “future directions,” section is the first thing you cut down to make a page limit, but it’s really an important section to both discuss what the issues with your idea are and how they can be fixed in the future.
bsder 3y ago
The heart of the problem in life science is that "Experiments are always somewhere between too underpowered, too expensive and too unethical." Consequently, when you actually manage to run one to completion, everbody wants to squeeze every last itty bitty drop of possible juice from it. So, you will torture the data as far as you can for anything which even approaches significance. There's really no good way out of this. At the end of the day, the things you can do to humans are simply a lot more limited than the things you can do to inanimate objects.
DoctorOetker 3y ago
I think your comment is spot-on, except for one sentence: > It’s an understandable problem — if you are a biologist, you want to think about biology, not statistics and most biology work doesn’t require math so it appeals to the scientifically-minded that dislike math and programming. I think you are being polite, but what on earth does scientifically-minded mean when math is elided from it? When it comes to biology or medicine the information sources tend to be very wishy washy on average, but tend to be illuminated crystal clear in settings of systems biology (which is basically mathematical models in the context of biology), where simplifying assumptions aren't hidden in a wall of text, and the effects of ignoring terms become clear. If you pay attention you can find non-systems biology descriptions of supposedly unexplained phenomena or interactions for further investigation that are entirely explained long before in systems biology texts... > most biology work doesn’t require math The proof of the pudding is in the tasting, not the high-school experience of what a field should continue to look like in higher education.

evrimoztamur 3y ago

I have a background in economics and finance with a stronger-than-average focus in statistics (compared to my peers in these fields), and I agree with you that statistical literacy (both in terms of application/interpretation but also an understanding of the mathematical foundations) are lacking in not only life sciences but really any field besides statistics. I am certain I must have done some bad-stats with the research I worked on too, because as the article also points out, it's hard to do it right and be certain you have a correct model with appropriate tests (even with so many eyes on it!) Increasing the literacy requirements is an option, but not realistically implementable. I think providing guidance and implementation support by actual practicing statisticians being involved in the data analysis sections of published research as a prerequisite is more realistic. A process in which experiments are conducted and data is collected, and the statisticians are involved in the interpretation in collaboration with the researchers is expensive in time and money, but I guess will produce much better quality analyses. Who wants to put this one to test?

tnecniv 3y ago
I agree that collaborating with dedicated statisticians is more feasible. I think part of the challenge might be ingraining in researchers that “stats is both hard and important so you need to take it seriously or find / hire someone who does.” If they don’t appreciate the problem, they won’t seek a better solution.
j16sdiz 3y ago
Have been there. They blame the statistician getting in their way the paper published.

Fomite 3y ago

So I'm an epidemiologist and/or a disease ecologist depending on what hat I'm currently wearing, so I think I count as someone who can address your question. And I'm going to give you the epidemiologists favorite answer: "It depends." Having a math and computation background certainly helps. There's a conference I go to every year that I refer to as a hypothesis generating conference - I go and listen to talks, and write down ways to address problems primarily with computational models that the presenters (mostly clinicians) think are intractable. But I've also seen people with very strong math and computation backgrounds run completely aground, either by being in love with a method more than a question, or just utterly lacking subject matter expertise. I think there's two more cultural things that are plaguing science, that don't necessarily get fixed by "take more calculus": 1) A desire for there to be rote guidelines. "In case of X, I do Y." A lot of the push recently (see the "Use confidence intervals, not p-values" or preregistration) are about making better rote guidelines, but they're still rote. As an example, the JAMA journals prohibit the use of causal language unless you're reporting an RCT. Which is mostly fine. Except, on occasion, when you get a mathematical model accepted into a JAMA journal. A model that is inherently causal, but doesn't fit "the list", and you end up writing the equivalent of "Two plus two is associated with an outcome of four." 2) Publishing. But not in the normal way people bring up publishing. There's a desire for papers to be a complete story. To be done. The experimental sciences sort of leaked into everything, so there's a notion that all studies have a beginning, middle, and end that closes the book on whatever small bit of a project you're working on, versus something closer to living progress on a question.

tnecniv 3y ago
Thanks! Yeah I should clarify that these friends I were talking to do the kind of bio / neuroscience where they spend most of their time in a lab running a physical experiment so their skills are focused around that kind of wet work and they interact very little with mathematical folks based on what they implied. Also, yes, in any field you have people that miss the science for the method. Sometimes that makes sense if their research is on the method and not the application, but the best papers are strong in both. On point 1, I feel like any kind of statistical cookbook that says “if x do y” will lead to the same problem. Any such recipe will not be able to capture the nuances of problems many scientists face. I’m not sure what the solution is other than some vague “we need better modeling tools” statement that is as wishful as asking people to learn stats better. On point 2, I also agree that’s an issue. In my field, the “future directions,” section is the first thing you cut down to make a page limit, but it’s really an important section to both discuss what the issues with your idea are and how they can be fixed in the future.

bsder 3y ago

The heart of the problem in life science is that "Experiments are always somewhere between too underpowered, too expensive and too unethical." Consequently, when you actually manage to run one to completion, everbody wants to squeeze every last itty bitty drop of possible juice from it. So, you will torture the data as far as you can for anything which even approaches significance. There's really no good way out of this. At the end of the day, the things you can do to humans are simply a lot more limited than the things you can do to inanimate objects.

DoctorOetker 3y ago

I think your comment is spot-on, except for one sentence: > It’s an understandable problem — if you are a biologist, you want to think about biology, not statistics and most biology work doesn’t require math so it appeals to the scientifically-minded that dislike math and programming. I think you are being polite, but what on earth does scientifically-minded mean when math is elided from it? When it comes to biology or medicine the information sources tend to be very wishy washy on average, but tend to be illuminated crystal clear in settings of systems biology (which is basically mathematical models in the context of biology), where simplifying assumptions aren't hidden in a wall of text, and the effects of ignoring terms become clear. If you pay attention you can find non-systems biology descriptions of supposedly unexplained phenomena or interactions for further investigation that are entirely explained long before in systems biology texts... > most biology work doesn’t require math The proof of the pudding is in the tasting, not the high-school experience of what a field should continue to look like in higher education.