4 ms·

It reads like most of your issue is with my phrasing, which was deliberate to soften the comment so it doesn’t come across as accusatory. >I’m not even sure ho

by bumby 2y ago

It reads like most of your issue is with my phrasing, which was deliberate to soften the comment so it doesn’t come across as accusatory. >I’m not even sure how you would answer the question. It should probably be answered in the same way we’d answer the converse. Something like “here is the data and the tests that show the data quality is high and the results are replicable.” My point was, if you can’t show that, then the claims that the work is good may not be scientifically valid. To put a finer point on it, we know there is replicability crisis in many corners of science. But we don’t likely know how much of that is due to bad science or outright fraud unless we do the science-y work to check. And unfortunately many of the incentives are aligned against that very thing.

matthewdgreen 2y ago
I have not claimed that all scientific work is good or valid. The post I responded to said: “given the shameless plague of fraudulent papers, and the gaming of citations, my faith in experts and scientists has been damaged.” I picked a bone with that given the total lack of evidence for the claim. More generally I have a problem with a broader class of people who make claims like this, since I find these claims unlikely and unjustified. You came back to me with a different question: “how do we know that all science is good?” I genuinely don’t know how to answer that question. And bluntly, I think it would be incredibly hard to answer it: the question is up there with “how many grains of sand are there on the world’s beaches.” I do think that clearly science, in whatever state it is in, has historically worked amazingly well. The world we live in is a testament to that. Maybe the answer is that it doesn’t matter, that ecience has always been mediocre and yet we go to space and cure diseases with even low-quality science. A more general answer is that everyone doing science is working incredibly hard to find the answers to hard questions. We are always doing so imperfectly, with imperfect methods and imperfect tools and imperfect brains and yet we soldier on. If you want to join this effort, either to further it or bring evidence to criticize and improve it, we are all excited to have you. But simply posing impossible-to-answer questions is not necessarily doing either thing.
- bumby 2y ago
  >more general answer is that everyone doing science is working incredibly hard to find the answers to hard questions. I think this is too romantic of a view. Yes, some (or even many) are working hard to answer hard questions. Others are seeking money, status and/or prestige as their top priority, just like practically any other field. It’s no different than saying financiers are working incredibly hard to create a rising tide that lifts all boats. It makes for a lofty narrative, but it’s also ignoring much of human nature. And if you think that question is impossible to answer, it should at the very least give you pause to think about the confidence you have in your conclusion. Otherwise it reads as “I can’t prove it, but I know it’s true.” That’s dogma, not science.
  - matthewdgreen 2y ago
    In this conversation we’ve traveled trom “scientific fraud is rampant” to “you must somehow prove to me that all science is good” to “maybe scientists are human beings”. I think the first claim is made without evidence and I called the poster on it; the second is a bizarre goalpost-shift that I don’t think we have the resources to ever answer. But the third question is easy for me to answer: sure, scientists are human beings, with all their flaws. But so what? What’s most interesting about the conversation we’re having is that even as folks make and fall back from specific arguments, the directional nature of the discussion doesn’t shift. All this tells me is that you don’t like scientists and are eager to land a punch against them, even if the punch is as weak as “they’re human.” This kind of conversation doesn’t have much value to me, since I can’t learn anything from it except that you have some strong emotions. But maybe it could be valuable to you?
    - bumby 2y ago
      >All this tells me is that you don’t like scientists I think you’re extrapolating too much. Again, you’re making a strong claim based on limited evidence which is the hallmark of bad reasoning. I work in research and publish/referee articles regularly. I have no qualms with scientists, but I’m also acutely aware of how the system is flawed. >I think the first claim is made without evidence I get the impression you’re too riled to actually see my point so I’ll be explicit. We have lots of evidence that bad science is pretty widespread, especially in certain disciplines. We also have anecdotal examples of fraud. The latter is a subset of the former, with the remainder being good work. How big of a subset is it? Your claim is that it's small; my claim is we don't know. There’s no goalpost shifting. All of the points made are related to the issue of fraud. Asking for evidence of your claim is not goalpost shifting, nor is understanding that fraud is a consequence of human incentives for status that exist in practically every domain. I simply made the point that we know there’s a problem, but likely don’t know the extent of it. Meanwhile, you made the claim that academic fraud is rare. And yet you also claim it’s impossible to prove that it’s not rare. It’s pretty clear that you’re not making a reasoned argument, which is the irony of someone defending the current state of science.

by bumby 2y ago

matthewdgreen 2y ago

I have not claimed that all scientific work is good or valid. The post I responded to said: “given the shameless plague of fraudulent papers, and the gaming of citations, my faith in experts and scientists has been damaged.” I picked a bone with that given the total lack of evidence for the claim. More generally I have a problem with a broader class of people who make claims like this, since I find these claims unlikely and unjustified. You came back to me with a different question: “how do we know that all science is good?” I genuinely don’t know how to answer that question. And bluntly, I think it would be incredibly hard to answer it: the question is up there with “how many grains of sand are there on the world’s beaches.” I do think that clearly science, in whatever state it is in, has historically worked amazingly well. The world we live in is a testament to that. Maybe the answer is that it doesn’t matter, that ecience has always been mediocre and yet we go to space and cure diseases with even low-quality science. A more general answer is that everyone doing science is working incredibly hard to find the answers to hard questions. We are always doing so imperfectly, with imperfect methods and imperfect tools and imperfect brains and yet we soldier on. If you want to join this effort, either to further it or bring evidence to criticize and improve it, we are all excited to have you. But simply posing impossible-to-answer questions is not necessarily doing either thing.

bumby 2y ago
>more general answer is that everyone doing science is working incredibly hard to find the answers to hard questions. I think this is too romantic of a view. Yes, some (or even many) are working hard to answer hard questions. Others are seeking money, status and/or prestige as their top priority, just like practically any other field. It’s no different than saying financiers are working incredibly hard to create a rising tide that lifts all boats. It makes for a lofty narrative, but it’s also ignoring much of human nature. And if you think that question is impossible to answer, it should at the very least give you pause to think about the confidence you have in your conclusion. Otherwise it reads as “I can’t prove it, but I know it’s true.” That’s dogma, not science.
- matthewdgreen 2y ago
  In this conversation we’ve traveled trom “scientific fraud is rampant” to “you must somehow prove to me that all science is good” to “maybe scientists are human beings”. I think the first claim is made without evidence and I called the poster on it; the second is a bizarre goalpost-shift that I don’t think we have the resources to ever answer. But the third question is easy for me to answer: sure, scientists are human beings, with all their flaws. But so what? What’s most interesting about the conversation we’re having is that even as folks make and fall back from specific arguments, the directional nature of the discussion doesn’t shift. All this tells me is that you don’t like scientists and are eager to land a punch against them, even if the punch is as weak as “they’re human.” This kind of conversation doesn’t have much value to me, since I can’t learn anything from it except that you have some strong emotions. But maybe it could be valuable to you?
  - bumby 2y ago
    >All this tells me is that you don’t like scientists I think you’re extrapolating too much. Again, you’re making a strong claim based on limited evidence which is the hallmark of bad reasoning. I work in research and publish/referee articles regularly. I have no qualms with scientists, but I’m also acutely aware of how the system is flawed. >I think the first claim is made without evidence I get the impression you’re too riled to actually see my point so I’ll be explicit. We have lots of evidence that bad science is pretty widespread, especially in certain disciplines. We also have anecdotal examples of fraud. The latter is a subset of the former, with the remainder being good work. How big of a subset is it? Your claim is that it's small; my claim is we don't know. There’s no goalpost shifting. All of the points made are related to the issue of fraud. Asking for evidence of your claim is not goalpost shifting, nor is understanding that fraud is a consequence of human incentives for status that exist in practically every domain. I simply made the point that we know there’s a problem, but likely don’t know the extent of it. Meanwhile, you made the claim that academic fraud is rare. And yet you also claim it’s impossible to prove that it’s not rare. It’s pretty clear that you’re not making a reasoned argument, which is the irony of someone defending the current state of science.