3 ms·

The study can be found here: https://journals.sagepub.com/doi/10.1177/19401612241311886 https://journals.sagepub.com/doi/10.1177/19401612241311886

by croon 2y ago

mike_hearn 2y ago
Let's take a look then. Firstly, is it replicable? Many academic papers aren't because their methodology either isn't specified, or the methodology isn't derived logically from the hypothesis. Here they are relying on Wikipedia and "Media Bias/Fact Check" to decide if a source is factual, and then decide anything published by any such source is wrong, and then that anyone who refers to any such article is also wrong. This is replicable at some surface level - you could use the same sources and same tweets to compute the same numbers, but lacks any form of deeper replicability because the choice of these two meta-sources doesn't follow obviously from the hypothesis and they don't specify how they chose them (beyond citing other papers that made the same choice). So an obvious criticism is how do you know these meta-sources aren't junk? They thought of this, and say: > To ensure the robustness of our methods, the factuality measure was manually validated to ensure that the articles shared in fact contain misinformation (see Supplemental Information File). We selected a stratified random sample of 50 articles from each of the five predefined factuality levels, totaling 250 articles. These were manually analyzed ... The results are reported in Supplemental Tables S6 and S7, But if we check the Supplementary Tables then we find no such validation. Ironically, this study itself contains misinformation. The articles they checked aren't listed, nor is how they decided what was or was not true. The tables are a masterclass in circular logic, comparing how accurate articles are against their "factuality group" and discovering that, shock, the academics who chose the sources agree that they are good sources. Nothing is presented that can actually be refined or refuted, because this entire "manual validation" boils down to circular reasoning of the form "the sources we chose are accurate because we checked and say they are accurate". It's just a restatement of the rest of the paper, dressed up with some tables to look sciencey. In the penultimate paragraph they do inch towards an actual evaluation, but it just shows how pseudo-scientific this paper is. They admit that in many cases sources they put in a low factuality group do report accurate news, but justify the decision by saying (about RT): > Although each individual article may be factually correct in its reporting on individual battles, the overall story is thus a false narrative of a highly successful ‘special military operation.’ This underscores the limitations of assessing factuality solely based on the accuracy of individual articles, offering further support for the validity of relying on media outlet-level analyses. This is a Kafka trap. If you report accurate stories you are nonetheless inaccurate because you don't report the things we think you should, and therefore anyone who cites you for anything at all, ever, is "spreading misinformation" regardless of topic or correctness of the statement made. The only type of source anyone is allowed to use is one that promotes the academics preferred slants, regardless of the accuracy of the information published. There are tens of thousands of studies like this coming out of academia every year that are little more than Guardian-bait. Zero of them are scientific, yet all are presented as if they are. When people call them out on this behavior, they're accused by the left of being "anti science". Where are the intellectual standards?
- lowmagnet 2y ago
  > Let's take a look then. Firstly, is it replicable? It's a survey of public statements by politicians, are you under the false belief that every white paper is tied to a novel experiment?
  - mike_hearn 2y ago
    It's a paper by people who get paid by the government because they claim to be doing political science: that is, studying politics and society in a neutral, scientific way to obtain generalizable conclusions. They present what they claim is a novel finding, in fact they say that "little is known of the relationship between party politics and the spread of misinformation" and that their paper addresses this problem. In this case one of the authors is funded by my local government yet they claim at the bottom of their article they received no funding to do this work, so that's yet more misinformation and it's my money being used for this, which is pretty irritating. If they wanted to write a personal politics blog there's Substack. They aren't: they're pretending to be scientists, like in tens of thousands of other papers out there on the same topic.

The study can be found here: https://journals.sagepub.com/doi/10.1177/19401612241311886 https://journals.sagepub.com/doi/10.1177/19401612241311886

by croon 2y ago

The study can be found here: https://journals.sagepub.com/doi/10.1177/19401612241311886 https://journals.sagepub.com/doi/10.1177/19401612241311886

mike_hearn 2y ago

Let's take a look then. Firstly, is it replicable? Many academic papers aren't because their methodology either isn't specified, or the methodology isn't derived logically from the hypothesis. Here they are relying on Wikipedia and "Media Bias/Fact Check" to decide if a source is factual, and then decide anything published by any such source is wrong, and then that anyone who refers to any such article is also wrong. This is replicable at some surface level - you could use the same sources and same tweets to compute the same numbers, but lacks any form of deeper replicability because the choice of these two meta-sources doesn't follow obviously from the hypothesis and they don't specify how they chose them (beyond citing other papers that made the same choice). So an obvious criticism is how do you know these meta-sources aren't junk? They thought of this, and say: > To ensure the robustness of our methods, the factuality measure was manually validated to ensure that the articles shared in fact contain misinformation (see Supplemental Information File). We selected a stratified random sample of 50 articles from each of the five predefined factuality levels, totaling 250 articles. These were manually analyzed ... The results are reported in Supplemental Tables S6 and S7, But if we check the Supplementary Tables then we find no such validation. Ironically, this study itself contains misinformation. The articles they checked aren't listed, nor is how they decided what was or was not true. The tables are a masterclass in circular logic, comparing how accurate articles are against their "factuality group" and discovering that, shock, the academics who chose the sources agree that they are good sources. Nothing is presented that can actually be refined or refuted, because this entire "manual validation" boils down to circular reasoning of the form "the sources we chose are accurate because we checked and say they are accurate". It's just a restatement of the rest of the paper, dressed up with some tables to look sciencey. In the penultimate paragraph they do inch towards an actual evaluation, but it just shows how pseudo-scientific this paper is. They admit that in many cases sources they put in a low factuality group do report accurate news, but justify the decision by saying (about RT): > Although each individual article may be factually correct in its reporting on individual battles, the overall story is thus a false narrative of a highly successful ‘special military operation.’ This underscores the limitations of assessing factuality solely based on the accuracy of individual articles, offering further support for the validity of relying on media outlet-level analyses. This is a Kafka trap. If you report accurate stories you are nonetheless inaccurate because you don't report the things we think you should, and therefore anyone who cites you for anything at all, ever, is "spreading misinformation" regardless of topic or correctness of the statement made. The only type of source anyone is allowed to use is one that promotes the academics preferred slants, regardless of the accuracy of the information published. There are tens of thousands of studies like this coming out of academia every year that are little more than Guardian-bait. Zero of them are scientific, yet all are presented as if they are. When people call them out on this behavior, they're accused by the left of being "anti science". Where are the intellectual standards?

lowmagnet 2y ago
> Let's take a look then. Firstly, is it replicable? It's a survey of public statements by politicians, are you under the false belief that every white paper is tied to a novel experiment?
- mike_hearn 2y ago
  It's a paper by people who get paid by the government because they claim to be doing political science: that is, studying politics and society in a neutral, scientific way to obtain generalizable conclusions. They present what they claim is a novel finding, in fact they say that "little is known of the relationship between party politics and the spread of misinformation" and that their paper addresses this problem. In this case one of the authors is funded by my local government yet they claim at the bottom of their article they received no funding to do this work, so that's yet more misinformation and it's my money being used for this, which is pretty irritating. If they wanted to write a personal politics blog there's Substack. They aren't: they're pretending to be scientists, like in tens of thousands of other papers out there on the same topic.