9 ms·

We're increasingly aware today of how the media operates cycles of self-referential and self-justifying citations: a TV show will quote an article that reports

We're increasingly aware today of how the media operates cycles of self-referential and self-justifying citations: a TV show will quote an article that reports "some people" taking an issue, which ends up being a quote from someone interviewed for another newspaper article.. and so on. This "legitimacy laundering" is rampant, and we're now getting towards media literacy levels which expose it for many people. However, most concerning: this is how academia has always worked. It's the great absurdity of peer review, and of citation. This is how entire fields can sustain themselves with little or no scientific validity (esp. see, psychometrics). We are no where near the equivalent "academic literacy" for generally informed members of the public to understand this problem. Entire fields can be sustained with zero "empirical pressure" to close down. So long as one can cite another who can cite another... and somewhere some government body will take these citations as prima facie evidence of a research programme, then funding will be given and more papers published.

littlestymaar 2y ago
Economics works exactly like this, but at a fractal level: there are many different subfields living in the grand family of economists which barely communicate with one another, often have dramatically different theories that lead to opposing conclusion, none of which even trying to give an accurate description of the world but instead promote one particular ideological bias. Every once is a while someone from one chapel is going to actually do science by refuting one other chapel's bullshit, but it's almost never seen as a good thing by anyone, and the Nobel committee sometimes ends up giving the price together to people from different chapels even when their work contradict each other (Fama being empirically being proven wrong by Schiller and both being given the price in 2013 to appease the feud). Economic behaviors are so complex that studying economics seriously requires gigantic efforts for little results, and people who attempt to do that stand no chance of survival in the publish-or-perish competition against the grandiose bullshitters, and that's how we get where we are.
- light_hue_1 2y ago
  This happens everywhere. Let me tell you a story about who studies language and how. Linguistics studies language right? It has a few hundred year history of doing so seriously. But once AI people wanted to also have a look linguists became unhappy. They didn't like the idea of someone else saying something about their problem with a different set of tools. Those tools don't provide explanations that they approve of. Even the thought of measuring things over a dataset to check if your linguistic theories work made them bristle with Chomsky famously rejecting the idea of evidence at scale. So AI people created natural language processing. Their own field to study the same problem. One in a while someone tried to do something to bridge the gap which led to the famous "every time I fire a linguist my performance goes up". Many people in NLP see linguists as outdated fossils. Most would probably vote to just shut down linguistics departments I suspect. Then something funny happened. The LLM people did to NLP what they did to linguistics. And the NLP community rejected them. Because LLMs don't provide an explanation that they approve of. Recently just like in linguistics the President of ACL, the main NLP organization, said recently that NLP is not about machine learning! The absurdity of that statement just boggles the mind. So the LLM people now have their own conference and field. A lot of people cannot accept that things must change over time. They want to do research the way they did it 10 or 40 years ago. So they create an immune response. It's human nature. As someone who doesn't feel this way and enjoys moving forward, I'm really sad about what's happened to NLP in the past few years. It would be nice if we could stop this rift. But it won't happen with the current hate filled President. I stopped sending any work there until they reform. Just the cycle of academic life. I wonder if science would move faster if we could select for people who are willing to change with the times.
  - mjburgess 2y ago
    The goal of physics isn't to produce video games -- that a video game can generate a frame using unphysical formulae and unphysical processes (eg., rasterisation) does not invalidate physics. There's linguistics which produces descriptive theories of the practice of language. There's "computational linguistics" which aims to build discrete algorithmic models of language. Both of these are interested in explanations of actual physical phenomena. Machine Learning is largely an engineering discipline which, like video games, can generate output using any method which will convince a human user. Indeed, often more radically, ML works by replaying variations of past convincing output rather than have any explicit simulation involved at all. Thus there's no sense in which LLMs supercede linguistics, nor somehow computational linguistics superseding this. This would be like claiming rasterisation means we should retire optics and the physics of light.
pacman001 2y ago
Every single time (some 4 times in my past life) when I’m familiar with/close to the background story of something that played out in the news I come to the same conclusion: news doesn’t report the facts. I’m in Western Europe, btw. I advise to run this experiment yourself.
- littlestymaar 2y ago
  That's the problem when you have people responsible for news having the business model of ad-sposored entertainment and not fact reporting.
  - pacman001 2y ago
    The conclusion I’m coming to is that this, too, is an exponent of an inherent weakness: we like to be told stories. As children we like to be told stories and this remains. We now have a complete spectrum of story-telling: science fiction, fiction, non-fiction, …, and news. News reporting itself is a continuum, with tabloid gossip on one side and political coverage on the other. We have so much stories to choose from.
  - zorked 2y ago
    In Western Europe a lot of the media isn't ad-sponsored.
    - aeim 2y ago
      Just "bad-sponsored"
    - littlestymaar 2y ago
      I live in France, and saying that 90% of the media are ad-sponsored is a conservative estimate. So I don't know which part of western Europe you're talking about, but it clearly doesn't apply to all of it.
  - deleted 2y ago
    [deleted]
- 2y ago
voidhorse 2y ago
Totally, and I think there is a fundamental, deeper, inherent problem in using statistics to determine how you want to manipulate an given object of study. Researches find that, on average, consumers want X. Companies decide they want to maximize reach and so begin to produce X. Consumers soon have little choice except for X, reaffirming, to future researchers, that consumers want X. This is why I think a plurality of data is so necessary especially when it comes to anything in the social domain, and why it's imperative that we begin to invest more into cross-disciplinary research. Specialization has gotten us far, but it's starting to lead to breakdowns. The only thing that might offset the statistical observation of consumer behavior is a statistical study of consumer opinion that proves to refute or contradict that behavior... but then in response some people will say "people don't know what they want" further reaffirming the conclusions that the action based on the data itself caused (behavior or observation bias). The application of statistics to social problems essentially becomes a self-fulfilling prophecy.
- mjburgess 2y ago
  I fully agree. How many IQ effects in studied populations are actually created by the use of IQ tests? Consider how infiltrated IQ-like assessment in throughout society, selecting for doctors, lawyers, postgrads -- military, police, etc. Then consider what data is offered as evidence that IQ 1) exists, and 2) causes observable measures in real-world outcomes. Filtering on the test becomes evidence the test is a measure of anything. The application of "statistics" I dislike the most is where these feedback cycles exist, and large swathes of academia have some extreme responsibility here. Whole fields of gene-traits studies were created over decade+ and then disappeared overnight as actual sequencing took place. All the rigour and splendour of "statistics", and then poof when science was done, it disappeared. Since there are basically no scientific theories of human psychology, society, and the like -- gluing together correlations here should be seen as prima facie absurd. The alternative? Rely on expertise, and build resilience-to-failure into the system and tolerance for higher variability. Human expertise obtained in domain-specific environments is vastly superior to the species correlations of surveys written by idiots who've never done any actual science.
InDubioProRubio 2y ago
Could one create a proof of pseudo-science, by injecting a faked fundamental corner stone paper, that becomes proof by inheritance that a full field is rotten? Also why does this remind me of european politicans, claiming everyone wants to life european lifes, meanwhile whole countries goto war and atrocities without big counter-demonstrations by those western valued citizens .. narrative glider guns going ad absurdum..
- kenjackson 2y ago
  > Could one create a proof of pseudo-science, by injecting a faked fundamental corner stone paper, that becomes proof by inheritance that a full field is rotten? I don't think there's any doubt that pseudoscience exists, even amongst the most optimistic of scientists. The problem is identifying what is bad science vs what is good. The fact that I can send this from a small phone from a parking lot is proof that someone did good science at some point in time. Or that I've seen therapies like CBT turn someone who was struggling mentally on a daily basis to thrive, or that I've seen valve replacement surgery give someone years of great life after a "six months to live diagnosis" -- all show that there is good science. I think we almost need a "discipline" of people who validate scientific results, and people should be held accountable for results that validate or don't. Peer review is great. It's not a farce (I've gotten some incredible feedback on papers from it), but it is also extremely limited by design. We need more.
- jjk166 2y ago
  Even a broken clock is right twice a day. I can publish a paper with results I think are wrong, but it's entirely possible that follow up studies confirm it was accidentally correct. While this is improbable in general, if we restrict ourselves to publishing claims that are sufficiently plausible that people in the field would accept them as a corner stone paper, then there is a very decent chance it sounds plausible because it's true. Even if I test the claim myself before publishing to confirm it's false, I may have made an error. Possibly repeatedly publishing bogus papers in a certain manner might be able to confidently weed out poor academic hygeine, but it's not a trivial thing.
brabel 2y ago
Great video about that by Kurzgesagt: https://www.youtube.com/watch?v=bgo7rm5Maqg&ab_channel=Kurzgesagt%E2%80%93InaNutshell https://www.youtube.com/watch?v=bgo7rm5Maqg&ab_channel=Kurzg... They show how deep they had to go to find the original source of the claim that a single human's blood vessels, if lined up, would stretch 100,000km... and how that was quoted by so many that no one really know where the claim came from. And of course, it was wrong.
- naasking 2y ago
  Damn it, I had just posted this video too. Off to delete it.
deleted 2y ago
[deleted]
renewiltord 2y ago
This is the danger with non-empirical government funded “science”. Empiricism allows verification. Private funding means that things that matter get studied. If you trace the oft-cited claim from “studies” that cats kill so and so animals a year you will find it’s just someone’s Fermi estimate. And you’ll find that the person has a personal distaste for housecats.
- rscho 2y ago
  The idea that private funding means that what is studied matters is highly dubious at best. Most private research pertains to application and product development, and often relies on fundamental discoveries coming from academic labs. Only academia can afford to let scientists run loose. It certainly results in a good amount of inapplicable theories, but is nevertheless very much essential.
  - renewiltord 2y ago
    I doubt gender studies is essential.

littlestymaar 2y ago

Economics works exactly like this, but at a fractal level: there are many different subfields living in the grand family of economists which barely communicate with one another, often have dramatically different theories that lead to opposing conclusion, none of which even trying to give an accurate description of the world but instead promote one particular ideological bias. Every once is a while someone from one chapel is going to actually do science by refuting one other chapel's bullshit, but it's almost never seen as a good thing by anyone, and the Nobel committee sometimes ends up giving the price together to people from different chapels even when their work contradict each other (Fama being empirically being proven wrong by Schiller and both being given the price in 2013 to appease the feud). Economic behaviors are so complex that studying economics seriously requires gigantic efforts for little results, and people who attempt to do that stand no chance of survival in the publish-or-perish competition against the grandiose bullshitters, and that's how we get where we are.

light_hue_1 2y ago
This happens everywhere. Let me tell you a story about who studies language and how. Linguistics studies language right? It has a few hundred year history of doing so seriously. But once AI people wanted to also have a look linguists became unhappy. They didn't like the idea of someone else saying something about their problem with a different set of tools. Those tools don't provide explanations that they approve of. Even the thought of measuring things over a dataset to check if your linguistic theories work made them bristle with Chomsky famously rejecting the idea of evidence at scale. So AI people created natural language processing. Their own field to study the same problem. One in a while someone tried to do something to bridge the gap which led to the famous "every time I fire a linguist my performance goes up". Many people in NLP see linguists as outdated fossils. Most would probably vote to just shut down linguistics departments I suspect. Then something funny happened. The LLM people did to NLP what they did to linguistics. And the NLP community rejected them. Because LLMs don't provide an explanation that they approve of. Recently just like in linguistics the President of ACL, the main NLP organization, said recently that NLP is not about machine learning! The absurdity of that statement just boggles the mind. So the LLM people now have their own conference and field. A lot of people cannot accept that things must change over time. They want to do research the way they did it 10 or 40 years ago. So they create an immune response. It's human nature. As someone who doesn't feel this way and enjoys moving forward, I'm really sad about what's happened to NLP in the past few years. It would be nice if we could stop this rift. But it won't happen with the current hate filled President. I stopped sending any work there until they reform. Just the cycle of academic life. I wonder if science would move faster if we could select for people who are willing to change with the times.
- mjburgess 2y ago
  The goal of physics isn't to produce video games -- that a video game can generate a frame using unphysical formulae and unphysical processes (eg., rasterisation) does not invalidate physics. There's linguistics which produces descriptive theories of the practice of language. There's "computational linguistics" which aims to build discrete algorithmic models of language. Both of these are interested in explanations of actual physical phenomena. Machine Learning is largely an engineering discipline which, like video games, can generate output using any method which will convince a human user. Indeed, often more radically, ML works by replaying variations of past convincing output rather than have any explicit simulation involved at all. Thus there's no sense in which LLMs supercede linguistics, nor somehow computational linguistics superseding this. This would be like claiming rasterisation means we should retire optics and the physics of light.

pacman001 2y ago

Every single time (some 4 times in my past life) when I’m familiar with/close to the background story of something that played out in the news I come to the same conclusion: news doesn’t report the facts. I’m in Western Europe, btw. I advise to run this experiment yourself.

littlestymaar 2y ago
That's the problem when you have people responsible for news having the business model of ad-sposored entertainment and not fact reporting.
- pacman001 2y ago
  The conclusion I’m coming to is that this, too, is an exponent of an inherent weakness: we like to be told stories. As children we like to be told stories and this remains. We now have a complete spectrum of story-telling: science fiction, fiction, non-fiction, …, and news. News reporting itself is a continuum, with tabloid gossip on one side and political coverage on the other. We have so much stories to choose from.
- zorked 2y ago
  In Western Europe a lot of the media isn't ad-sponsored.
  - aeim 2y ago
    Just "bad-sponsored"
  - littlestymaar 2y ago
    I live in France, and saying that 90% of the media are ad-sponsored is a conservative estimate. So I don't know which part of western Europe you're talking about, but it clearly doesn't apply to all of it.
- deleted 2y ago
  [deleted]
2y ago

voidhorse 2y ago

Totally, and I think there is a fundamental, deeper, inherent problem in using statistics to determine how you want to manipulate an given object of study. Researches find that, on average, consumers want X. Companies decide they want to maximize reach and so begin to produce X. Consumers soon have little choice except for X, reaffirming, to future researchers, that consumers want X. This is why I think a plurality of data is so necessary especially when it comes to anything in the social domain, and why it's imperative that we begin to invest more into cross-disciplinary research. Specialization has gotten us far, but it's starting to lead to breakdowns. The only thing that might offset the statistical observation of consumer behavior is a statistical study of consumer opinion that proves to refute or contradict that behavior... but then in response some people will say "people don't know what they want" further reaffirming the conclusions that the action based on the data itself caused (behavior or observation bias). The application of statistics to social problems essentially becomes a self-fulfilling prophecy.

mjburgess 2y ago
I fully agree. How many IQ effects in studied populations are actually created by the use of IQ tests? Consider how infiltrated IQ-like assessment in throughout society, selecting for doctors, lawyers, postgrads -- military, police, etc. Then consider what data is offered as evidence that IQ 1) exists, and 2) causes observable measures in real-world outcomes. Filtering on the test becomes evidence the test is a measure of anything. The application of "statistics" I dislike the most is where these feedback cycles exist, and large swathes of academia have some extreme responsibility here. Whole fields of gene-traits studies were created over decade+ and then disappeared overnight as actual sequencing took place. All the rigour and splendour of "statistics", and then poof when science was done, it disappeared. Since there are basically no scientific theories of human psychology, society, and the like -- gluing together correlations here should be seen as prima facie absurd. The alternative? Rely on expertise, and build resilience-to-failure into the system and tolerance for higher variability. Human expertise obtained in domain-specific environments is vastly superior to the species correlations of surveys written by idiots who've never done any actual science.

InDubioProRubio 2y ago

Could one create a proof of pseudo-science, by injecting a faked fundamental corner stone paper, that becomes proof by inheritance that a full field is rotten? Also why does this remind me of european politicans, claiming everyone wants to life european lifes, meanwhile whole countries goto war and atrocities without big counter-demonstrations by those western valued citizens .. narrative glider guns going ad absurdum..

kenjackson 2y ago
> Could one create a proof of pseudo-science, by injecting a faked fundamental corner stone paper, that becomes proof by inheritance that a full field is rotten? I don't think there's any doubt that pseudoscience exists, even amongst the most optimistic of scientists. The problem is identifying what is bad science vs what is good. The fact that I can send this from a small phone from a parking lot is proof that someone did good science at some point in time. Or that I've seen therapies like CBT turn someone who was struggling mentally on a daily basis to thrive, or that I've seen valve replacement surgery give someone years of great life after a "six months to live diagnosis" -- all show that there is good science. I think we almost need a "discipline" of people who validate scientific results, and people should be held accountable for results that validate or don't. Peer review is great. It's not a farce (I've gotten some incredible feedback on papers from it), but it is also extremely limited by design. We need more.
jjk166 2y ago
Even a broken clock is right twice a day. I can publish a paper with results I think are wrong, but it's entirely possible that follow up studies confirm it was accidentally correct. While this is improbable in general, if we restrict ourselves to publishing claims that are sufficiently plausible that people in the field would accept them as a corner stone paper, then there is a very decent chance it sounds plausible because it's true. Even if I test the claim myself before publishing to confirm it's false, I may have made an error. Possibly repeatedly publishing bogus papers in a certain manner might be able to confidently weed out poor academic hygeine, but it's not a trivial thing.

brabel 2y ago

Great video about that by Kurzgesagt: https://www.youtube.com/watch?v=bgo7rm5Maqg&ab_channel=Kurzgesagt%E2%80%93InaNutshell https://www.youtube.com/watch?v=bgo7rm5Maqg&ab_channel=Kurzg... They show how deep they had to go to find the original source of the claim that a single human's blood vessels, if lined up, would stretch 100,000km... and how that was quoted by so many that no one really know where the claim came from. And of course, it was wrong.

naasking 2y ago
Damn it, I had just posted this video too. Off to delete it.

deleted 2y ago

[deleted]

renewiltord 2y ago

This is the danger with non-empirical government funded “science”. Empiricism allows verification. Private funding means that things that matter get studied. If you trace the oft-cited claim from “studies” that cats kill so and so animals a year you will find it’s just someone’s Fermi estimate. And you’ll find that the person has a personal distaste for housecats.

rscho 2y ago
The idea that private funding means that what is studied matters is highly dubious at best. Most private research pertains to application and product development, and often relies on fundamental discoveries coming from academic labs. Only academia can afford to let scientists run loose. It certainly results in a good amount of inapplicable theories, but is nevertheless very much essential.
- renewiltord 2y ago
  I doubt gender studies is essential.