4 ms·

This is evidence of science working, and self correcting. Should we expect everything that gets published to be definitive, the end word, worthy of enshrining

This is evidence of science working, and self correcting. Should we expect everything that gets published to be definitive, the end word, worthy of enshrining on marble and being unerring in perpetuity? Of course not. There is no more unscientific view of knowledge and of science. Journal articles are the communications between practitioners. Blog posts before there were blog posts. Sometimes they are revolutionary. Sometimes they are work-a-day, filling in essential knowledge but not revolutionary. Many are arduous journeys that are hated by all by that touch the manuscripts by the time they get through the process of writing and peer review. Some are wrong. The scientific literature is not a series of proofs and lemmas, it's an accumulation of evidence paired with contemporaneous interpretation of what that data might mean, and those interpretations should be viewed as fluid. The data should be forever (unless there is error or fraud). The interpretation is moving. I don't agree that need we tons more "rigor" in science, instead it would be better to have clearer communication along with less busy work.

raverbashing 2y ago
> This is evidence of science working Is it? Looks more like academic bureaucracy to me And herein lies a big problem with modern science, the two are conflated. Publishing by itself does little to advance science. I feel there would be more science done if scientists discussed in an online forum than through papers
- JellyBeanThief 2y ago
  It couldn't be just any online forum, though. It would have to support scientists' specific needs. It would need to host datasets; make it easy to create and reference charts, diagrams, tables, figures; track post edits and the reasons for them; disincentivize both lengthy and low-content posts; provide advanced search and filter functionality; assist with jargon control; and support consensus building and measurement. And then there's need for accessibility, for users with physical or mental handicaps, and for users with limited computing and bandwidth resources.
  - raverbashing 2y ago
    Yes of course. Github works for some part. Or Arxiv.
  - KeplerBoy 2y ago
    > It would need to host datasets I haven't seen ieeexplore hosting datasets and yet we pay them to publish our stuff. Heck, most people don't even bother with publishing code or data, because there is very little or even negative incentive to do so. The system is completely borked, because most people publish papers in order to advance their PhD studies, not their respective fields.
- dotnet00 2y ago
  Considering all the endless meetings, conferences and discussions scientists have to attend and find room to do research in between, I doubt that the issue is a lack of means for them to discuss... I also highly disagree that publishing does very little to advance science, it puts a relatively self-contained result into the record. Replacing publishing with forums would be the euivalent of replacing documentation with a chat.
  - kelipso 2y ago
    Lol yeah, cannot imagine equating published papers and forum posts. I knock the peer review process all the time but it is a significant hurdle that acts as a filter for all sorts of trash.
- ComodoHacker 2y ago
  >Publishing by itself does little to advance science I strongly disagree. Publishing advances science the same way communication advances culture. Imagine Einstein didn't published any of his work. How many decades back in technology and quality of life we'd have been now? It's like any evolutionary beneficial trait gets transferred to all population in one generation instead of many. It saves precious time.
  - raverbashing 2y ago
    Let me put it this way: there is "publishing" and there's Publishing™ Before electronic communications became popular, publications would publish "letters to the editor" which were kinda like an informal paper with a partial discussion or results (some publications still do that, btw). My point here is that Publishing™ took place of simply publishing to share results with your colleagues and now the main objectives of Publishing™ is to dodge picky reviewers and to get your publishing score up to get more grants Thank FSM for Arxiv and for researchers sending "drafts" or "unofficial" versions around
Log_out_ 2y ago
Yes, i expect every latex document section i read to form a tree of links of thesistext and evidence that auto checks online for citations,counter thesis+ citations and retractions and dependency hell + citation cycles among ideas. If its undone , i want it to become red and then fade. Making all things i can read without red, true forever ..
keiferski 2y ago
I wonder if there is a formal term for this phenomenon: when the regulating system is only noticed because it regulates something correctly, but is criticized because this is the only time anyone hears about it. “Properly functioning systems don’t get any press,” or something like that.
- dumpsterdiver 2y ago
  Not a formal term, but a common colloquialism: “No news is good news.”
tsimionescu 2y ago
There is a well-known trade-off though. Since publishing is a communication channel between scientists, it only has value as long as it hits some threshold of signal-to-noise. And papers that are badly written, or use sloppy methods, or contain significant mistakes, or of course fraud - all of those add noise. They actively reduce the usefulness of the channel. Unfortunately, the incentives in worldwide academia encourage publishing at all cost, so as academia grows, so does the noise produced. Hopefully the signal grows as well, and I'm not sure where the ratio is today - but you definitely need to police the medium to keep publishing as a useful channel of communication in science.
bmacho 2y ago
> This is evidence of science working, and self correcting. This is a statement that can be said, and is said every time when something changes. Is it true? No, obviously it isn't. Is literally any change in literally anywhere an evidence for - that thing working - that thing becoming better - that thing being governed by scientific ideals? How does that make any sense? Actually 'science' stating A and not A is an evidence for science lying and/or being dumb.
- kergonath 2y ago
  > This is a statement that can be said, and is said every time when something changes. Is it true? No, obviously it isn't. Is literally any change in literally anywhere an evidence for This is a statement that can be said about any self-correcting system. If there is no change, then either we are at an optimum (I don’t see anybody seriously suggesting that about science), or the system is not working. Obviously, change itself is not sufficient as it could be in the wrong direction. Our scientific understanding improving is a sign of progress. If our ideas are not allowed to change, we’d hit a brick wall pretty quickly. > How does that make any sense? Actually 'science' stating A and not A is an evidence for science lying and/or being dumb. Science is not an person, it can do neither.
  - bmacho 2y ago
    > This is a statement that can be said about any self-correcting system. Definitely can be said about any organization ever, without second thinking. It changed? Then is an evidence that it self corrects, improves. Also it must have happened among scientific principles (because.), so it is a supporting evidence for those too. >> How does that make any sense? Actually 'science' stating A and not A is an evidence for science lying and/or being dumb. > Science is not an person, it can do neither. Meh. Science seen from outside, how it affects us, how it communicates with us, what it does, what is its inner state, etc, etc, it definitely is something that can lie and/or can be dumb. If you open a dictionary, you'll probably find that science is defined as an activity, so science is not even an entity, but that shouldn't matter much.
kergonath 2y ago
> I don't agree that need we tons more "rigor" in science, instead it would be better to have clearer communication along with less busy work. We do, even with a self-correcting system. Spurious interpretations and dodgy conclusions remain in the literature forever and the signal to noise ratio degrades significantly over time. Even if no interpretation is perfect, raising the bar is better. And then, the limit between lack of rigour and outright fraud is blurry. It’s better to stay well clear, and that requires rigour.
- Turskarama 2y ago
  Right, the system is self correcting but even so it can take literally decades sometimes for the corrections to happen, and in the meantime someone may have tried to build a career on what turned out to be a lie. There really needs to be at least one organization out there putting as much effort as possible into verifying high impact papers. I've had the thought before that if I was a Billionaire this would be my pet project.
  - kergonath 2y ago
    > I've had the thought before that if I was a Billionaire this would be my pet project. That would be a worthwhile endeavour.
    - tpoacher 2y ago
      https://i4replication.org/ https://i4replication.org/
  - tpoacher 2y ago
    https://i4replication.org/ https://i4replication.org/
- epistasis 2y ago
  > Even if no interpretation is perfect, raising the bar is better. Can you explain why? What metric are you using? What is the end goal? I ask because there are many metrics that I care about: - pace of discovery - cost of discovery - ability to work in directions that have high risk and high reward What I don't really care about - outsiders skimming over the literature, misinterpreting it, and coming to bad conclusions. Raising the "rigor" bar only seems to be positive for this metric that I don't care about.

raverbashing 2y ago

> This is evidence of science working Is it? Looks more like academic bureaucracy to me And herein lies a big problem with modern science, the two are conflated. Publishing by itself does little to advance science. I feel there would be more science done if scientists discussed in an online forum than through papers

JellyBeanThief 2y ago
It couldn't be just any online forum, though. It would have to support scientists' specific needs. It would need to host datasets; make it easy to create and reference charts, diagrams, tables, figures; track post edits and the reasons for them; disincentivize both lengthy and low-content posts; provide advanced search and filter functionality; assist with jargon control; and support consensus building and measurement. And then there's need for accessibility, for users with physical or mental handicaps, and for users with limited computing and bandwidth resources.
- raverbashing 2y ago
  Yes of course. Github works for some part. Or Arxiv.
- KeplerBoy 2y ago
  > It would need to host datasets I haven't seen ieeexplore hosting datasets and yet we pay them to publish our stuff. Heck, most people don't even bother with publishing code or data, because there is very little or even negative incentive to do so. The system is completely borked, because most people publish papers in order to advance their PhD studies, not their respective fields.
dotnet00 2y ago
Considering all the endless meetings, conferences and discussions scientists have to attend and find room to do research in between, I doubt that the issue is a lack of means for them to discuss... I also highly disagree that publishing does very little to advance science, it puts a relatively self-contained result into the record. Replacing publishing with forums would be the euivalent of replacing documentation with a chat.
- kelipso 2y ago
  Lol yeah, cannot imagine equating published papers and forum posts. I knock the peer review process all the time but it is a significant hurdle that acts as a filter for all sorts of trash.
ComodoHacker 2y ago
>Publishing by itself does little to advance science I strongly disagree. Publishing advances science the same way communication advances culture. Imagine Einstein didn't published any of his work. How many decades back in technology and quality of life we'd have been now? It's like any evolutionary beneficial trait gets transferred to all population in one generation instead of many. It saves precious time.
- raverbashing 2y ago
  Let me put it this way: there is "publishing" and there's Publishing™ Before electronic communications became popular, publications would publish "letters to the editor" which were kinda like an informal paper with a partial discussion or results (some publications still do that, btw). My point here is that Publishing™ took place of simply publishing to share results with your colleagues and now the main objectives of Publishing™ is to dodge picky reviewers and to get your publishing score up to get more grants Thank FSM for Arxiv and for researchers sending "drafts" or "unofficial" versions around

Log_out_ 2y ago

Yes, i expect every latex document section i read to form a tree of links of thesistext and evidence that auto checks online for citations,counter thesis+ citations and retractions and dependency hell + citation cycles among ideas. If its undone , i want it to become red and then fade. Making all things i can read without red, true forever ..

keiferski 2y ago

I wonder if there is a formal term for this phenomenon: when the regulating system is only noticed because it regulates something correctly, but is criticized because this is the only time anyone hears about it. “Properly functioning systems don’t get any press,” or something like that.

dumpsterdiver 2y ago
Not a formal term, but a common colloquialism: “No news is good news.”

tsimionescu 2y ago

There is a well-known trade-off though. Since publishing is a communication channel between scientists, it only has value as long as it hits some threshold of signal-to-noise. And papers that are badly written, or use sloppy methods, or contain significant mistakes, or of course fraud - all of those add noise. They actively reduce the usefulness of the channel. Unfortunately, the incentives in worldwide academia encourage publishing at all cost, so as academia grows, so does the noise produced. Hopefully the signal grows as well, and I'm not sure where the ratio is today - but you definitely need to police the medium to keep publishing as a useful channel of communication in science.

bmacho 2y ago

> This is evidence of science working, and self correcting. This is a statement that can be said, and is said every time when something changes. Is it true? No, obviously it isn't. Is literally any change in literally anywhere an evidence for - that thing working - that thing becoming better - that thing being governed by scientific ideals? How does that make any sense? Actually 'science' stating A and not A is an evidence for science lying and/or being dumb.

kergonath 2y ago
> This is a statement that can be said, and is said every time when something changes. Is it true? No, obviously it isn't. Is literally any change in literally anywhere an evidence for This is a statement that can be said about any self-correcting system. If there is no change, then either we are at an optimum (I don’t see anybody seriously suggesting that about science), or the system is not working. Obviously, change itself is not sufficient as it could be in the wrong direction. Our scientific understanding improving is a sign of progress. If our ideas are not allowed to change, we’d hit a brick wall pretty quickly. > How does that make any sense? Actually 'science' stating A and not A is an evidence for science lying and/or being dumb. Science is not an person, it can do neither.
- bmacho 2y ago
  > This is a statement that can be said about any self-correcting system. Definitely can be said about any organization ever, without second thinking. It changed? Then is an evidence that it self corrects, improves. Also it must have happened among scientific principles (because.), so it is a supporting evidence for those too. >> How does that make any sense? Actually 'science' stating A and not A is an evidence for science lying and/or being dumb. > Science is not an person, it can do neither. Meh. Science seen from outside, how it affects us, how it communicates with us, what it does, what is its inner state, etc, etc, it definitely is something that can lie and/or can be dumb. If you open a dictionary, you'll probably find that science is defined as an activity, so science is not even an entity, but that shouldn't matter much.

kergonath 2y ago

> I don't agree that need we tons more "rigor" in science, instead it would be better to have clearer communication along with less busy work. We do, even with a self-correcting system. Spurious interpretations and dodgy conclusions remain in the literature forever and the signal to noise ratio degrades significantly over time. Even if no interpretation is perfect, raising the bar is better. And then, the limit between lack of rigour and outright fraud is blurry. It’s better to stay well clear, and that requires rigour.

Turskarama 2y ago
Right, the system is self correcting but even so it can take literally decades sometimes for the corrections to happen, and in the meantime someone may have tried to build a career on what turned out to be a lie. There really needs to be at least one organization out there putting as much effort as possible into verifying high impact papers. I've had the thought before that if I was a Billionaire this would be my pet project.
- kergonath 2y ago
  > I've had the thought before that if I was a Billionaire this would be my pet project. That would be a worthwhile endeavour.
  - tpoacher 2y ago
    https://i4replication.org/ https://i4replication.org/
- tpoacher 2y ago
  https://i4replication.org/ https://i4replication.org/
epistasis 2y ago
> Even if no interpretation is perfect, raising the bar is better. Can you explain why? What metric are you using? What is the end goal? I ask because there are many metrics that I care about: - pace of discovery - cost of discovery - ability to work in directions that have high risk and high reward What I don't really care about - outsiders skimming over the literature, misinterpreting it, and coming to bad conclusions. Raising the "rigor" bar only seems to be positive for this metric that I don't care about.