3 ms·

No, science definitely goes worse when there are more bad papers. There's a popular argument about freedom of speech that comes down to "the solution to bad sp

No, science definitely goes worse when there are more bad papers. There's a popular argument about freedom of speech that comes down to "the solution to bad speech is more good speech, not limiting bad speech," but that only works (assuming it's true at all) when there's some way for a listener to discern truth with some research or effort. > "Good papers are good because they have good arguments" No they are not. Good papers are good when they collected data correctly and then presented the results fairly. Arguments about what that data means hardly enters into it. You can't argue whether the data is correct or not without gathering it yourself, and you can't gather it yourself.

cortesoft 3y ago
Yeah, the problem with that "more speech" argument is the imbalance... lies are way easier and faster to generate than truth, and can be crafted perfectly to play on what the audience wants to hear. The truth has to be the truth, so it can't be crafted in the same way.
stale2002 3y ago
> You can't argue whether the data is correct or not without gathering it yourself, and you can't gather it yourself. You actually can argue about the data being "correct". The way to do is, is studies need to be much more radically open with their raw data sources. In AI, that would mean actually releasing the code you used, so people can try it themselves. Or, in more sociology stuff, where you are doing, I don't know, interviews with humans, you could release the physical videos of your interviews. And then for science stuff, show pictures/vidoes of the science you are doing. I'm sure there would still be way to hack this. But significant more code and data transparency would do a huge amount for allowing other people to replicate or verify your work.
huijzer 3y ago
> There's a popular argument about freedom of speech that comes down to "the solution to bad speech is more good speech, not limiting bad speech," but that only works (assuming it's true at all) when there's some way for a listener to discern truth with some research or effort. Yes exactly. If it is too hard for a group of scientists to figure out a flaw, then it is also too hard for two peer reviewers to figure out a flaw. Only time can detect all errors in science, assuming that nonconformist papers are allowed into the conversation. > Good papers are good when they collected data correctly and then presented the results fairly. This is only true for empirical papers. Good data is a part of a good argument in the case of empirical papers.
- notahacker 3y ago
  > If it is too hard for a group of scientists to figure out a flaw, then it is also too hard for two peer reviewers to figure out a flaw. The issue isn't that flawed papers are necessarily hard to spot, the issue is that indiscriminate publishing means that anyone studying a topic has to wade through a lot of flawed ones before they find anything halfway useful. Time and attention are finite. Peer review and journals caring about reputation, in theory, caps the number of readers of rubbish papers at 2 (and disincentivises writing unpublishable crap to submit in the first place... although this is now eroded since generating rubbish papers is now effort-free)

cortesoft 3y ago

Yeah, the problem with that "more speech" argument is the imbalance... lies are way easier and faster to generate than truth, and can be crafted perfectly to play on what the audience wants to hear. The truth has to be the truth, so it can't be crafted in the same way.

stale2002 3y ago

> You can't argue whether the data is correct or not without gathering it yourself, and you can't gather it yourself. You actually can argue about the data being "correct". The way to do is, is studies need to be much more radically open with their raw data sources. In AI, that would mean actually releasing the code you used, so people can try it themselves. Or, in more sociology stuff, where you are doing, I don't know, interviews with humans, you could release the physical videos of your interviews. And then for science stuff, show pictures/vidoes of the science you are doing. I'm sure there would still be way to hack this. But significant more code and data transparency would do a huge amount for allowing other people to replicate or verify your work.

huijzer 3y ago

> There's a popular argument about freedom of speech that comes down to "the solution to bad speech is more good speech, not limiting bad speech," but that only works (assuming it's true at all) when there's some way for a listener to discern truth with some research or effort. Yes exactly. If it is too hard for a group of scientists to figure out a flaw, then it is also too hard for two peer reviewers to figure out a flaw. Only time can detect all errors in science, assuming that nonconformist papers are allowed into the conversation. > Good papers are good when they collected data correctly and then presented the results fairly. This is only true for empirical papers. Good data is a part of a good argument in the case of empirical papers.

notahacker 3y ago
> If it is too hard for a group of scientists to figure out a flaw, then it is also too hard for two peer reviewers to figure out a flaw. The issue isn't that flawed papers are necessarily hard to spot, the issue is that indiscriminate publishing means that anyone studying a topic has to wade through a lot of flawed ones before they find anything halfway useful. Time and attention are finite. Peer review and journals caring about reputation, in theory, caps the number of readers of rubbish papers at 2 (and disincentivises writing unpublishable crap to submit in the first place... although this is now eroded since generating rubbish papers is now effort-free)