3 ms·

> I wish I had your generosity of spirit about contemporary practice and motivations of academic science. Ha! I seem to have given you a false impression. Ther

> I wish I had your generosity of spirit about contemporary practice and motivations of academic science. Ha! I seem to have given you a false impression. There's... so many problems. However, I don't think most (any?) researchers are actually ignorant of those problems. They are actually doing research somehow, so they've figured out the game well enough. I apparently managed to disguise this well when I said 'assign the appropriate amount of significance.' The unsaid implication here was that it may well be virtually none, particularly if it doesn't fit with a larger body of work. In this article for example, it's paired up with another similar finding from an actual physical experiment, which lends it more weight. > But the researchers are clearly happy to give interviews and be quoted I always wonder about this. When I've seen longer form interviews with researchers, they almost always seem appropriately uncertain. At least they do sometimes, I might just not be knowledgeable enough about the field to know if/when they aren't. Meanwhile, you don't get much of that in popsci work, beyond maybe a small disclaimer at the beginning. I think Adam Ragusea (a cooking youtuber of all people, though admittedly he's an ex-journalism professor as well, and also dealt being the interviewee and having things go poorly) actually covered the intricacies of this really well in a video[0]. I've tried too many times to write "Long story short, ..." here, and have ended up with way too much meandering. Just watch the video. It's worth it, and does a great job of covering the perspective of all parties involved. I think it's appropriate to assume good faith in individual instances - there are reasonable justifications for misrepresenting the certainty of some work in this kind of context. It's unlikely to cause any actual harm, and probably gets more people intellectually stimulated (which is probably good in both objective and subjective ways). In this article it even lets them make a concrete suggestion that has no cost to try and can probably help some people. I don't know that there is an objective way to weigh those benefits against presenting a more accurate representation of the certainty of some information, when a misunderstanding of it is unlikely to ever harm anybody. I assign quite a bit of value to that accuracy inherently, but that's coming from the core of my value system. I can't present a logical argument for it's correctness any more than I can the exact degree to which I think human suffering is bad. As a result I think there's room here for reasonable disagreements about where to draw the line. [0]https://www.youtube.com/watch?v=fxUnwsttr_8 https://www.youtube.com/watch?v=fxUnwsttr_8 (I think this is very illuminating if you're interested in this topic. I highly suggest finding the time for it, even if you understandably don't find the time to read the rest of this over-long comment.)

jrochkind1 5y ago
> However, I don't think most (any?) researchers are actually ignorant of those problems. They are actually doing research somehow, so they've figured out the game well enough The problem is that "winning the game" is about publications, tenure, and (less important but increasingly also) popular recognition, rather than about the size of your contribution to valid research results (one hopes that publications and tenure correlate, and yet...). So, getting results of dubious validity published, and covered in popular press, can be exactly "figuring out the game well enough", an end in itself, rather than a minor step toward a research program. You are right that I don't want to assume bad faith in any individual instances, and yet... here we are, in aggregate. What are the costs? I dunno, mis-education of the popular audience, for one, I guess. Maybe that doesn't matter, but, it kind of does? And the opportunity cost of all the scientific labor being spent on "easiest to publish the most times" instead of what might be highest priority to discover. And the aggregate collective effect of a scientific community enabling each other on that.
- Sohcahtoa82 5y ago
  > The problem is that "winning the game" is about publications, tenure, and (less important but increasingly also) popular recognition, rather than about the size of your contribution to valid research results (one hopes that publications and tenure correlate, and yet...). > So, getting results of dubious validity published, and covered in popular press, can be exactly "figuring out the game well enough", an end in itself, rather than a minor step toward a research program. You're describing Science by Press Conference/Press Release. https://en.wikipedia.org/wiki/Science_by_press_conference https://en.wikipedia.org/wiki/Science_by_press_conference
- foerbert 5y ago
  > The problem is that "winning the game" is about publications, tenure, and ... Totally. 100% with you there. I didn't mean researchers aren't doing those things, but rather that papers built for those ends are probably not polluting 'science' as a whole. The body of work product is getting increasingly swampy, but the researchers are also the best positioned people in the world to sieve through it. A large part of their job is to sort out what a paper actually tested and what it actually observed to begin with. These 'game playing' papers are quite blatant in comparison to the kinds of problems researchers already need to look for. > What are the costs? I dunno, mis-education of the popular audience, for one, I guess. Maybe that doesn't matter, but, it kind of does? Totally agree. I personally find it kind of abhorrent. I also can't honestly tell you why beyond "because" and am pretty certain most people don't feel as strongly about it as I do. I try to be mindful of that, but I think it mostly just gets used when I end up playing devil's advocate. > And the opportunity cost of all the scientific labor being spent on "easiest to publish the most times" instead of what might be highest priority to discover. Sure thing. It also increases time spent on literature searches for all that, and will for quite some time even if things changed tomorrow. On the other hand, maybe it provides an easier on-ramp for teaching prospective researchers to be skeptical even if it's in a Paper, and even how to analyze them? At least it's nice to try to hope there's some benefit to all this.

jrochkind1 5y ago

> However, I don't think most (any?) researchers are actually ignorant of those problems. They are actually doing research somehow, so they've figured out the game well enough The problem is that "winning the game" is about publications, tenure, and (less important but increasingly also) popular recognition, rather than about the size of your contribution to valid research results (one hopes that publications and tenure correlate, and yet...). So, getting results of dubious validity published, and covered in popular press, can be exactly "figuring out the game well enough", an end in itself, rather than a minor step toward a research program. You are right that I don't want to assume bad faith in any individual instances, and yet... here we are, in aggregate. What are the costs? I dunno, mis-education of the popular audience, for one, I guess. Maybe that doesn't matter, but, it kind of does? And the opportunity cost of all the scientific labor being spent on "easiest to publish the most times" instead of what might be highest priority to discover. And the aggregate collective effect of a scientific community enabling each other on that.

Sohcahtoa82 5y ago
> The problem is that "winning the game" is about publications, tenure, and (less important but increasingly also) popular recognition, rather than about the size of your contribution to valid research results (one hopes that publications and tenure correlate, and yet...). > So, getting results of dubious validity published, and covered in popular press, can be exactly "figuring out the game well enough", an end in itself, rather than a minor step toward a research program. You're describing Science by Press Conference/Press Release. https://en.wikipedia.org/wiki/Science_by_press_conference https://en.wikipedia.org/wiki/Science_by_press_conference
foerbert 5y ago
> The problem is that "winning the game" is about publications, tenure, and ... Totally. 100% with you there. I didn't mean researchers aren't doing those things, but rather that papers built for those ends are probably not polluting 'science' as a whole. The body of work product is getting increasingly swampy, but the researchers are also the best positioned people in the world to sieve through it. A large part of their job is to sort out what a paper actually tested and what it actually observed to begin with. These 'game playing' papers are quite blatant in comparison to the kinds of problems researchers already need to look for. > What are the costs? I dunno, mis-education of the popular audience, for one, I guess. Maybe that doesn't matter, but, it kind of does? Totally agree. I personally find it kind of abhorrent. I also can't honestly tell you why beyond "because" and am pretty certain most people don't feel as strongly about it as I do. I try to be mindful of that, but I think it mostly just gets used when I end up playing devil's advocate. > And the opportunity cost of all the scientific labor being spent on "easiest to publish the most times" instead of what might be highest priority to discover. Sure thing. It also increases time spent on literature searches for all that, and will for quite some time even if things changed tomorrow. On the other hand, maybe it provides an easier on-ramp for teaching prospective researchers to be skeptical even if it's in a Paper, and even how to analyze them? At least it's nice to try to hope there's some benefit to all this.