4 ms·

> Most of your rebuttals seem to hinge on "yeah, but that problem isn't unique to publishing." That is a kind of side-stepping that misses the point. If there

by rhelz 3y ago

> Most of your rebuttals seem to hinge on "yeah, but that problem isn't unique to publishing." That is a kind of side-stepping that misses the point. If there is a specific objection you'd like to revisit, I'd be happy to discuss it. But I wouldn't self-describe what I'm doing as "sidestepping"--I'd say its avoiding bike shedding and keeping the conversation focused. I mean, it's a pretty facile objection to say some variation on "but if we pay them how do we know we'll get our money's worth?" when we pay for goods and services all the time with very high confidence that we'll get what we pay for. Surely, there's plenty of considerations to discuss, and I've tried to squarely address all objections which are specific to this proposal. But how to hire and use consultants, or how to ensure you get what you contracted for, are largely solved problems, and off-topic. > This would mean the advocated process would result in a lot less sharing of methods. I don't think my proposal would even apply to internal R&D groups who wanted to keep things proprietary. I mean, I can certainly understand wanting to reserve some methods or data as being proprietary. But choosing to do that is, ipso facto, not sharing them. How would paying reviewers and replicators for their time cause any less sharing to happen? I mean, if your paper doesn't describe the experiments you performed in enough detail to allow other groups to replicate it, its not a scientific paper to begin with. It's either a press release, or a whitepaper, or some other form of creative writing, and publishing it is either public relations, or advertisement--not science. Which is not to say that it's immoral or useless, or to denigrate it in any way. Not everything we do has to be science. My proposal is just for scientists communicating scientific results with other scientists. Maybe I'm missing something, but I don't see how it would inhibit the kinds of practices you are describing in any way. It would make it harder for people to claim their "results" are scientific, but are not. It would be a big obstacle to publishing fraudulent papers in scientific journals. It would make it harder for somebody to claim the mantel of "science" to give credibility to their claims. But I really don't see how paying reviewers and replicators would stop anybody from sharing as much or as little as they wanted to.

bumby 3y ago
Apologies, but when the central claim is about mitigating downsides of adding money into a system and you acknowledge the potential for downsides exists but fail to provide any mitigation, it is sidestepping the main focus of the discussion. I also think there is a misunderstanding when you’re talking about internal R&D. The situation I’m talking about isn’t where someone wants to protect a proprietary method, but rather proprietary data. I could have sensitive information that I don’t want to share, but also recognize a method I’ve developed is useful to others. The harder you make it to share that method (by requiring me to sanitize all the data to make it non-sensitive) the less likely I’m going to share it. When things like security or law come into play, the easiest path is always “no.” >If there is a specific objection you'd like to revisit Take the fact that whenever you inject pay into a system, it tends to pervert that system away from the original goal and into a goal of maximizing pay. You acknowledge that but just say it isn't unique. I agree it's not unique, but what I'm after is how do you propose to mitigate it (assuming your goal isn't to simply maximize pay, but rather provide some balance of quality, pay, and quantity). What guardrails do you put in place? Maximum on the number of reviews per quarter? That might limit those reviewers who can crank out many quality reviews. Do you instead provide a framework for reviewing the reviews for quality? That adds another layer of bureaucracy to an already bureaucratic system. Do you implement reviewer scorecards? A decaying rate of pay for each review? And on and on. Again, the intent wasn't to imply these are unique problems but to probe for good fixes. Those aspects you say are digressions (consultancy etc) are topics you brought to the discussion, seemingly to address the mitigation question without actually providing a specific response. Doing "whatever they do elsewhere" isn't really an answer.

by rhelz 3y ago

bumby 3y ago

Apologies, but when the central claim is about mitigating downsides of adding money into a system and you acknowledge the potential for downsides exists but fail to provide any mitigation, it is sidestepping the main focus of the discussion. I also think there is a misunderstanding when you’re talking about internal R&D. The situation I’m talking about isn’t where someone wants to protect a proprietary method, but rather proprietary data. I could have sensitive information that I don’t want to share, but also recognize a method I’ve developed is useful to others. The harder you make it to share that method (by requiring me to sanitize all the data to make it non-sensitive) the less likely I’m going to share it. When things like security or law come into play, the easiest path is always “no.” >If there is a specific objection you'd like to revisit Take the fact that whenever you inject pay into a system, it tends to pervert that system away from the original goal and into a goal of maximizing pay. You acknowledge that but just say it isn't unique. I agree it's not unique, but what I'm after is how do you propose to mitigate it (assuming your goal isn't to simply maximize pay, but rather provide some balance of quality, pay, and quantity). What guardrails do you put in place? Maximum on the number of reviews per quarter? That might limit those reviewers who can crank out many quality reviews. Do you instead provide a framework for reviewing the reviews for quality? That adds another layer of bureaucracy to an already bureaucratic system. Do you implement reviewer scorecards? A decaying rate of pay for each review? And on and on. Again, the intent wasn't to imply these are unique problems but to probe for good fixes. Those aspects you say are digressions (consultancy etc) are topics you brought to the discussion, seemingly to address the mitigation question without actually providing a specific response. Doing "whatever they do elsewhere" isn't really an answer.