6 ms·

Wouldn't it be nice if there was a platform, some kind of hub where all research papers could be published and visible for anyone with an interest in the partic

Wouldn't it be nice if there was a platform, some kind of hub where all research papers could be published and visible for anyone with an interest in the particular field. The platform would allow anyone, be it academics or just field experts to request for a change or addition of text. Anyone could see these (pull)requests and the initial author has the power to accept or deny it. This allows for peer reviews and corrections being integrated into the paper, making the whole process more organic. Any change to paper is logged and tracked so you could see the origin of a change and it's reason. Fraudulent papers could be marked by the research community as such. I'm being a bit sarcastic here, but a technology such as git with a platform such as GitHub could maybe make the process of publishing and reviewing research papers easier to control. Maybe you have to tailor it a bit to make it easier to work with this use case. Like open source, peers can start adding and reviewing directly at the source, instead writing a new paper or writing to all of the scientific journals to issue a different statement.

semi-extrinsic 8y ago
There are certainly initiatives going this way. E.g. LiveCOMS in my own field. The crux of the matter is whether these will gain traction or not. https://livecomsjournal.github.io/about/ https://livecomsjournal.github.io/about/
lkrubner 8y ago
Subversion would be the obvious choice, not git. The whole point of a DVCS is that it avoids having a canonical repo, whereas in science we very much want to have a canonical version of the truth (otherwise there would be no such thing as fraud, if everyone is allowed their own version of the truth), and Subversion defaults to having a canonical version.
- bounder 8y ago
  I agree. As I said probably the tools and technology should be tailor for the specific use case of scientific papers. It would be nice however that these papers evolve online, instead of the written in stone papers that are publish today in scientific journals. A small note instructing the reader that a certain theory has been disproved or changed in the original paper would already help.
speedplane 8y ago
It's not a terribly good assumption. With programming and git (or any version control system), everything is submitted to the repository, functional features, tests, data, etc. With scientific papers, the only thing submitted is the final published paper itself. There is a countless amount of data, settings, computer code, etc, that is not included with a scientific paper. The ability to inspect, comment, and modify a scientific publication doesn't mean much if the underlying work is unavailable.
- beojan 8y ago
  That doesn't mean there isn't a git repository (or multiple) somewhere with all the code and TeX source for the paper. It's just that these aren't usually published.
  - speedplane 8y ago
    Your focus on the Tex source is pretty amusing. Imagine working for a year on a project, building countless experiments that your write code for, edit, annotate in random files or on a notepad, build huge datasets, spin up and down servers or various equipment. Then, when you finally get a result, you put write a nicely formatted document in Tex or Microsoft Word or whatever. That final document represents a tiny fraction of the work and thought you put into it. The ability to mess with that final product without the underlying 99% of thinking and effort that went into it gets you nowhere.
    - COGlory 8y ago
      Seriously, this whole git idea shows the level of ignorance surrounding the actual scientific process and workflow. The data files for my work (protein crystallography) are terabytes upon terabytes large. There's no reasonable way to distribute them. They go through multiple layers of data paring. Any final raw data is included in the supplemental material, but it has already been heavily curated and reduced. And also biased. The solution to this problem is not iteration and revision.
    - beojan 8y ago
Phenomenit 8y ago
I rather see reproduction platform where someone can use a paper and create a new paper or fork some parts of it with credit. The original paper should be immutable.
tucnak 8y ago
https://veritas.icu/blackpaper https://veritas.icu/blackpaper
LoSboccacc 8y ago
Peer review is not about reaching consensus tho, is about verifying methodologies and reproducibility, a system like that could easily become a tyranny of the mediocre, see for example Wikipedia.
Qworg 8y ago
It is hard to make happen in competitive fields because good data is an expensive advantage over other labs.
ordu 8y ago
I have a slightly another idea. Any research activity of a scientist should be recorded by him/her in his/her own repository. Any activity, like reading a paper, or moments like "I have an idea", or "lets try a small scale pilot experiment with fellow scientists of mine as a subjects", or "pilot experiment was unsuccessful, but can be modified...". Any publication should be forked from the main repository, selecting from any relevant data from parent, and in that fork strict procedures of scientific research are recorded, processed, and finally end up with an article. In this case you can track how an idea was formed, how it was modified to match empirical data. You can track when methods of data processing was chosen (before gathering of data or after), and so on. Such a repository can give enough information to decide was the research is a fraud or no.
- eXpl0it3r 8y ago
  It would also heavily detract from actually writing the paper. It's easy in hindsight to think about certain steps it took to get where you end up, but if you are at the start, there are simply too many intentional and unintentional steps to record everything. I much rather have people focus on the actual science than documenting their steps ever few minutes.
  - ordu 8y ago
    An article is a hindsight reflections on what was done. If you are doing research you need to make notes. I personally can't remember what I've read and where I got an idea. I need to write things down, to be able to refer to it later. I'm logging my thoughts on regular basis, just to be sure they are verbalized and formalized. The goal is not to document every step in a formal manner, the goal is to leave a trail that can be tracked by a curious researcher to see how article was formed. Current situation is bad, because articles do not capture all the information that needed to validate the research findings. p<0.05 is not enough for this. Including more information into the article is one of the ways to deal with that problem. But the other is to leave footprints, to create some historical data, that could be analized. Or not analized if no one needed it. It is like lazy-evaluation: just generate some data to allow your followers to reason about what precisely you've done. If they needed it, they will research your trail. You do not need to go through a difficult task of writing a high-quality texts about every your step. The only thing you need to change is to make such notes public. It is coupled with some potential ethical issues, like privacy of subjects for example. If I run experiment, I gathered some data, I couldn't just throw this data into public access. It is possible that there are other ethical issues, maybe something about privacy of researcher as a person. These issues can make logging research activities a pain.
tmaly 8y ago
I once suggested the same idea for bills in Congress. They use to stuff those little addons to the bills for billions of dollars in spending and you could not determine who did it.
joppy 8y ago
I'm all for research papers being made more open and available, rather than hidden away through paid-for journals --- this is in my opinion one of the worst parts of how knowledge is shared. However, I'm not sure that making the papers actually able to be modified through a platform like this will make research more accessible or authentic, at least without changing virtually everything else about how science is done in academic institutions. If the content of a paper changes over time, it becomes more difficult to reference that paper. You now have to include both the paper and something which identifies which version of the paper you read. This is possible of course, but is a fairly big inconvenience for mostly stupid reasons, such as various citation styles not allowing hyperlinks of any form. There is also the possibility of the content of a paper changing completely, though we should hope that people would use a tool like this sensibly and it wouldn't be an issue. This proposed platform puts a burden on the authors of the paper to maintain a paper indefinitely. Of course, any paper you author you are at least partially responsible for, and any legitimate errors should concern you. However, having to keep up with many comments and/or critiques after a paper is published would be very tiring. The metrics by which academics are measured for their employment are heavily biased towards publishing a lot of papers, and having to spend a lot of time curating old work would be detrimental to anybody's career. (These metrics are also a reason that people would rather write a new paper to build on the old, rather than incorporating it into an existing work). Another issues is papers with many authors: do they all need to agree on a change to have it merged? How often do you want to try to contact 10 people in 4 different timezones to debate about minor changes to old research? I also don't really understand how a system like this would address the problem of fraudulent papers in a better way than is done today, which (as far as I am aware) is academics contacting journals to get the papers retracted. I don't think it's a bad idea, but it is so far away from how people publish today that many things (journals, metrics, work loads) are against you.
- gbrown 8y ago
  I can't even get coauthors to use LaTeX+git. Word+EndNote+Emails is far more popular, despite how terrible it makes collaboration.
  - 8y ago
antpls 8y ago
I would drop the concept of "paper", and use a unique, gigantic graph of hypothesis and experiments that keeps growing as research goes on. Everyone would contritube to the same graph without duplicate data, or unaccepted piece of proof/knowledge
fleetin 8y ago
It is a great idea. However, we do not have a distributed free internet throughout the world.

semi-extrinsic 8y ago

There are certainly initiatives going this way. E.g. LiveCOMS in my own field. The crux of the matter is whether these will gain traction or not. https://livecomsjournal.github.io/about/ https://livecomsjournal.github.io/about/

lkrubner 8y ago

Subversion would be the obvious choice, not git. The whole point of a DVCS is that it avoids having a canonical repo, whereas in science we very much want to have a canonical version of the truth (otherwise there would be no such thing as fraud, if everyone is allowed their own version of the truth), and Subversion defaults to having a canonical version.

bounder 8y ago
I agree. As I said probably the tools and technology should be tailor for the specific use case of scientific papers. It would be nice however that these papers evolve online, instead of the written in stone papers that are publish today in scientific journals. A small note instructing the reader that a certain theory has been disproved or changed in the original paper would already help.

speedplane 8y ago

It's not a terribly good assumption. With programming and git (or any version control system), everything is submitted to the repository, functional features, tests, data, etc. With scientific papers, the only thing submitted is the final published paper itself. There is a countless amount of data, settings, computer code, etc, that is not included with a scientific paper. The ability to inspect, comment, and modify a scientific publication doesn't mean much if the underlying work is unavailable.

beojan 8y ago
That doesn't mean there isn't a git repository (or multiple) somewhere with all the code and TeX source for the paper. It's just that these aren't usually published.
- speedplane 8y ago
  Your focus on the Tex source is pretty amusing. Imagine working for a year on a project, building countless experiments that your write code for, edit, annotate in random files or on a notepad, build huge datasets, spin up and down servers or various equipment. Then, when you finally get a result, you put write a nicely formatted document in Tex or Microsoft Word or whatever. That final document represents a tiny fraction of the work and thought you put into it. The ability to mess with that final product without the underlying 99% of thinking and effort that went into it gets you nowhere.
  - COGlory 8y ago
    Seriously, this whole git idea shows the level of ignorance surrounding the actual scientific process and workflow. The data files for my work (protein crystallography) are terabytes upon terabytes large. There's no reasonable way to distribute them. They go through multiple layers of data paring. Any final raw data is included in the supplemental material, but it has already been heavily curated and reduced. And also biased. The solution to this problem is not iteration and revision.
  - beojan 8y ago

Phenomenit 8y ago

I rather see reproduction platform where someone can use a paper and create a new paper or fork some parts of it with credit. The original paper should be immutable.

tucnak 8y ago

https://veritas.icu/blackpaper https://veritas.icu/blackpaper

LoSboccacc 8y ago

Peer review is not about reaching consensus tho, is about verifying methodologies and reproducibility, a system like that could easily become a tyranny of the mediocre, see for example Wikipedia.

Qworg 8y ago

It is hard to make happen in competitive fields because good data is an expensive advantage over other labs.

ordu 8y ago

I have a slightly another idea. Any research activity of a scientist should be recorded by him/her in his/her own repository. Any activity, like reading a paper, or moments like "I have an idea", or "lets try a small scale pilot experiment with fellow scientists of mine as a subjects", or "pilot experiment was unsuccessful, but can be modified...". Any publication should be forked from the main repository, selecting from any relevant data from parent, and in that fork strict procedures of scientific research are recorded, processed, and finally end up with an article. In this case you can track how an idea was formed, how it was modified to match empirical data. You can track when methods of data processing was chosen (before gathering of data or after), and so on. Such a repository can give enough information to decide was the research is a fraud or no.

eXpl0it3r 8y ago
It would also heavily detract from actually writing the paper. It's easy in hindsight to think about certain steps it took to get where you end up, but if you are at the start, there are simply too many intentional and unintentional steps to record everything. I much rather have people focus on the actual science than documenting their steps ever few minutes.
- ordu 8y ago
  An article is a hindsight reflections on what was done. If you are doing research you need to make notes. I personally can't remember what I've read and where I got an idea. I need to write things down, to be able to refer to it later. I'm logging my thoughts on regular basis, just to be sure they are verbalized and formalized. The goal is not to document every step in a formal manner, the goal is to leave a trail that can be tracked by a curious researcher to see how article was formed. Current situation is bad, because articles do not capture all the information that needed to validate the research findings. p<0.05 is not enough for this. Including more information into the article is one of the ways to deal with that problem. But the other is to leave footprints, to create some historical data, that could be analized. Or not analized if no one needed it. It is like lazy-evaluation: just generate some data to allow your followers to reason about what precisely you've done. If they needed it, they will research your trail. You do not need to go through a difficult task of writing a high-quality texts about every your step. The only thing you need to change is to make such notes public. It is coupled with some potential ethical issues, like privacy of subjects for example. If I run experiment, I gathered some data, I couldn't just throw this data into public access. It is possible that there are other ethical issues, maybe something about privacy of researcher as a person. These issues can make logging research activities a pain.

tmaly 8y ago

I once suggested the same idea for bills in Congress. They use to stuff those little addons to the bills for billions of dollars in spending and you could not determine who did it.

joppy 8y ago

I'm all for research papers being made more open and available, rather than hidden away through paid-for journals --- this is in my opinion one of the worst parts of how knowledge is shared. However, I'm not sure that making the papers actually able to be modified through a platform like this will make research more accessible or authentic, at least without changing virtually everything else about how science is done in academic institutions. If the content of a paper changes over time, it becomes more difficult to reference that paper. You now have to include both the paper and something which identifies which version of the paper you read. This is possible of course, but is a fairly big inconvenience for mostly stupid reasons, such as various citation styles not allowing hyperlinks of any form. There is also the possibility of the content of a paper changing completely, though we should hope that people would use a tool like this sensibly and it wouldn't be an issue. This proposed platform puts a burden on the authors of the paper to maintain a paper indefinitely. Of course, any paper you author you are at least partially responsible for, and any legitimate errors should concern you. However, having to keep up with many comments and/or critiques after a paper is published would be very tiring. The metrics by which academics are measured for their employment are heavily biased towards publishing a lot of papers, and having to spend a lot of time curating old work would be detrimental to anybody's career. (These metrics are also a reason that people would rather write a new paper to build on the old, rather than incorporating it into an existing work). Another issues is papers with many authors: do they all need to agree on a change to have it merged? How often do you want to try to contact 10 people in 4 different timezones to debate about minor changes to old research? I also don't really understand how a system like this would address the problem of fraudulent papers in a better way than is done today, which (as far as I am aware) is academics contacting journals to get the papers retracted. I don't think it's a bad idea, but it is so far away from how people publish today that many things (journals, metrics, work loads) are against you.

gbrown 8y ago
I can't even get coauthors to use LaTeX+git. Word+EndNote+Emails is far more popular, despite how terrible it makes collaboration.
- 8y ago

antpls 8y ago

I would drop the concept of "paper", and use a unique, gigantic graph of hypothesis and experiments that keeps growing as research goes on. Everyone would contritube to the same graph without duplicate data, or unaccepted piece of proof/knowledge

fleetin 8y ago

It is a great idea. However, we do not have a distributed free internet throughout the world.