8 ms·

One reason blogs are of lower scientific quality -- anyone can make series of large and unresearched claims, cherry picking a few examples to claim a broad poin

One reason blogs are of lower scientific quality -- anyone can make series of large and unresearched claims, cherry picking a few examples to claim a broad point. Good journals do have peer review, and they wouldn't have accepted this. Also, one big disadvantage of blogs -- depending on where you host it, they often don't last long, and point 4 (easy to edit) is a blessing and a curse, particularly if people edit things without telling you they did it. Of course, blogs have their place, I keep one myself, it's great for breaking news, explaining things in greater depth, sharing and understanding. And journals could learn from blogs / arxiv (make it easier to make corrections, allow discussions).

stinos 10y ago
anyone I think is they key point here which the OP seems to ignore completely: I sort of agree with everything the OP states, but he also forgets some key aspects on blogging. For instance anyone can post whatever even on subjects he/she really doesn't know a lot about. Also, not every blog post does get reviews from all sides and might only get 'yes good' replies originating solely from confirmation bias while the lack of any criticism doesn't make everything right but for instance just indicates the critics didn't find their way to the blog post.
- ansgri 10y ago
  There's an inverse side to that "anyone": professional scientists are under pressure to publish long scientific-looking articles, and their fellow reviewers have almost no motivation to spend any significant time reviewing (no pay, no nothing, just a small non-quantifiable reputation penalty for declining to review). Whereas most scientific bloggers post things they are interested in and therefore feel have something to say.
  - stinos 10y ago
    professional scientists are under pressure to publish long scientific-looking articles, and their fellow reviewers have almost no motivation to spend any significant time reviewing You have a point but from all scientists I know myself there's not one for which this applies. Might depend on the field though and I'm not denying there's a bunch of impossible-to-reproduce-crap published and there are rotten apples everywhere. But what I see (in fundamental research which is usually not as publicly known as other kinds) doesn't come near what you describe, even though it's anecdotal of course.. Yes there is pressure to publish but their papers aren't 'scinetific-looking', they're properly scientific and they are just the length needed to describe the findings on the subject. And there is an abundance of motivation for reviewing mostly inspired by wanting to make sure everything is as correct as possible, for the sake of research.
    - ansgri 10y ago
      Thanks for the balancing point. My experience describes applied research in such "hot topics" as computer vision and machine learning, and many people, myself included, are more interested in getting the technology to work and apply it in real commercial projects, where paper publishing may seem like a price to pay for research grants subsidizing your R&D. These grants are aimed at building a working technology and strengthening national economy, so there's nothing wrong with the approach per se, but when your reviewers share your values they have little motivation to find every possible mistake in your paper, at least for second- or third-rate journals (which are still indexed in WoS so are perfectly enough for the funding agency).
- TheOtherHobbes 10y ago
  The value of content depends on the collective intelligence of the network around it. Good networks concentrate intelligence by providing feedback and support that helps the network converge on useful, original insight. Bad/malicious networks destroy it with noise and false information, creating a feedback network that suppresses reality-based original insight. The process of blogging is irrelevant. So is the process of peer review. It's the quality of the networks around each area of interest that defines the real value, not the process. E.g. peer review works well when it's part of a high quality network. When it isn't, it's no better than random posting.
  - marcosdumay 10y ago
    Do not underestimate the effect of tooling on the shape of a community. Free and gated idea exchange create completely different communities, with completely different problems and advantages.
shubhamjain 10y ago
Getting a research paper published sounds so meritorious, no matter how idiotic or trivial it is. Take for example, Princeton 'research' in 2014 that predicted that Facebook will lose 80% of users by 2017 [1]. The evidence? Google trends data which has near-zero correlation with DAUs of Facebook. The research not only got published, but also got a wider-audience after major media websites covered it. Had it been a blog post, I don't think anyone would have taken more than a cursory look at the analysis. Yes, anyone can fit data points and reach a conclusion with blogs, but isn't academic research fraught with similar problems already? [1]: https://www.theguardian.com/technology/2014/jan/22/facebook-princeton-researchers-infectious-disease https://www.theguardian.com/technology/2014/jan/22/facebook-...
- kilburn 10y ago
  You picked a bad example. The actual source article is published at... arxiv. From an academic perspective it is worth exactly as much as a blog. Academic literature has many problems but it is way above the blogosphere in that regard. And media coverage is going to suck no matter where stuff is published...
  - gpm 10y ago
    > From an academic perspective it is worth exactly as much as a blog. From the perspective of peer review, yes. But per google scholar that paper has been 66 cited times, including in many articles that are in at least somewhat reputable journals (published via Elsevier, ieee, Springer, etc) and apparently two books [0]. So I think it's pretty clear that being a "scientific paper from Princeton" was significant reputation wise regardless of whether or not it was peer reviewed. [0] https://scholar.google.com/scholar?q=Epidemiological%20modeling%20of%20online%20social%20network%20dynamics&btnG=Search&as_sdt=800000000001&as_sdtp=on https://scholar.google.com/scholar?q=Epidemiological%20model... I was also skeptical about the citations so I checked that two with freely available pdfs were actually this paper, they were [1] https://pdfs.semanticscholar.org/4ebb/b654c59aff454ad97301e6ecd9e82c69122b.pdf https://pdfs.semanticscholar.org/4ebb/b654c59aff454ad97301e6... [2] https://arxiv.org/pdf/1509.07805.pdf https://arxiv.org/pdf/1509.07805.pdf
lucb1e 10y ago
> depending on where you host it, they often don't last long I wrote my bachelor's thesis last semester. You wouldn't believe for how many papers I needed to use archive.org. By the end of it, I made a small donation and recommended the company to do the same. Blog posts might not be better, but hosted papers aren't that stable either.
BeetleB 10y ago
>Good journals do have peer review, and they wouldn't have accepted this. Most peer reviewers do not get to see the data in the article.
- rspeer 10y ago
  When I submitted a paper to AAAI, there were dire warnings that the paper would be rejected if I included links to any supplemental materials (such as code or data), because this would compromise blind review. The conference software they were using had a sketchy form for uploading supplementary data, but it appeared that they expected it to be used for small data tables or something. It certainly wasn't going to be an option to upload 12 GB of data into their conference software; they mentioned nothing about code, particularly how to specify its dependencies and the computing environment it needed to run in; and it also certainly wasn't going to appear in any form that was convenient to review if I did. How could you submit code to blind review anyway? Do you make alternate versions of all the dependencies that don't credit any authors that overlap with the authors of the paper? In short, because of blind review, I was forbidden from doing anything that would make my paper reproducible at review time.
  - kilburn 10y ago
    You are confounding reproducibility and repeatability. Repeatability is the ability to repeat your exact experiment under the same conditions, whereas reproducibility refers to other people being able to reproduce your experiment under similar conditions and obtain congruent results. Of course, both are needed for great science. Nonetheless, your paper alone should provide a good enough description of the conditions and methods you used for the paper to be reproducible. In fact, it can easily be argued that the ability to just run your code instead of re-implementing it according to your description is actually detrimental. To see how, consider that a bug in your code, re-used by other researchers, can easily lead to multiple derivative works finding entirely wrong results. In contrast, if those other researchers re-implemented your method there would be a much lower probability of them doing the same mistake you did, leading to incongruent results and hence raising alarms that probably lead to the discovery of your bug. Although re-implementing your algorithms is significantly more work, the overall quality of our research would benefit from doing so...

stinos 10y ago

anyone I think is they key point here which the OP seems to ignore completely: I sort of agree with everything the OP states, but he also forgets some key aspects on blogging. For instance anyone can post whatever even on subjects he/she really doesn't know a lot about. Also, not every blog post does get reviews from all sides and might only get 'yes good' replies originating solely from confirmation bias while the lack of any criticism doesn't make everything right but for instance just indicates the critics didn't find their way to the blog post.

ansgri 10y ago
There's an inverse side to that "anyone": professional scientists are under pressure to publish long scientific-looking articles, and their fellow reviewers have almost no motivation to spend any significant time reviewing (no pay, no nothing, just a small non-quantifiable reputation penalty for declining to review). Whereas most scientific bloggers post things they are interested in and therefore feel have something to say.
- stinos 10y ago
  professional scientists are under pressure to publish long scientific-looking articles, and their fellow reviewers have almost no motivation to spend any significant time reviewing You have a point but from all scientists I know myself there's not one for which this applies. Might depend on the field though and I'm not denying there's a bunch of impossible-to-reproduce-crap published and there are rotten apples everywhere. But what I see (in fundamental research which is usually not as publicly known as other kinds) doesn't come near what you describe, even though it's anecdotal of course.. Yes there is pressure to publish but their papers aren't 'scinetific-looking', they're properly scientific and they are just the length needed to describe the findings on the subject. And there is an abundance of motivation for reviewing mostly inspired by wanting to make sure everything is as correct as possible, for the sake of research.
  - ansgri 10y ago
    Thanks for the balancing point. My experience describes applied research in such "hot topics" as computer vision and machine learning, and many people, myself included, are more interested in getting the technology to work and apply it in real commercial projects, where paper publishing may seem like a price to pay for research grants subsidizing your R&D. These grants are aimed at building a working technology and strengthening national economy, so there's nothing wrong with the approach per se, but when your reviewers share your values they have little motivation to find every possible mistake in your paper, at least for second- or third-rate journals (which are still indexed in WoS so are perfectly enough for the funding agency).
TheOtherHobbes 10y ago
The value of content depends on the collective intelligence of the network around it. Good networks concentrate intelligence by providing feedback and support that helps the network converge on useful, original insight. Bad/malicious networks destroy it with noise and false information, creating a feedback network that suppresses reality-based original insight. The process of blogging is irrelevant. So is the process of peer review. It's the quality of the networks around each area of interest that defines the real value, not the process. E.g. peer review works well when it's part of a high quality network. When it isn't, it's no better than random posting.
- marcosdumay 10y ago
  Do not underestimate the effect of tooling on the shape of a community. Free and gated idea exchange create completely different communities, with completely different problems and advantages.

shubhamjain 10y ago

Getting a research paper published sounds so meritorious, no matter how idiotic or trivial it is. Take for example, Princeton 'research' in 2014 that predicted that Facebook will lose 80% of users by 2017 [1]. The evidence? Google trends data which has near-zero correlation with DAUs of Facebook. The research not only got published, but also got a wider-audience after major media websites covered it. Had it been a blog post, I don't think anyone would have taken more than a cursory look at the analysis. Yes, anyone can fit data points and reach a conclusion with blogs, but isn't academic research fraught with similar problems already? [1]: https://www.theguardian.com/technology/2014/jan/22/facebook-princeton-researchers-infectious-disease https://www.theguardian.com/technology/2014/jan/22/facebook-...

kilburn 10y ago
You picked a bad example. The actual source article is published at... arxiv. From an academic perspective it is worth exactly as much as a blog. Academic literature has many problems but it is way above the blogosphere in that regard. And media coverage is going to suck no matter where stuff is published...
- gpm 10y ago
  > From an academic perspective it is worth exactly as much as a blog. From the perspective of peer review, yes. But per google scholar that paper has been 66 cited times, including in many articles that are in at least somewhat reputable journals (published via Elsevier, ieee, Springer, etc) and apparently two books [0]. So I think it's pretty clear that being a "scientific paper from Princeton" was significant reputation wise regardless of whether or not it was peer reviewed. [0] https://scholar.google.com/scholar?q=Epidemiological%20modeling%20of%20online%20social%20network%20dynamics&btnG=Search&as_sdt=800000000001&as_sdtp=on https://scholar.google.com/scholar?q=Epidemiological%20model... I was also skeptical about the citations so I checked that two with freely available pdfs were actually this paper, they were [1] https://pdfs.semanticscholar.org/4ebb/b654c59aff454ad97301e6ecd9e82c69122b.pdf https://pdfs.semanticscholar.org/4ebb/b654c59aff454ad97301e6... [2] https://arxiv.org/pdf/1509.07805.pdf https://arxiv.org/pdf/1509.07805.pdf

lucb1e 10y ago

> depending on where you host it, they often don't last long I wrote my bachelor's thesis last semester. You wouldn't believe for how many papers I needed to use archive.org. By the end of it, I made a small donation and recommended the company to do the same. Blog posts might not be better, but hosted papers aren't that stable either.

BeetleB 10y ago

>Good journals do have peer review, and they wouldn't have accepted this. Most peer reviewers do not get to see the data in the article.

rspeer 10y ago
When I submitted a paper to AAAI, there were dire warnings that the paper would be rejected if I included links to any supplemental materials (such as code or data), because this would compromise blind review. The conference software they were using had a sketchy form for uploading supplementary data, but it appeared that they expected it to be used for small data tables or something. It certainly wasn't going to be an option to upload 12 GB of data into their conference software; they mentioned nothing about code, particularly how to specify its dependencies and the computing environment it needed to run in; and it also certainly wasn't going to appear in any form that was convenient to review if I did. How could you submit code to blind review anyway? Do you make alternate versions of all the dependencies that don't credit any authors that overlap with the authors of the paper? In short, because of blind review, I was forbidden from doing anything that would make my paper reproducible at review time.
- kilburn 10y ago
  You are confounding reproducibility and repeatability. Repeatability is the ability to repeat your exact experiment under the same conditions, whereas reproducibility refers to other people being able to reproduce your experiment under similar conditions and obtain congruent results. Of course, both are needed for great science. Nonetheless, your paper alone should provide a good enough description of the conditions and methods you used for the paper to be reproducible. In fact, it can easily be argued that the ability to just run your code instead of re-implementing it according to your description is actually detrimental. To see how, consider that a bug in your code, re-used by other researchers, can easily lead to multiple derivative works finding entirely wrong results. In contrast, if those other researchers re-implemented your method there would be a much lower probability of them doing the same mistake you did, leading to incongruent results and hence raising alarms that probably lead to the discovery of your bug. Although re-implementing your algorithms is significantly more work, the overall quality of our research would benefit from doing so...