11 ms·

Reasons blog posts can be of higher scientific quality than journal articles

233 points by vixen99 10y ago

shouldbworking 10y ago
I feel comment sections on well targeted sites like HN are better than peer review sometimes. It's not uncommon for an industry expert or creator of a product to Koolaid Man into the comment section around here.
- marcosdumay 10y ago
  I didn't stay any long on the academic world, but the comments on targeted tend to be better than nearly all peer review I've seen there. The thing is, the very small exception is incredibly good.
CJefferson 10y ago
One reason blogs are of lower scientific quality -- anyone can make series of large and unresearched claims, cherry picking a few examples to claim a broad point. Good journals do have peer review, and they wouldn't have accepted this. Also, one big disadvantage of blogs -- depending on where you host it, they often don't last long, and point 4 (easy to edit) is a blessing and a curse, particularly if people edit things without telling you they did it. Of course, blogs have their place, I keep one myself, it's great for breaking news, explaining things in greater depth, sharing and understanding. And journals could learn from blogs / arxiv (make it easier to make corrections, allow discussions).
- stinos 10y ago
  anyone I think is they key point here which the OP seems to ignore completely: I sort of agree with everything the OP states, but he also forgets some key aspects on blogging. For instance anyone can post whatever even on subjects he/she really doesn't know a lot about. Also, not every blog post does get reviews from all sides and might only get 'yes good' replies originating solely from confirmation bias while the lack of any criticism doesn't make everything right but for instance just indicates the critics didn't find their way to the blog post.
  - ansgri 10y ago
    There's an inverse side to that "anyone": professional scientists are under pressure to publish long scientific-looking articles, and their fellow reviewers have almost no motivation to spend any significant time reviewing (no pay, no nothing, just a small non-quantifiable reputation penalty for declining to review). Whereas most scientific bloggers post things they are interested in and therefore feel have something to say.
    - stinos 10y ago
      professional scientists are under pressure to publish long scientific-looking articles, and their fellow reviewers have almost no motivation to spend any significant time reviewing You have a point but from all scientists I know myself there's not one for which this applies. Might depend on the field though and I'm not denying there's a bunch of impossible-to-reproduce-crap published and there are rotten apples everywhere. But what I see (in fundamental research which is usually not as publicly known as other kinds) doesn't come near what you describe, even though it's anecdotal of course.. Yes there is pressure to publish but their papers aren't 'scinetific-looking', they're properly scientific and they are just the length needed to describe the findings on the subject. And there is an abundance of motivation for reviewing mostly inspired by wanting to make sure everything is as correct as possible, for the sake of research.
      - ansgri 10y ago
        Thanks for the balancing point. My experience describes applied research in such "hot topics" as computer vision and machine learning, and many people, myself included, are more interested in getting the technology to work and apply it in real commercial projects, where paper publishing may seem like a price to pay for research grants subsidizing your R&D. These grants are aimed at building a working technology and strengthening national economy, so there's nothing wrong with the approach per se, but when your reviewers share your values they have little motivation to find every possible mistake in your paper, at least for second- or third-rate journals (which are still indexed in WoS so are perfectly enough for the funding agency).
  - 10y ago
- shubhamjain 10y ago
  Getting a research paper published sounds so meritorious, no matter how idiotic or trivial it is. Take for example, Princeton 'research' in 2014 that predicted that Facebook will lose 80% of users by 2017 [1]. The evidence? Google trends data which has near-zero correlation with DAUs of Facebook. The research not only got published, but also got a wider-audience after major media websites covered it. Had it been a blog post, I don't think anyone would have taken more than a cursory look at the analysis. Yes, anyone can fit data points and reach a conclusion with blogs, but isn't academic research fraught with similar problems already? [1]: https://www.theguardian.com/technology/2014/jan/22/facebook-princeton-researchers-infectious-disease https://www.theguardian.com/technology/2014/jan/22/facebook-...
  - kilburn 10y ago
    You picked a bad example. The actual source article is published at... arxiv. From an academic perspective it is worth exactly as much as a blog. Academic literature has many problems but it is way above the blogosphere in that regard. And media coverage is going to suck no matter where stuff is published...
    - gpm 10y ago
      > From an academic perspective it is worth exactly as much as a blog. From the perspective of peer review, yes. But per google scholar that paper has been 66 cited times, including in many articles that are in at least somewhat reputable journals (published via Elsevier, ieee, Springer, etc) and apparently two books [0]. So I think it's pretty clear that being a "scientific paper from Princeton" was significant reputation wise regardless of whether or not it was peer reviewed. [0] https://scholar.google.com/scholar?q=Epidemiological%20modeling%20of%20online%20social%20network%20dynamics&btnG=Search&as_sdt=800000000001&as_sdtp=on https://scholar.google.com/scholar?q=Epidemiological%20model... I was also skeptical about the citations so I checked that two with freely available pdfs were actually this paper, they were [1] https://pdfs.semanticscholar.org/4ebb/b654c59aff454ad97301e6ecd9e82c69122b.pdf https://pdfs.semanticscholar.org/4ebb/b654c59aff454ad97301e6... [2] https://arxiv.org/pdf/1509.07805.pdf https://arxiv.org/pdf/1509.07805.pdf
- lucb1e 10y ago
  > depending on where you host it, they often don't last long I wrote my bachelor's thesis last semester. You wouldn't believe for how many papers I needed to use archive.org. By the end of it, I made a small donation and recommended the company to do the same. Blog posts might not be better, but hosted papers aren't that stable either.
- BeetleB 10y ago
  >Good journals do have peer review, and they wouldn't have accepted this. Most peer reviewers do not get to see the data in the article.
  - rspeer 10y ago
    When I submitted a paper to AAAI, there were dire warnings that the paper would be rejected if I included links to any supplemental materials (such as code or data), because this would compromise blind review. The conference software they were using had a sketchy form for uploading supplementary data, but it appeared that they expected it to be used for small data tables or something. It certainly wasn't going to be an option to upload 12 GB of data into their conference software; they mentioned nothing about code, particularly how to specify its dependencies and the computing environment it needed to run in; and it also certainly wasn't going to appear in any form that was convenient to review if I did. How could you submit code to blind review anyway? Do you make alternate versions of all the dependencies that don't credit any authors that overlap with the authors of the paper? In short, because of blind review, I was forbidden from doing anything that would make my paper reproducible at review time.
    - kilburn 10y ago
      You are confounding reproducibility and repeatability. Repeatability is the ability to repeat your exact experiment under the same conditions, whereas reproducibility refers to other people being able to reproduce your experiment under similar conditions and obtain congruent results. Of course, both are needed for great science. Nonetheless, your paper alone should provide a good enough description of the conditions and methods you used for the paper to be reproducible. In fact, it can easily be argued that the ability to just run your code instead of re-implementing it according to your description is actually detrimental. To see how, consider that a bug in your code, re-used by other researchers, can easily lead to multiple derivative works finding entirely wrong results. In contrast, if those other researchers re-implemented your method there would be a much lower probability of them doing the same mistake you did, leading to incongruent results and hence raising alarms that probably lead to the discovery of your bug. Although re-implementing your algorithms is significantly more work, the overall quality of our research would benefit from doing so...
IanCal 10y ago
Unless they're properly archived along with all their resources as well as only referencing other properly archived material, I'd be very cautious before saying they're better. This can be done, but I suspect it is rare whereas it's the norm with a journal article.
- steventhedev 10y ago
  I think they serve different purposes, and the quality varies heavily across subfields. Academic research will produce a journal article as the primary output, whereas a blog post is typically a side effect of a practical project, and most will contain a link to the actual code. Ultimately, I think they're both good, but they do have different uses.
  - IanCal 10y ago
    I'm still highly concerned about the loss of these things. I very regularly follow links to code that's just gone. The code for research definitely should not only exist in a related of post link. Archiving, for me, is a requirement if you think it should be used over more than a year or so.
    - steventhedev 10y ago
      Obviously, having a professional librarian working on archiving data, code, and articles is ideal. However, I've found that the public facing side of journals experience bit rot even faster than most of the older blogs. Of course, there are exceptions, and even the most public figures can decide to delete everything, code included (e.g. _why) Gwern has tackled link rot[0] in an exhaustive way, and may give you a starting point from where to continue. [0]: https://www.gwern.net/Archiving%20URLs https://www.gwern.net/Archiving%20URLs
    - afandian 10y ago
      Zenodo is a good example of this, as is Figshare. They allow the assignment of a DataCite DOI to some kind of digital asset (dataset, code, github repo etc) for citation https://zenodo.org/ https://zenodo.org/
soufron 10y ago
Can't they?
JohnStrange 10y ago
In my opinion blog posts have no scientific quality at all, since they are not peer reviewed. They may occasionally have quality, but that's another matter. To be fair, some fringe journals have no scientific value either, because they do not peer review properly, but they are easy to spot and everybody in the scientific community knows them. On a side note, originality is one of the key requirements for publication in a reputed journal and I have not ever seen a blog post that made any scientifically original claims. But maybe that's because I don't read blogs very often.
- maaaats 10y ago
  https://en.wikipedia.org/wiki/Polymath_Project https://en.wikipedia.org/wiki/Polymath_Project Bunch of mathematicians collaborating through blogs.
baali 10y ago
One more related link on the same subject, "The top 10 reasons why blog posts are better than scientific papers": http://ivory.idyll.org/blog/2017-top-ten-reasons-blog-posts.html http://ivory.idyll.org/blog/2017-top-ten-reasons-blog-posts.... * there is a "maybe satire" tag added by the author to this post.
afandian 10y ago
Can anyone recommend scholarly / academic blogs (or aggregators)?
- pjc50 10y ago
  http://blogs.sciencemag.org/pipeline/ http://blogs.sciencemag.org/pipeline/ "In The Pipeline" is good and has been running for years.
  - afandian 10y ago
    Thanks! Looks great.
afandian 10y ago
(to repeat a comment I made on the blog) We're seeing a huge amount of non-traditional scholarly activity (if you'll pardon the dry phrase) happening in blogs. Alternative metrics aka altmetrics (as distinct from altmetric.com) have been taking in to consideration the scholarly activity that happens around traditional publishing for a while. Crossref, the organisation who brought you DOIs for scholarly publications, thought that it would be a good idea to help collect this kind of data as a counterpoint to traditional publishing and citations. We're building Crossref Event Data, which is a free (libre, gratis) service for collecting mentions of articles on blogs and social media, so that it can be used by the community in all kinds of ways. Discoverability, recommendations, and yes, maybe more metrics. How you use it is up to you. The article raises a good point about blogs as primary methods of publishing rather than, for example, as a venue for the discussion of traditionally published articles. Establishing an open 'citation'-graph-like-dataset of blogs is a good first step toward that. We're heading into Beta soon, and you can read more about it https://www.crossref.org/services/event-data https://www.crossref.org/services/event-data . The User guide is a work in progress, but might answer any questions you have: https://www.eventdata.crossref.org/guide https://www.eventdata.crossref.org/guide . You can also contact me at jwass@crossref.org if you have any questions.
rechecker 10y ago
"Why can... " Uh, shouldn't that be "How can... "?
- scottmf 10y ago
  Why?
notahacker 10y ago
Advantages that scientific journal articles have over blogs is that they are: (i) not written in clickbait format (ii) impact is measured in terms of other research which sees it as a relevant study to cite rather than numbers of pageviews (see also (i)) (iii) one of the things a peer review process can flag up is that bold and implausible claims like "On blogs, the norm is to provide access to the underlying data, code, and materials" probably need some form of qualification for what is meant by "blogs" and/or quantitative evidence to support that claim Blogs can be well reasoned and evidence-based, comments can (even more infrequently) be enlightening and journal articles aren't immune to flaws, but the linked blog article is a prime example of why arguments expressed in blogs are seldom afforded the respect of arguments advanced in journals.
- lucb1e 10y ago
  > (i) not written in clickbait format Really? I find them to be incredibly clickbaity whereas blog posts are much more matter-of-fact. Blogs are almost never a commercial entity/enterprise and while no writer would shun search engine traffic, they usually write for peers, not for publicity like journalists or scientists, both whose funding depends on this.
  - notahacker 10y ago
    I have yet to see the stereotypical "5 reasons why [bold claim involving zero research]" format the linked blog post uses uses in any academic journal. Scientific writing in closed-access journals is the definition of writing for peers, and never includes AdSense, affiliate links or digital tip jars.
pdkl95 10y ago
For some time I've thought that the concept of a "journal" artificially forces together several features that can exist independently. This usually includes something roughly similar to: (not a complete list, or in any particular order) 1. people that want to publish their research, theories, response, etc 2. a mechanism for having submissions peer reviewed 3. curating which submissions will be included in the publication 4. actually physically (or electronically) publishing and disseminating the papers to the interested community 5. people actually reading it and responding (possibly recursively) 6. reputation and status derived from authorship, being referenced by others, etc 7. archiving the papers - and hopefully the data These features are traditionally part of the same process for practical reasons like the cost/time to physically publish before modern printing technology. This worked, creating resistance to any change. Now, I suspect that over reliance on the concept of a "journal" limits your thinking. This article demonstrates some of that with it's framing of blogs as an alternative or competitor to journals. Instead, consider that modern computing and the internet make #4 very easy and almost free. We have various ways to archive (#7), which include actual verification (e.g. "git fsck", signed commits). We already curate (#3) as a separate step with specialty blogs and aggregators like HN. I'm not saying journals are bad or obsolete. I'm just suggesting that there might be better ways to organize the process, and that different granularities of "journal"-like process can probably coexist.
- BHSPitMonkey 10y ago
  Many of these topics are constantly being championed by the somewhat obscure digital libraries space. These departments are mainly found in universities, and they focus heavily on the pipeline you described. Despite the point you made that throwing content on the internet is usually easy and "almost free", it's actually a pretty large and costly effort when you're trying to do it at scale. A lot of clerical and engineering effort goes into tagging/organizing/storing/replicating the data, guarding against bit rot and other integrity errors, and making the data easily searchable and browsable (and that's to say nothing of the political and psychological battles involved in simply getting researchers to follow through and provide the data to these processes, in the proper formats, before they finish their degree and disappear). If the hacker side of these challenges interest you, I suggest checking out the code4lib community: https://code4lib.org https://code4lib.org
pjc50 10y ago
"Impact factor" systems incentivise the production of poor quality journal articles. Effectively researchers end up needing to produce a certain amount of "journal-bait" in order to stay funded. (See HN passim "Every attempt to manage academia makes it worse")
lr4444lr 10y ago
The author calls "Nature" a "low quality journal". Is that really it's reputation?
- stinos 10y ago
  Depends on who you ask probably, but in the scientific world it's generally not regarded as low quality, rather the opposite. Also it's impact factor suggests that. Though the way that is calculated is also under debate by some, I'd say it still gives an indication of quality in general.
- collyw 10y ago
  No. For biology at least it is the one to aim for.
- capnrefsmmat 10y ago
  Its reputation among people like Lakens, and others complaining about the poor state of statistics and rigor in modern science, is that it features overinterpreted overhyped results from small underpowered experiments, and when errors are pointed out it's usually not too interested in correcting them. The editors are more interested in "important" results than methodologically rigorous ones. It's similar to how Andrew Gelman always sarcastically refers to the Proceedings of the National Academy of Sciences as the Prestigious Proceedings, because that's invariably the adjective used in news reports, yet many of the articles are of the "female-named hurricanes cause more damage" type.
- mattmcknight 10y ago
  "Nature" comes in for a lot of criticism because they have a news division that publicizes results widely, yet does very little when those results are retracted...and they are corrected or retracted increasingly frequently: http://retractionwatch.com/category/by-journal/nature-retractions/ http://retractionwatch.com/category/by-journal/nature-retrac...
- deleted 10y ago
  [deleted]
- toufka 10y ago
deleted 10y ago
[deleted]
logicallee 10y ago
They left out the biggest one! If you say something dumb in it, the comments will crucify you for it :)
- daxelrod 10y ago
  And also if you say something controversial.
  - logicallee 10y ago
    What you point out is an interesting effect. Blog posts that are controversial know that there's a chance a sh$#storm will ensue in the comments. In the case of general bloggers, such as researchers i.e. not people who are actively trying to troll or generate pageviews, but on personal blogs, how does this affect writing that is controversial? I don't know academic publishing well enough to compare. I think that as far as controversial claims that would make it to somewhere like Hacker News, the effect is that if the controversial statements are wrong, then the comments will shoot them down, and the story will be buried. Furthermore, the rest of the comments will pile on the first comment that points out that the blog post is mistaken. Perhaps an effect of this is that you can tell from the tone that some blog posts take, that they are being extremely careful to lay a rock-solid foundation, and kind of "convert" their readers who might otherwise vehemently disagree with them. For a recent example, check out this blog post: https://gregfallis.com/2017/04/14/seriously-the-guy-has-a-point/ https://gregfallis.com/2017/04/14/seriously-the-guy-has-a-po... I actually just noticed something really interesting! The very first words in this blog post are: "I got metaphorically spanked a couple of days ago." Would any academic article on any subject start in this way? So the effect of the backlash against controversial statements, on the writing style of blog posts, is interesting and in many cases highly visible. It is hard for me to compare this with academic writing. (But it certainly wouldn't start with words like that.)
aviggiano 10y ago
It seems the future lies in something like GitHub, stack overflow or quora for articles, where people with the best code/answers naturally gain more reputation and where you still have an incentive for peer review and open data.
ChicagoBoy11 10y ago
I'm surprised that so many of the comments seem to attack the claims in the post along the dimension of "well, anyone can write a blog post." I think as a community HN would staunchly advocate that the recipe for building great companies or software is an almost radical devotion to reducing any and all barriers to entry, except in this domain somehow people feel that the rigidity and exclusivity which journals provide somehow lead to the betterment of science. Makes me think about pg's "what you can't say" post. There is short shrift given to the distortion that gets introduced in order to "get published". The "file drawer" effect is very real, and it gives us a very skewed view of what the actual scientific work being done really is; p-hacking and other statistical trickery are table stakes to anyone trying to survive in the academic world, as being published is a key metric to success yet it is surprisingly orthogonal to being right. As Brian Nosek's "reproducibility project" is uncovering, there is a non-trivial amount of "science" out there that is pure garbage, and much of it was produced in order to "get published." In a world where "random hackers" can write twitter bots to parse scientific papers and uncover obvious mathematical flaws (which sometimes invalidate central claims of the paper), I don't understand how we aren't immediately gravitating to standards which promote more and more openness. Lastly, the biggest issue we have to address is the fact that notoriety is now the chief indicator of success, and not genuine scientific discovery. In college I had world-famous faculty who have made a career, fortune, and placed themselves well on the spotlight on the backs of research which was later completely reversed -- and not in the way "science is supposed to work" sense, but rather in the "you purposefully fudged the math" sense. And yet there was virtually no reputational damage done to their career. Worse yet, I think it is clear that this lack of ethics actually is a direct cause of the success they enjoy today. Given that these are pressures faced by all in the academic community, true openness is the one avenue we have to counteract this sort of thing, and hopefully enterprising individuals come along and find a way to make "truth-seeking" reputation something researchers care about.
- pc86 10y ago
  That's because a lot of folks on HN come from Harvard or Stanford or MIT, where it's drilled into you that this pedigree is super important. There are companies that will not hire you for an engineering role if you don't have a CS degree from a short list of universities unless you're part of an acquisition (and even then). This gets ignored the second they get $800k in VC money but for some reason the same idea never translates to "real science" or academia.
agumonkey 10y ago
There's a lot of publications. Some of them are really small step attempts at understanding. Sometimes it's just a test. A careful one, but nothing more. I guess in some fields we're still at the poking level.
dahart 10y ago
> Blogs have Open Peer Review No, they don't. This is conflating having an opinion with the scientific review process. The scientific review process can prevent publication of bad research in journals and require changes before publication, the blog "review" process cannot do either of these things.
tkt 10y ago
Titus Brown also a blog post on this earlier this year "The top 10 reasons why blog posts are better than scientific papers" http://ivory.idyll.org/blog/2017-top-ten-reasons-blog-posts.html http://ivory.idyll.org/blog/2017-top-ten-reasons-blog-posts.... and it generated similar discussion. There are a few elements they emphasize. One is what the blog format enables that traditional publishing doesn't support. Those are things like having real-time feedback and comments, being able to version and making the blog post interactive, rather than a static document. Another element to the format is a lack of gatekeepers, so it can be quickly disseminated and disseminated by anyone, so there aren't barriers to participation in the scientific discourse. Another is norms and expectations. In blogging, it is more the norm that data and code are open. Open is still possible in traditional publishing; it just isn't yet the norm. A new format however, enables new norms and it's easier to set them from the start, than try to revise existing ones. Finally, there's the element of 'correctness'. Going through peer review and being in a traditional journal certainly doesn't ensure that the paper is correct. You can just look at retractions to see that http://retractionwatch.com/2011/08/11/is-it-time-for-a-retraction-index/ http://retractionwatch.com/2011/08/11/is-it-time-for-a-retra.... However it would be interesting to see more evidence around whether the blog format does ultimately lead to more 'correct' conclusions, on the whole, and not just for the posts that lead to a lot of discussion.
leecarraher 10y ago
blog post on blogspot advocating that blogs are better than journals seems a bit biased, unfortunately i don't think i can get a journal article about journal articles being better, accepted in any peer reviewed journals. checkmate daniel.
mrdrozdov 10y ago
Is it so hard to publish at arxiv, share the paper on twitter, and submit a link to HN? At a minimum, arxiv is better than a blog post because there's some sense of versioning. For anyone who is curious, some conferences share the peer review comments (even though they do not share the identities of author or commenter). Here are the comments from iclr2017 (https://openreview.net/group?id=ICLR.cc/2017/conference https://openreview.net/group?id=ICLR.cc/2017/conference) and here are comments on Ian Goodfellow's GAN paper by Jürgen Schmidhuber and others (https://media.nips.cc/nipsbooks/nipspapers/paper_files/nips27/reviews/1384.html https://media.nips.cc/nipsbooks/nipspapers/paper_files/nips2...).
colorincorrect 10y ago
As in, "if you follow these best practices, you too can produce high quality research" There is no reason why this post should be focused on blogs in general, asides from generating discussion + clickbait. The key here is actually following the best practice, but do most blogs actually do this? Alternatively: there is a high minimal requirement for research articles, but there is no minimal requirement for blogs.
Spooky23 10y ago
IMO This is one of those cases where people are committed to an existing system and have bent their thinking around that system to the point that they cannot see the big picture. There's alot of value to the existing journal system, but perhaps it would be appropriate to have a more formal process and a more informal/collaborative process. Maybe that would address the problems with having so many journals.
mikk14 10y ago
People here seem to attack only the argument about "anyone can write a blog", but I was expecting HN to give a couple of thoughts also about his first point. To me it reads something like: "publish all your data out in the open for everybody to see". I find it extremely problematic. Sure, sharing data is necessary to verify that the conclusions are supported by it and they are not due to methodological errors. But to anyone? Out in the open? I would expect better data ethics, especially from a psychologist. Your data can contain political opinions, health records, sexual orientation, contact information, the places a person has visited, and when. People sign up for these studies usually with the agreement that the information that can harm them cannot be freely shared, unless to people involved in studies with similar data protection systems. Data like I mentioned has sometimes to be stored in computers not connected to the Internet, to reduce the risk of data leak. Free access to this data paves the way to persecution and shaming. If I were to sign up as subject to a psychology study and had a person with his ideas leading it, I'd withdraw immediately. I'd question if this person should be a psychology researcher at all. Sharing data is good, but protocols are there for a reason.
- metalliqaz 10y ago
  Personally identifiable details can be removed from data.
  - Jweb_Guru 10y ago
    They actually usually cannot, especially if you want to still be able to do research into them (which is the motivation behind differential privacy).
  - mikk14 10y ago
    There is a seminal paper showing how carelessly adopting this point of view can lead to disasters: "87% of the U.S. population is uniquely identified by date of birth, gender, postal code" [1] In many other cases, as pointed out, it's just not possible. I'm studying mobility patterns through cellphone metadata. Even if you strip out the actual phone number with a random ID you still know where a person is going, and thus re-identify them if you have other public data. [1] https://en.wikipedia.org/wiki/Latanya_Sweeney https://en.wikipedia.org/wiki/Latanya_Sweeney
88e282102ae2e5b 10y ago
I think it might be better to say that contemporary psychology is frequently not being performed scientifically.
westurner 10y ago
So, schema.org, has classes (C:) -- subclasses of CreativeWork and Article -- for property (P:) domains (D:) and ranges (R:) which cover this domain: - CreativeWork: http://schema.org/CreativeWork http://schema.org/CreativeWork - - BlogPosting: http://schema.org/BlogPosting http://schema.org/BlogPosting - - Article: http://schema.org/Article http://schema.org/Article - - - NewsArticle: http://schema.org/NewsArticle http://schema.org/NewsArticle - - - Report: http://schema.org/Report http://schema.org/Report - - - ScholarlyArticle: http://schema.org/ScholarlyArticle http://schema.org/ScholarlyArticle - - - SocialMediaPosting: http://schema.org/SocialMediaPosting http://schema.org/SocialMediaPosting - - - TechArticle: http://schema.org/TechArticle http://schema.org/TechArticle Thing: (name, [url], [identifier], [#about], [description[_gh_markdown_html]]) - C: CreativeWork: - - P: comment R: Comment - - C: Comment: https://schema.org/Comment https://schema.org/Comment
4258HzG 10y ago
#1 Reason: Blogs aren't included in the publication count which is needed to gain research funding, get hired as a professor and get tenure. A bit cynical, but a lot of the problems the writer suggests are more of a symptom of trying to get as many papers out as possible than the cause (and that one rarely reviews the supplementary material). If scientific blogs counted like publications many of the same problems would appear there.
ylem 10y ago
I can't comment on the field of psychiatry. However, I will comment on my field of physics. The author seems to completely neglect the signal to noise problem. There has been an explosion of journals and scientific publication. It's rather hard to keep up to date on the literature and editors do provide a valuable function in filtering what they think is interesting (which is further filtered by the referees). Also, for the referee process--it's not perfect and will probably not catch complete fraud. But, it is good for checking the basics of whether or not the story being told in a paper is consistent. Also, again related to signal/noise, the editors hopefully choose referees with a minimum level of competence in the field (not always, but usually). Also, in terms of dilution--if there's a blog, there's no guarantee that a critical mass of people will read it to try to give it a critical review. Whereas, even for a journal that is not widely read, the papers in it will have received some peer review where someone in the field tried to take a look through it for obvious errors. I am not convinced by open peer review. Let's consider two scenarios. Suppose there is a leader in the field trying to publish something (or perhaps even a friend or collaborator). If I notice that they are wrong, am I more likely to call them on it if I am anonymous or if they know who I am? One can be idealistic, but researchers are people too. I think that the anonymous review allows people to speak frankly (though it does sometimes let people be mean--that seems to be the cost of anonymity in forums unless there is strong moderation). Let's consider another scenario. Suppose I am a junior researcher and I get a referee comment that I determine is wrong. If it comes from the leader in my field, maybe I won't push back as hard as if it came from someone else junior--again, in an ideal world, it wouldn't matter, but researchers are people too. It would seem that the editor knowing the identity of the referees would serve as a check on bias, not 100 percent, but I think that more is gained than is lost from a closed referee system. In terms of error correction, there are errata that are published if someone finds an error in a major study and in journals like the physical review are linked back to the original article. The open data problem is hard. In my particular field, our raw data is available on the web, but the problem is that the meta-data needed to interpret is not. And that's nontrivial. For example, I may perform several operations on my data (for example, background subtraction) before fitting it to a model. We are experimenting with dataflow languages where we embed the series of "filters" that we apply to the data, but if we want someone to be able to reproduce this 20 years from now, then there's a whole ecosystem that would have to be maintained--for example, not just my code, but all of the libraries that it depends on to run. We can describe the basic process in the paper, but for true long term reproducibility, it's a hard problem...There are groups that are working on open-data and reproducible research, so I don't think that there isn't interest, just that it's not as easy a problem as you might think. But, for reduced data, I agree, it would be nice to have that available for papers in a machine readable format... Open access. This is also a hard problem. Increasingly, funding agencies are requiring that publications be made available in an open format after some embargo period. I think that may be the best we can hope for. Just paying for competent editors requires funding. If we want additional features like data attached to papers (for decades), it will take more funding. That money has to come from somewhere. For typical open access journals, the author pays, but that seems to create difficult incentives--not to mention that it makes it difficult for poorer funded researchers to publish. I'm not sure what the right answer is, because I agree that the public should be able to see the results that they paid for--perhaps an embargo period is the best solution... I think a blog is a great way for communication of ideas and for education, but I do not think that it is able to replace publication in a refereed journal.

233 points by vixen99 10y ago

I feel comment sections on well targeted sites like HN are better than peer review sometimes. It's not uncommon for an industry expert or creator of a product to Koolaid Man into the comment section around here.

marcosdumay 10y ago
I didn't stay any long on the academic world, but the comments on targeted tend to be better than nearly all peer review I've seen there. The thing is, the very small exception is incredibly good.

CJefferson 10y ago

One reason blogs are of lower scientific quality -- anyone can make series of large and unresearched claims, cherry picking a few examples to claim a broad point. Good journals do have peer review, and they wouldn't have accepted this. Also, one big disadvantage of blogs -- depending on where you host it, they often don't last long, and point 4 (easy to edit) is a blessing and a curse, particularly if people edit things without telling you they did it. Of course, blogs have their place, I keep one myself, it's great for breaking news, explaining things in greater depth, sharing and understanding. And journals could learn from blogs / arxiv (make it easier to make corrections, allow discussions).

stinos 10y ago
anyone I think is they key point here which the OP seems to ignore completely: I sort of agree with everything the OP states, but he also forgets some key aspects on blogging. For instance anyone can post whatever even on subjects he/she really doesn't know a lot about. Also, not every blog post does get reviews from all sides and might only get 'yes good' replies originating solely from confirmation bias while the lack of any criticism doesn't make everything right but for instance just indicates the critics didn't find their way to the blog post.
- ansgri 10y ago
  There's an inverse side to that "anyone": professional scientists are under pressure to publish long scientific-looking articles, and their fellow reviewers have almost no motivation to spend any significant time reviewing (no pay, no nothing, just a small non-quantifiable reputation penalty for declining to review). Whereas most scientific bloggers post things they are interested in and therefore feel have something to say.
  - stinos 10y ago
    professional scientists are under pressure to publish long scientific-looking articles, and their fellow reviewers have almost no motivation to spend any significant time reviewing You have a point but from all scientists I know myself there's not one for which this applies. Might depend on the field though and I'm not denying there's a bunch of impossible-to-reproduce-crap published and there are rotten apples everywhere. But what I see (in fundamental research which is usually not as publicly known as other kinds) doesn't come near what you describe, even though it's anecdotal of course.. Yes there is pressure to publish but their papers aren't 'scinetific-looking', they're properly scientific and they are just the length needed to describe the findings on the subject. And there is an abundance of motivation for reviewing mostly inspired by wanting to make sure everything is as correct as possible, for the sake of research.
    - ansgri 10y ago
      Thanks for the balancing point. My experience describes applied research in such "hot topics" as computer vision and machine learning, and many people, myself included, are more interested in getting the technology to work and apply it in real commercial projects, where paper publishing may seem like a price to pay for research grants subsidizing your R&D. These grants are aimed at building a working technology and strengthening national economy, so there's nothing wrong with the approach per se, but when your reviewers share your values they have little motivation to find every possible mistake in your paper, at least for second- or third-rate journals (which are still indexed in WoS so are perfectly enough for the funding agency).
- 10y ago
shubhamjain 10y ago
Getting a research paper published sounds so meritorious, no matter how idiotic or trivial it is. Take for example, Princeton 'research' in 2014 that predicted that Facebook will lose 80% of users by 2017 [1]. The evidence? Google trends data which has near-zero correlation with DAUs of Facebook. The research not only got published, but also got a wider-audience after major media websites covered it. Had it been a blog post, I don't think anyone would have taken more than a cursory look at the analysis. Yes, anyone can fit data points and reach a conclusion with blogs, but isn't academic research fraught with similar problems already? [1]: https://www.theguardian.com/technology/2014/jan/22/facebook-princeton-researchers-infectious-disease https://www.theguardian.com/technology/2014/jan/22/facebook-...
- kilburn 10y ago
  You picked a bad example. The actual source article is published at... arxiv. From an academic perspective it is worth exactly as much as a blog. Academic literature has many problems but it is way above the blogosphere in that regard. And media coverage is going to suck no matter where stuff is published...
  - gpm 10y ago
    > From an academic perspective it is worth exactly as much as a blog. From the perspective of peer review, yes. But per google scholar that paper has been 66 cited times, including in many articles that are in at least somewhat reputable journals (published via Elsevier, ieee, Springer, etc) and apparently two books [0]. So I think it's pretty clear that being a "scientific paper from Princeton" was significant reputation wise regardless of whether or not it was peer reviewed. [0] https://scholar.google.com/scholar?q=Epidemiological%20modeling%20of%20online%20social%20network%20dynamics&btnG=Search&as_sdt=800000000001&as_sdtp=on https://scholar.google.com/scholar?q=Epidemiological%20model... I was also skeptical about the citations so I checked that two with freely available pdfs were actually this paper, they were [1] https://pdfs.semanticscholar.org/4ebb/b654c59aff454ad97301e6ecd9e82c69122b.pdf https://pdfs.semanticscholar.org/4ebb/b654c59aff454ad97301e6... [2] https://arxiv.org/pdf/1509.07805.pdf https://arxiv.org/pdf/1509.07805.pdf
lucb1e 10y ago
> depending on where you host it, they often don't last long I wrote my bachelor's thesis last semester. You wouldn't believe for how many papers I needed to use archive.org. By the end of it, I made a small donation and recommended the company to do the same. Blog posts might not be better, but hosted papers aren't that stable either.
BeetleB 10y ago
>Good journals do have peer review, and they wouldn't have accepted this. Most peer reviewers do not get to see the data in the article.
- rspeer 10y ago
  When I submitted a paper to AAAI, there were dire warnings that the paper would be rejected if I included links to any supplemental materials (such as code or data), because this would compromise blind review. The conference software they were using had a sketchy form for uploading supplementary data, but it appeared that they expected it to be used for small data tables or something. It certainly wasn't going to be an option to upload 12 GB of data into their conference software; they mentioned nothing about code, particularly how to specify its dependencies and the computing environment it needed to run in; and it also certainly wasn't going to appear in any form that was convenient to review if I did. How could you submit code to blind review anyway? Do you make alternate versions of all the dependencies that don't credit any authors that overlap with the authors of the paper? In short, because of blind review, I was forbidden from doing anything that would make my paper reproducible at review time.
  - kilburn 10y ago
    You are confounding reproducibility and repeatability. Repeatability is the ability to repeat your exact experiment under the same conditions, whereas reproducibility refers to other people being able to reproduce your experiment under similar conditions and obtain congruent results. Of course, both are needed for great science. Nonetheless, your paper alone should provide a good enough description of the conditions and methods you used for the paper to be reproducible. In fact, it can easily be argued that the ability to just run your code instead of re-implementing it according to your description is actually detrimental. To see how, consider that a bug in your code, re-used by other researchers, can easily lead to multiple derivative works finding entirely wrong results. In contrast, if those other researchers re-implemented your method there would be a much lower probability of them doing the same mistake you did, leading to incongruent results and hence raising alarms that probably lead to the discovery of your bug. Although re-implementing your algorithms is significantly more work, the overall quality of our research would benefit from doing so...

IanCal 10y ago

Unless they're properly archived along with all their resources as well as only referencing other properly archived material, I'd be very cautious before saying they're better. This can be done, but I suspect it is rare whereas it's the norm with a journal article.

steventhedev 10y ago
I think they serve different purposes, and the quality varies heavily across subfields. Academic research will produce a journal article as the primary output, whereas a blog post is typically a side effect of a practical project, and most will contain a link to the actual code. Ultimately, I think they're both good, but they do have different uses.
- IanCal 10y ago
  I'm still highly concerned about the loss of these things. I very regularly follow links to code that's just gone. The code for research definitely should not only exist in a related of post link. Archiving, for me, is a requirement if you think it should be used over more than a year or so.
  - steventhedev 10y ago
    Obviously, having a professional librarian working on archiving data, code, and articles is ideal. However, I've found that the public facing side of journals experience bit rot even faster than most of the older blogs. Of course, there are exceptions, and even the most public figures can decide to delete everything, code included (e.g. _why) Gwern has tackled link rot[0] in an exhaustive way, and may give you a starting point from where to continue. [0]: https://www.gwern.net/Archiving%20URLs https://www.gwern.net/Archiving%20URLs
  - afandian 10y ago
    Zenodo is a good example of this, as is Figshare. They allow the assignment of a DataCite DOI to some kind of digital asset (dataset, code, github repo etc) for citation https://zenodo.org/ https://zenodo.org/

soufron 10y ago

Can't they?

JohnStrange 10y ago

In my opinion blog posts have no scientific quality at all, since they are not peer reviewed. They may occasionally have quality, but that's another matter. To be fair, some fringe journals have no scientific value either, because they do not peer review properly, but they are easy to spot and everybody in the scientific community knows them. On a side note, originality is one of the key requirements for publication in a reputed journal and I have not ever seen a blog post that made any scientifically original claims. But maybe that's because I don't read blogs very often.

maaaats 10y ago
https://en.wikipedia.org/wiki/Polymath_Project https://en.wikipedia.org/wiki/Polymath_Project Bunch of mathematicians collaborating through blogs.

baali 10y ago

One more related link on the same subject, "The top 10 reasons why blog posts are better than scientific papers": http://ivory.idyll.org/blog/2017-top-ten-reasons-blog-posts.html http://ivory.idyll.org/blog/2017-top-ten-reasons-blog-posts.... * there is a "maybe satire" tag added by the author to this post.

afandian 10y ago

Can anyone recommend scholarly / academic blogs (or aggregators)?

pjc50 10y ago
http://blogs.sciencemag.org/pipeline/ http://blogs.sciencemag.org/pipeline/ "In The Pipeline" is good and has been running for years.
- afandian 10y ago
  Thanks! Looks great.

afandian 10y ago

(to repeat a comment I made on the blog) We're seeing a huge amount of non-traditional scholarly activity (if you'll pardon the dry phrase) happening in blogs. Alternative metrics aka altmetrics (as distinct from altmetric.com) have been taking in to consideration the scholarly activity that happens around traditional publishing for a while. Crossref, the organisation who brought you DOIs for scholarly publications, thought that it would be a good idea to help collect this kind of data as a counterpoint to traditional publishing and citations. We're building Crossref Event Data, which is a free (libre, gratis) service for collecting mentions of articles on blogs and social media, so that it can be used by the community in all kinds of ways. Discoverability, recommendations, and yes, maybe more metrics. How you use it is up to you. The article raises a good point about blogs as primary methods of publishing rather than, for example, as a venue for the discussion of traditionally published articles. Establishing an open 'citation'-graph-like-dataset of blogs is a good first step toward that. We're heading into Beta soon, and you can read more about it https://www.crossref.org/services/event-data https://www.crossref.org/services/event-data . The User guide is a work in progress, but might answer any questions you have: https://www.eventdata.crossref.org/guide https://www.eventdata.crossref.org/guide . You can also contact me at jwass@crossref.org if you have any questions.

rechecker 10y ago

"Why can... " Uh, shouldn't that be "How can... "?

scottmf 10y ago
Why?

notahacker 10y ago

Advantages that scientific journal articles have over blogs is that they are: (i) not written in clickbait format (ii) impact is measured in terms of other research which sees it as a relevant study to cite rather than numbers of pageviews (see also (i)) (iii) one of the things a peer review process can flag up is that bold and implausible claims like "On blogs, the norm is to provide access to the underlying data, code, and materials" probably need some form of qualification for what is meant by "blogs" and/or quantitative evidence to support that claim Blogs can be well reasoned and evidence-based, comments can (even more infrequently) be enlightening and journal articles aren't immune to flaws, but the linked blog article is a prime example of why arguments expressed in blogs are seldom afforded the respect of arguments advanced in journals.

lucb1e 10y ago
> (i) not written in clickbait format Really? I find them to be incredibly clickbaity whereas blog posts are much more matter-of-fact. Blogs are almost never a commercial entity/enterprise and while no writer would shun search engine traffic, they usually write for peers, not for publicity like journalists or scientists, both whose funding depends on this.
- notahacker 10y ago
  I have yet to see the stereotypical "5 reasons why [bold claim involving zero research]" format the linked blog post uses uses in any academic journal. Scientific writing in closed-access journals is the definition of writing for peers, and never includes AdSense, affiliate links or digital tip jars.

pdkl95 10y ago

For some time I've thought that the concept of a "journal" artificially forces together several features that can exist independently. This usually includes something roughly similar to: (not a complete list, or in any particular order) 1. people that want to publish their research, theories, response, etc 2. a mechanism for having submissions peer reviewed 3. curating which submissions will be included in the publication 4. actually physically (or electronically) publishing and disseminating the papers to the interested community 5. people actually reading it and responding (possibly recursively) 6. reputation and status derived from authorship, being referenced by others, etc 7. archiving the papers - and hopefully the data These features are traditionally part of the same process for practical reasons like the cost/time to physically publish before modern printing technology. This worked, creating resistance to any change. Now, I suspect that over reliance on the concept of a "journal" limits your thinking. This article demonstrates some of that with it's framing of blogs as an alternative or competitor to journals. Instead, consider that modern computing and the internet make #4 very easy and almost free. We have various ways to archive (#7), which include actual verification (e.g. "git fsck", signed commits). We already curate (#3) as a separate step with specialty blogs and aggregators like HN. I'm not saying journals are bad or obsolete. I'm just suggesting that there might be better ways to organize the process, and that different granularities of "journal"-like process can probably coexist.

BHSPitMonkey 10y ago
Many of these topics are constantly being championed by the somewhat obscure digital libraries space. These departments are mainly found in universities, and they focus heavily on the pipeline you described. Despite the point you made that throwing content on the internet is usually easy and "almost free", it's actually a pretty large and costly effort when you're trying to do it at scale. A lot of clerical and engineering effort goes into tagging/organizing/storing/replicating the data, guarding against bit rot and other integrity errors, and making the data easily searchable and browsable (and that's to say nothing of the political and psychological battles involved in simply getting researchers to follow through and provide the data to these processes, in the proper formats, before they finish their degree and disappear). If the hacker side of these challenges interest you, I suggest checking out the code4lib community: https://code4lib.org https://code4lib.org

pjc50 10y ago

"Impact factor" systems incentivise the production of poor quality journal articles. Effectively researchers end up needing to produce a certain amount of "journal-bait" in order to stay funded. (See HN passim "Every attempt to manage academia makes it worse")

lr4444lr 10y ago

The author calls "Nature" a "low quality journal". Is that really it's reputation?

stinos 10y ago
Depends on who you ask probably, but in the scientific world it's generally not regarded as low quality, rather the opposite. Also it's impact factor suggests that. Though the way that is calculated is also under debate by some, I'd say it still gives an indication of quality in general.
collyw 10y ago
No. For biology at least it is the one to aim for.
capnrefsmmat 10y ago
Its reputation among people like Lakens, and others complaining about the poor state of statistics and rigor in modern science, is that it features overinterpreted overhyped results from small underpowered experiments, and when errors are pointed out it's usually not too interested in correcting them. The editors are more interested in "important" results than methodologically rigorous ones. It's similar to how Andrew Gelman always sarcastically refers to the Proceedings of the National Academy of Sciences as the Prestigious Proceedings, because that's invariably the adjective used in news reports, yet many of the articles are of the "female-named hurricanes cause more damage" type.
mattmcknight 10y ago
"Nature" comes in for a lot of criticism because they have a news division that publicizes results widely, yet does very little when those results are retracted...and they are corrected or retracted increasingly frequently: http://retractionwatch.com/category/by-journal/nature-retractions/ http://retractionwatch.com/category/by-journal/nature-retrac...
deleted 10y ago
[deleted]
toufka 10y ago

deleted 10y ago

[deleted]

logicallee 10y ago

They left out the biggest one! If you say something dumb in it, the comments will crucify you for it :)

daxelrod 10y ago
And also if you say something controversial.
- logicallee 10y ago
  What you point out is an interesting effect. Blog posts that are controversial know that there's a chance a sh$#storm will ensue in the comments. In the case of general bloggers, such as researchers i.e. not people who are actively trying to troll or generate pageviews, but on personal blogs, how does this affect writing that is controversial? I don't know academic publishing well enough to compare. I think that as far as controversial claims that would make it to somewhere like Hacker News, the effect is that if the controversial statements are wrong, then the comments will shoot them down, and the story will be buried. Furthermore, the rest of the comments will pile on the first comment that points out that the blog post is mistaken. Perhaps an effect of this is that you can tell from the tone that some blog posts take, that they are being extremely careful to lay a rock-solid foundation, and kind of "convert" their readers who might otherwise vehemently disagree with them. For a recent example, check out this blog post: https://gregfallis.com/2017/04/14/seriously-the-guy-has-a-point/ https://gregfallis.com/2017/04/14/seriously-the-guy-has-a-po... I actually just noticed something really interesting! The very first words in this blog post are: "I got metaphorically spanked a couple of days ago." Would any academic article on any subject start in this way? So the effect of the backlash against controversial statements, on the writing style of blog posts, is interesting and in many cases highly visible. It is hard for me to compare this with academic writing. (But it certainly wouldn't start with words like that.)

aviggiano 10y ago

It seems the future lies in something like GitHub, stack overflow or quora for articles, where people with the best code/answers naturally gain more reputation and where you still have an incentive for peer review and open data.

ChicagoBoy11 10y ago

I'm surprised that so many of the comments seem to attack the claims in the post along the dimension of "well, anyone can write a blog post." I think as a community HN would staunchly advocate that the recipe for building great companies or software is an almost radical devotion to reducing any and all barriers to entry, except in this domain somehow people feel that the rigidity and exclusivity which journals provide somehow lead to the betterment of science. Makes me think about pg's "what you can't say" post. There is short shrift given to the distortion that gets introduced in order to "get published". The "file drawer" effect is very real, and it gives us a very skewed view of what the actual scientific work being done really is; p-hacking and other statistical trickery are table stakes to anyone trying to survive in the academic world, as being published is a key metric to success yet it is surprisingly orthogonal to being right. As Brian Nosek's "reproducibility project" is uncovering, there is a non-trivial amount of "science" out there that is pure garbage, and much of it was produced in order to "get published." In a world where "random hackers" can write twitter bots to parse scientific papers and uncover obvious mathematical flaws (which sometimes invalidate central claims of the paper), I don't understand how we aren't immediately gravitating to standards which promote more and more openness. Lastly, the biggest issue we have to address is the fact that notoriety is now the chief indicator of success, and not genuine scientific discovery. In college I had world-famous faculty who have made a career, fortune, and placed themselves well on the spotlight on the backs of research which was later completely reversed -- and not in the way "science is supposed to work" sense, but rather in the "you purposefully fudged the math" sense. And yet there was virtually no reputational damage done to their career. Worse yet, I think it is clear that this lack of ethics actually is a direct cause of the success they enjoy today. Given that these are pressures faced by all in the academic community, true openness is the one avenue we have to counteract this sort of thing, and hopefully enterprising individuals come along and find a way to make "truth-seeking" reputation something researchers care about.

pc86 10y ago
That's because a lot of folks on HN come from Harvard or Stanford or MIT, where it's drilled into you that this pedigree is super important. There are companies that will not hire you for an engineering role if you don't have a CS degree from a short list of universities unless you're part of an acquisition (and even then). This gets ignored the second they get $800k in VC money but for some reason the same idea never translates to "real science" or academia.

agumonkey 10y ago

There's a lot of publications. Some of them are really small step attempts at understanding. Sometimes it's just a test. A careful one, but nothing more. I guess in some fields we're still at the poking level.

dahart 10y ago

> Blogs have Open Peer Review No, they don't. This is conflating having an opinion with the scientific review process. The scientific review process can prevent publication of bad research in journals and require changes before publication, the blog "review" process cannot do either of these things.

tkt 10y ago

Titus Brown also a blog post on this earlier this year "The top 10 reasons why blog posts are better than scientific papers" http://ivory.idyll.org/blog/2017-top-ten-reasons-blog-posts.html http://ivory.idyll.org/blog/2017-top-ten-reasons-blog-posts.... and it generated similar discussion. There are a few elements they emphasize. One is what the blog format enables that traditional publishing doesn't support. Those are things like having real-time feedback and comments, being able to version and making the blog post interactive, rather than a static document. Another element to the format is a lack of gatekeepers, so it can be quickly disseminated and disseminated by anyone, so there aren't barriers to participation in the scientific discourse. Another is norms and expectations. In blogging, it is more the norm that data and code are open. Open is still possible in traditional publishing; it just isn't yet the norm. A new format however, enables new norms and it's easier to set them from the start, than try to revise existing ones. Finally, there's the element of 'correctness'. Going through peer review and being in a traditional journal certainly doesn't ensure that the paper is correct. You can just look at retractions to see that http://retractionwatch.com/2011/08/11/is-it-time-for-a-retraction-index/ http://retractionwatch.com/2011/08/11/is-it-time-for-a-retra.... However it would be interesting to see more evidence around whether the blog format does ultimately lead to more 'correct' conclusions, on the whole, and not just for the posts that lead to a lot of discussion.

leecarraher 10y ago

blog post on blogspot advocating that blogs are better than journals seems a bit biased, unfortunately i don't think i can get a journal article about journal articles being better, accepted in any peer reviewed journals. checkmate daniel.

mrdrozdov 10y ago

Is it so hard to publish at arxiv, share the paper on twitter, and submit a link to HN? At a minimum, arxiv is better than a blog post because there's some sense of versioning. For anyone who is curious, some conferences share the peer review comments (even though they do not share the identities of author or commenter). Here are the comments from iclr2017 (https://openreview.net/group?id=ICLR.cc/2017/conference https://openreview.net/group?id=ICLR.cc/2017/conference) and here are comments on Ian Goodfellow's GAN paper by Jürgen Schmidhuber and others (https://media.nips.cc/nipsbooks/nipspapers/paper_files/nips27/reviews/1384.html https://media.nips.cc/nipsbooks/nipspapers/paper_files/nips2...).

colorincorrect 10y ago

As in, "if you follow these best practices, you too can produce high quality research" There is no reason why this post should be focused on blogs in general, asides from generating discussion + clickbait. The key here is actually following the best practice, but do most blogs actually do this? Alternatively: there is a high minimal requirement for research articles, but there is no minimal requirement for blogs.

Spooky23 10y ago

IMO This is one of those cases where people are committed to an existing system and have bent their thinking around that system to the point that they cannot see the big picture. There's alot of value to the existing journal system, but perhaps it would be appropriate to have a more formal process and a more informal/collaborative process. Maybe that would address the problems with having so many journals.

mikk14 10y ago

People here seem to attack only the argument about "anyone can write a blog", but I was expecting HN to give a couple of thoughts also about his first point. To me it reads something like: "publish all your data out in the open for everybody to see". I find it extremely problematic. Sure, sharing data is necessary to verify that the conclusions are supported by it and they are not due to methodological errors. But to anyone? Out in the open? I would expect better data ethics, especially from a psychologist. Your data can contain political opinions, health records, sexual orientation, contact information, the places a person has visited, and when. People sign up for these studies usually with the agreement that the information that can harm them cannot be freely shared, unless to people involved in studies with similar data protection systems. Data like I mentioned has sometimes to be stored in computers not connected to the Internet, to reduce the risk of data leak. Free access to this data paves the way to persecution and shaming. If I were to sign up as subject to a psychology study and had a person with his ideas leading it, I'd withdraw immediately. I'd question if this person should be a psychology researcher at all. Sharing data is good, but protocols are there for a reason.

metalliqaz 10y ago
Personally identifiable details can be removed from data.
- Jweb_Guru 10y ago
  They actually usually cannot, especially if you want to still be able to do research into them (which is the motivation behind differential privacy).
- mikk14 10y ago
  There is a seminal paper showing how carelessly adopting this point of view can lead to disasters: "87% of the U.S. population is uniquely identified by date of birth, gender, postal code" [1] In many other cases, as pointed out, it's just not possible. I'm studying mobility patterns through cellphone metadata. Even if you strip out the actual phone number with a random ID you still know where a person is going, and thus re-identify them if you have other public data. [1] https://en.wikipedia.org/wiki/Latanya_Sweeney https://en.wikipedia.org/wiki/Latanya_Sweeney

88e282102ae2e5b 10y ago

I think it might be better to say that contemporary psychology is frequently not being performed scientifically.

westurner 10y ago

So, schema.org, has classes (C:) -- subclasses of CreativeWork and Article -- for property (P:) domains (D:) and ranges (R:) which cover this domain: - CreativeWork: http://schema.org/CreativeWork http://schema.org/CreativeWork - - BlogPosting: http://schema.org/BlogPosting http://schema.org/BlogPosting - - Article: http://schema.org/Article http://schema.org/Article - - - NewsArticle: http://schema.org/NewsArticle http://schema.org/NewsArticle - - - Report: http://schema.org/Report http://schema.org/Report - - - ScholarlyArticle: http://schema.org/ScholarlyArticle http://schema.org/ScholarlyArticle - - - SocialMediaPosting: http://schema.org/SocialMediaPosting http://schema.org/SocialMediaPosting - - - TechArticle: http://schema.org/TechArticle http://schema.org/TechArticle Thing: (name, [url], [identifier], [#about], [description[_gh_markdown_html]]) - C: CreativeWork: - - P: comment R: Comment - - C: Comment: https://schema.org/Comment https://schema.org/Comment

4258HzG 10y ago

#1 Reason: Blogs aren't included in the publication count which is needed to gain research funding, get hired as a professor and get tenure. A bit cynical, but a lot of the problems the writer suggests are more of a symptom of trying to get as many papers out as possible than the cause (and that one rarely reviews the supplementary material). If scientific blogs counted like publications many of the same problems would appear there.

ylem 10y ago

I can't comment on the field of psychiatry. However, I will comment on my field of physics. The author seems to completely neglect the signal to noise problem. There has been an explosion of journals and scientific publication. It's rather hard to keep up to date on the literature and editors do provide a valuable function in filtering what they think is interesting (which is further filtered by the referees). Also, for the referee process--it's not perfect and will probably not catch complete fraud. But, it is good for checking the basics of whether or not the story being told in a paper is consistent. Also, again related to signal/noise, the editors hopefully choose referees with a minimum level of competence in the field (not always, but usually). Also, in terms of dilution--if there's a blog, there's no guarantee that a critical mass of people will read it to try to give it a critical review. Whereas, even for a journal that is not widely read, the papers in it will have received some peer review where someone in the field tried to take a look through it for obvious errors. I am not convinced by open peer review. Let's consider two scenarios. Suppose there is a leader in the field trying to publish something (or perhaps even a friend or collaborator). If I notice that they are wrong, am I more likely to call them on it if I am anonymous or if they know who I am? One can be idealistic, but researchers are people too. I think that the anonymous review allows people to speak frankly (though it does sometimes let people be mean--that seems to be the cost of anonymity in forums unless there is strong moderation). Let's consider another scenario. Suppose I am a junior researcher and I get a referee comment that I determine is wrong. If it comes from the leader in my field, maybe I won't push back as hard as if it came from someone else junior--again, in an ideal world, it wouldn't matter, but researchers are people too. It would seem that the editor knowing the identity of the referees would serve as a check on bias, not 100 percent, but I think that more is gained than is lost from a closed referee system. In terms of error correction, there are errata that are published if someone finds an error in a major study and in journals like the physical review are linked back to the original article. The open data problem is hard. In my particular field, our raw data is available on the web, but the problem is that the meta-data needed to interpret is not. And that's nontrivial. For example, I may perform several operations on my data (for example, background subtraction) before fitting it to a model. We are experimenting with dataflow languages where we embed the series of "filters" that we apply to the data, but if we want someone to be able to reproduce this 20 years from now, then there's a whole ecosystem that would have to be maintained--for example, not just my code, but all of the libraries that it depends on to run. We can describe the basic process in the paper, but for true long term reproducibility, it's a hard problem...There are groups that are working on open-data and reproducible research, so I don't think that there isn't interest, just that it's not as easy a problem as you might think. But, for reduced data, I agree, it would be nice to have that available for papers in a machine readable format... Open access. This is also a hard problem. Increasingly, funding agencies are requiring that publications be made available in an open format after some embargo period. I think that may be the best we can hope for. Just paying for competent editors requires funding. If we want additional features like data attached to papers (for decades), it will take more funding. That money has to come from somewhere. For typical open access journals, the author pays, but that seems to create difficult incentives--not to mention that it makes it difficult for poorer funded researchers to publish. I'm not sure what the right answer is, because I agree that the public should be able to see the results that they paid for--perhaps an embargo period is the best solution... I think a blog is a great way for communication of ideas and for education, but I do not think that it is able to replace publication in a refereed journal.