5 ms·

I think academia has a lot to learn from internet-based free open source projects. I find much easier to learn from RFCs, git-hosted documentation projects, an

I think academia has a lot to learn from internet-based free open source projects. I find much easier to learn from RFCs, git-hosted documentation projects, and wikis. (Going from more professional/less open to less professional/more open - not that these are necessarily exclusive, but it seems costly to have a system that satisfies both.) For documentation, I really love sphinx[1] and rst[2]. I know some people love markdown and it's good for light manuals/READMEs but IMO not adequate for full documentation. [1] http://sphinx-doc.org/ http://sphinx-doc.org/ [2] http://docutils.sourceforge.net/docs/user/rst/quickref.html http://docutils.sourceforge.net/docs/user/rst/quickref.html

javert 13y ago
I really disagree. Writing actual papers and having them peer reviewed is a useful model for actually doing science, and you can learn a ton from reading good academic papers. I don't know why you bring up documentation, the current tools used to write academic papers seem fine. As the OP states, the problem is the incentive system, not the particular set of tools and practices. (However, I don't think his suggestions are fundamental enough to change things, and I am skeptical about his claimed expertise on the stock market and copyright.)
- jbooth 13y ago
  Writing down your thoughts and having them read by people is indeed pretty critical for science. What does the current publishing model do for us that a blog wouldn't? What does academia-speak accomplish? From the few CS papers I've read, generally speaking the harder the paper is to understand, the less profound the insights contained. What does the formalism get us, besides formalism itself?
  - glesica 13y ago
    Peer review doesn't demand arcane terminology or difficult-to-understand writing. Ideally, peer review is just someone writing down some thoughts and other people who have some expertise in the field providing comments on those thoughts before they are published. The problem with a blog is that there is no way to screen for quality or obvious mistakes. Of course the current method of doing this might be imperfect, or even completely broken depending on the field, but the solution isn't to just tell academics to start blogs and call it a day. It's a more complex problem than that.
    - jbooth 13y ago
      I guess I'm not seeing a categorical difference. Someone posts a blog to HN that's full of garbage and bad numbers and it gets debunked. We don't have journal subscriptions or piles of bureaucratic BS, just some people who are knowledgeable in their area talking shop. I don't see where all the ceremony creates enough value to pay for itself.
      - glesica 13y ago
        That's true, and it works well for subjective or popular topics. However, it works much less well for highly specialized, technical topics. I have noticed that quite a few very technical, or highly specialized articles end up with just a small handful of comments. This is a problem in research because just about everything is technical and specialized. Another problem is recruiting enough people to actually review all the research produced. HN has a handful of links per day. There are tens of thousands of researchers around the world doing work. Peer reviewers aren't generally compensated monetarily, but they have a strong moral incentive to help out and review articles, especially if they themselves have published in the journal in question. Again, I don't want to come across as defending the entire system. There are certainly things to be improved. All I'm saying is that I don't think the answer is to just tell everyone to start a blog. It's an appealing idea from a populist, egalitarian, and perhaps "hacker" perspective (just get the information out there!), but the current system provides a bit more than a simple communications medium.
  - javert 13y ago
    I think blogs can complement the peer review system, but not replace it. Peer review gets you a lot. Anyway, the person I was responding to wasn't advocating blogs, but wikis, RFCs and documentation projects. Those things can also complement the peer review system, but not in a major way. Academics do already use those things where appropriate.
- infinity0 13y ago
  The things I talked about all involve heavy peer-review, from a much wider audience instead of a few academic referees. Academic papers are definitely not good learning materials, unless one is already familiar with the subject area. You would be hugely irresponsible if your taught a class simply by pointing your students to academic papers. By contrast, for the sorts of projects I listed, there is more structure on the background for understanding a particular document, and even without explicit structure, hyperlinks make the entire mass of content easier to navigate and teach yourself with, instead of having to spend minutes digging out a reference. This is from personal experience of reading both academic papers and working with those sorts of projects, though granted I have more in the latter. Good tools lower the cost of creating good documentation, meaning that more positive incentives can have higher priority. If current tools are "fine" then the OP would have used them to publish a paper instead of writing a series of blog posts.
  - javert 13y ago
    > The things I talked about all involve heavy peer-review, from a much wider audience instead of a few academic referees. The academic peer review process basically curates the literature by filtering out the majority of submitted papers, which often are low quality, cover something already done, etc. So, it's somewhat a different beast. More importantly, though, > Academic papers are definitely not good learning materials, unless one is already familiar with the subject area. That's right, because a lot of the cutting-edge research is really hard stuff that takes years of study to really understand. The point isn't to educate, it's to advance the knowledge of a small group (other researchers) that specilizes in a narrow area. Hopefully, those people (or their students) will eventually distill that knowledge into something usable by others (e.g. software). The point _can't_ be to educate (in general), because the stuff is just too hard. That's just the way research is. (If it's good research.) I don't think the OP's experiences are typical: He needed to publish blogs because his work had immediate impact. That is rare overall in academia, including in CS. Most of the good work that is done will _eventually_ end up in a product, but it will take time. But it's worth noting that the OP _also_ publishes papers. Really don't think there is a tool problem there, though there is always room for improvement.
ssivark 13y ago
It is quite easy to check the correctness of code excerpts, and implementing a piece of code can mostly be considered the analog of a theoretical calculation (we might make mistakes, but in principle you just have to turn the crank and do it right). Otoh, with scientific experiments, it is critical to ensure validity and correctness so that further research may be built on a solid footing. Unlike code, it is not easy to "fork" and verify someone's work. So some form of peer review and verification are crucial. It's not just about openness (scientists have arXiv, PLOS, etc) or about formats/systems (LaTeX is as good as anything else). It's fundamentally about the need to ensure correctness, to establish a foundation for future work.
- infinity0 13y ago
  I was not thinking primarily about code, but design documents, which are peer-reviewed for qualities beyond "correctness" - for example clarity and completeness, which are important for communicating your ideas to many people. And even code has a subjective quality beyond "correctness", so your first claim is false and has no bearing on real-world software projects. OP was not talking about correctness and I am not talking about correctness, but the economics of collaboration and publication in academia.

javert 13y ago

I really disagree. Writing actual papers and having them peer reviewed is a useful model for actually doing science, and you can learn a ton from reading good academic papers. I don't know why you bring up documentation, the current tools used to write academic papers seem fine. As the OP states, the problem is the incentive system, not the particular set of tools and practices. (However, I don't think his suggestions are fundamental enough to change things, and I am skeptical about his claimed expertise on the stock market and copyright.)

jbooth 13y ago
Writing down your thoughts and having them read by people is indeed pretty critical for science. What does the current publishing model do for us that a blog wouldn't? What does academia-speak accomplish? From the few CS papers I've read, generally speaking the harder the paper is to understand, the less profound the insights contained. What does the formalism get us, besides formalism itself?
- glesica 13y ago
  Peer review doesn't demand arcane terminology or difficult-to-understand writing. Ideally, peer review is just someone writing down some thoughts and other people who have some expertise in the field providing comments on those thoughts before they are published. The problem with a blog is that there is no way to screen for quality or obvious mistakes. Of course the current method of doing this might be imperfect, or even completely broken depending on the field, but the solution isn't to just tell academics to start blogs and call it a day. It's a more complex problem than that.
  - jbooth 13y ago
    I guess I'm not seeing a categorical difference. Someone posts a blog to HN that's full of garbage and bad numbers and it gets debunked. We don't have journal subscriptions or piles of bureaucratic BS, just some people who are knowledgeable in their area talking shop. I don't see where all the ceremony creates enough value to pay for itself.
    - glesica 13y ago
      That's true, and it works well for subjective or popular topics. However, it works much less well for highly specialized, technical topics. I have noticed that quite a few very technical, or highly specialized articles end up with just a small handful of comments. This is a problem in research because just about everything is technical and specialized. Another problem is recruiting enough people to actually review all the research produced. HN has a handful of links per day. There are tens of thousands of researchers around the world doing work. Peer reviewers aren't generally compensated monetarily, but they have a strong moral incentive to help out and review articles, especially if they themselves have published in the journal in question. Again, I don't want to come across as defending the entire system. There are certainly things to be improved. All I'm saying is that I don't think the answer is to just tell everyone to start a blog. It's an appealing idea from a populist, egalitarian, and perhaps "hacker" perspective (just get the information out there!), but the current system provides a bit more than a simple communications medium.
- javert 13y ago
  I think blogs can complement the peer review system, but not replace it. Peer review gets you a lot. Anyway, the person I was responding to wasn't advocating blogs, but wikis, RFCs and documentation projects. Those things can also complement the peer review system, but not in a major way. Academics do already use those things where appropriate.
infinity0 13y ago
The things I talked about all involve heavy peer-review, from a much wider audience instead of a few academic referees. Academic papers are definitely not good learning materials, unless one is already familiar with the subject area. You would be hugely irresponsible if your taught a class simply by pointing your students to academic papers. By contrast, for the sorts of projects I listed, there is more structure on the background for understanding a particular document, and even without explicit structure, hyperlinks make the entire mass of content easier to navigate and teach yourself with, instead of having to spend minutes digging out a reference. This is from personal experience of reading both academic papers and working with those sorts of projects, though granted I have more in the latter. Good tools lower the cost of creating good documentation, meaning that more positive incentives can have higher priority. If current tools are "fine" then the OP would have used them to publish a paper instead of writing a series of blog posts.
- javert 13y ago
  > The things I talked about all involve heavy peer-review, from a much wider audience instead of a few academic referees. The academic peer review process basically curates the literature by filtering out the majority of submitted papers, which often are low quality, cover something already done, etc. So, it's somewhat a different beast. More importantly, though, > Academic papers are definitely not good learning materials, unless one is already familiar with the subject area. That's right, because a lot of the cutting-edge research is really hard stuff that takes years of study to really understand. The point isn't to educate, it's to advance the knowledge of a small group (other researchers) that specilizes in a narrow area. Hopefully, those people (or their students) will eventually distill that knowledge into something usable by others (e.g. software). The point _can't_ be to educate (in general), because the stuff is just too hard. That's just the way research is. (If it's good research.) I don't think the OP's experiences are typical: He needed to publish blogs because his work had immediate impact. That is rare overall in academia, including in CS. Most of the good work that is done will _eventually_ end up in a product, but it will take time. But it's worth noting that the OP _also_ publishes papers. Really don't think there is a tool problem there, though there is always room for improvement.

ssivark 13y ago

It is quite easy to check the correctness of code excerpts, and implementing a piece of code can mostly be considered the analog of a theoretical calculation (we might make mistakes, but in principle you just have to turn the crank and do it right). Otoh, with scientific experiments, it is critical to ensure validity and correctness so that further research may be built on a solid footing. Unlike code, it is not easy to "fork" and verify someone's work. So some form of peer review and verification are crucial. It's not just about openness (scientists have arXiv, PLOS, etc) or about formats/systems (LaTeX is as good as anything else). It's fundamentally about the need to ensure correctness, to establish a foundation for future work.

infinity0 13y ago
I was not thinking primarily about code, but design documents, which are peer-reviewed for qualities beyond "correctness" - for example clarity and completeness, which are important for communicating your ideas to many people. And even code has a subjective quality beyond "correctness", so your first claim is false and has no bearing on real-world software projects. OP was not talking about correctness and I am not talking about correctness, but the economics of collaboration and publication in academia.