3 ms·

I think an important thing to take into account is that a very important part of what makes a good scientist is not just how good they are at "doing science", b

I think an important thing to take into account is that a very important part of what makes a good scientist is not just how good they are at "doing science", but how good they are at communicating it. A good scientist is a good communicator. Even if that means finding people who you can work with who are good communicators, it's part of the job. Coding up a really clever solution and just kind of hoping people will come across it and appreciate is not really doing science. (Not saying that's what you are doing, I'm just going on a tangent here.) The point is, ever since the enlightenment, a strictly critical part of the job of being a scientist has been the ability to elaborate your ideas to others, and to argue for its merit. So yeah, doing the work is the fun and motivating part, but a big part of the reason that the academic community values well-written scientific papers is because all that work can go to waste if it is not described (and defended!) in a way that is clear and convincing (to the relevant audience of course). Grad students may complain about having to learn to write effective papers, but it's literally just as important as the technical work they are doing --- maybe moreso, since in the future they may very likely change topics, and the writing and communication skills is the part that will carry forward throughout their career.

l0b0 8y ago
I'm reminded of the saying that "Publishing research without data is simply advertising, not science". Not even publishing the research means it doesn't have a chance at advancing the state of the art. I don't understand how such research can be successful at anything other than selling a few extra copies of a software product likely to disappear within a handful of years. Unless it's implemented as the absolute killer feature of a very successful product it's very unlikely anyone will ever know there's anything novel in there.
- jandrewrogers 8y ago
  The research does get disseminated, just slowly. The transmission vector is contact with other people with firsthand knowledge of the research. It becomes an oral tradition, which makes it difficult to break into and is inefficient. I've come across many non-publishing research communities in various parts of computer science that continue to develop and share the state-of-the-art amongst themselves. The research isn't being lost, it is just compartmentalized. For example, there is an elegant cache replacement algorithm I learned from a database kernel engineer a decade ago, one of those algorithms that is so obvious in hindsight that you can't believe it never occurred to you. It solves a key problem with the algorithms everyone knows in literature. I've used it ever since and taught a bunch of other database kernel engineers the same algorithm. I have no idea where the person I learned it from got it, but the literature still treats it like an unsolved problem. Many engineers are using it in more and more systems, and there are a few different variants in the wild, but no one has bothered to write it down AFAICT. The salient point is that academia is not the only community that develops and defines the state-of-the-art in computer science, but at least they publish. Without an incentive for non-publishing research communities to publish, it becomes difficult for an outside academic to break into some fields because there isn't an obvious entry point.
  - d-- 8y ago
    Is there a chance you can point us to a code snippet of said cache algorithm, or elaborate a bit? If it's simple and elegant in hindsight this insight might be interesting in other CS applications. Not cross sharing and reinventing wheels is a somewhat sad implication from the compartmentalization you mention.
  - phoenixstrike 8y ago
    That is the whole point. You're sitting on a (claimed) solution to an unsolved problem in literature. That is antagonistic to science. Publication and collaboration is literally part of the definition of doing "research" or "science." It's not just about solving any unsolved problem out there. If you're not sharing your solution by publishing it and opening it up for examination and discussion in an established forum, it means nothing. Just think, if you had published that solution, maybe someone else would have read it and been inspired by it and thought of an even better solution, or made a connection to another unsolved problem. Or something like that. It's also very telling that you say you only work on problems that are directly relevant to a software product, and that you chose not to publish because of the time commitment. It seems you only consider the personal gains and sacrifices involved; it's intellectual masturbation. The spirit of research is about something greater than yourself. You're not a researcher, and you don't conduct research. You are a very good -- excellent -- software engineer.
  - infinite8s 8y ago
    Is the issue with publishing your findings the grind of submitting to a journal and going through the standard review process? Because in that case, would just throwing up a high level overview on arxiv work for you? It sounds like you aren't adverse to sharing the results of your research if the activation energy was eliminated (or substantially reduced).
mattkrause 8y ago
Grad students complain about a lot of things, but I’ve literally never heard one complain about having to write good papers. They could complain, however, that PhD training contains shockingly little on how to write effectively, given the skill’s importance to most post-PhD jobs. To make matters worse, any training is often front-loaded at the beginning of a program so those skills are atrophied by the time PhD candidates have their own data to write up. If I am ever in charge of a graduate program, I’m putting everyone through a 10 week expository writing class around year 3.5-4. I took one in grade 12 and that public high school class literally beat most Ivy League grad seminars for (academic) career prep.
- Ar-Curunir 8y ago
  Literally downthread: https://news.ycombinator.com/item?id=17432281 https://news.ycombinator.com/item?id=17432281
Iv 8y ago
It is possible to recognize how much the publishing process is useful and still feel it is perfectible. It made me think a bit about the state of research when Paul Krugman was explaining that at his point, communication between researchers basically happens between arxiv and blogs. Official publications come at a later time as part of a validation of the whole process, but its cycle is too slow to be useful anymore. In CS and especially in things related to deep learning, I can understand that pausing to publish can be felt like a delay. I feel like the typical publication format is not well adapted to CS. For instance, a big point of formal publications is to give other researchers the tools to reproduce your experiments. Why is it not the norm to offer source code or even docker containers then?

l0b0 8y ago

I'm reminded of the saying that "Publishing research without data is simply advertising, not science". Not even publishing the research means it doesn't have a chance at advancing the state of the art. I don't understand how such research can be successful at anything other than selling a few extra copies of a software product likely to disappear within a handful of years. Unless it's implemented as the absolute killer feature of a very successful product it's very unlikely anyone will ever know there's anything novel in there.

jandrewrogers 8y ago
The research does get disseminated, just slowly. The transmission vector is contact with other people with firsthand knowledge of the research. It becomes an oral tradition, which makes it difficult to break into and is inefficient. I've come across many non-publishing research communities in various parts of computer science that continue to develop and share the state-of-the-art amongst themselves. The research isn't being lost, it is just compartmentalized. For example, there is an elegant cache replacement algorithm I learned from a database kernel engineer a decade ago, one of those algorithms that is so obvious in hindsight that you can't believe it never occurred to you. It solves a key problem with the algorithms everyone knows in literature. I've used it ever since and taught a bunch of other database kernel engineers the same algorithm. I have no idea where the person I learned it from got it, but the literature still treats it like an unsolved problem. Many engineers are using it in more and more systems, and there are a few different variants in the wild, but no one has bothered to write it down AFAICT. The salient point is that academia is not the only community that develops and defines the state-of-the-art in computer science, but at least they publish. Without an incentive for non-publishing research communities to publish, it becomes difficult for an outside academic to break into some fields because there isn't an obvious entry point.
- d-- 8y ago
  Is there a chance you can point us to a code snippet of said cache algorithm, or elaborate a bit? If it's simple and elegant in hindsight this insight might be interesting in other CS applications. Not cross sharing and reinventing wheels is a somewhat sad implication from the compartmentalization you mention.
- phoenixstrike 8y ago
  That is the whole point. You're sitting on a (claimed) solution to an unsolved problem in literature. That is antagonistic to science. Publication and collaboration is literally part of the definition of doing "research" or "science." It's not just about solving any unsolved problem out there. If you're not sharing your solution by publishing it and opening it up for examination and discussion in an established forum, it means nothing. Just think, if you had published that solution, maybe someone else would have read it and been inspired by it and thought of an even better solution, or made a connection to another unsolved problem. Or something like that. It's also very telling that you say you only work on problems that are directly relevant to a software product, and that you chose not to publish because of the time commitment. It seems you only consider the personal gains and sacrifices involved; it's intellectual masturbation. The spirit of research is about something greater than yourself. You're not a researcher, and you don't conduct research. You are a very good -- excellent -- software engineer.
- infinite8s 8y ago
  Is the issue with publishing your findings the grind of submitting to a journal and going through the standard review process? Because in that case, would just throwing up a high level overview on arxiv work for you? It sounds like you aren't adverse to sharing the results of your research if the activation energy was eliminated (or substantially reduced).

mattkrause 8y ago

Grad students complain about a lot of things, but I’ve literally never heard one complain about having to write good papers. They could complain, however, that PhD training contains shockingly little on how to write effectively, given the skill’s importance to most post-PhD jobs. To make matters worse, any training is often front-loaded at the beginning of a program so those skills are atrophied by the time PhD candidates have their own data to write up. If I am ever in charge of a graduate program, I’m putting everyone through a 10 week expository writing class around year 3.5-4. I took one in grade 12 and that public high school class literally beat most Ivy League grad seminars for (academic) career prep.

Ar-Curunir 8y ago
Literally downthread: https://news.ycombinator.com/item?id=17432281 https://news.ycombinator.com/item?id=17432281

Iv 8y ago

It is possible to recognize how much the publishing process is useful and still feel it is perfectible. It made me think a bit about the state of research when Paul Krugman was explaining that at his point, communication between researchers basically happens between arxiv and blogs. Official publications come at a later time as part of a validation of the whole process, but its cycle is too slow to be useful anymore. In CS and especially in things related to deep learning, I can understand that pausing to publish can be felt like a delay. I feel like the typical publication format is not well adapted to CS. For instance, a big point of formal publications is to give other researchers the tools to reproduce your experiments. Why is it not the norm to offer source code or even docker containers then?