3 ms·

This is a common failure mode when people outside academic CS read CS papers. They take the papers too literally. Computer science studies computation as an ab

This is a common failure mode when people outside academic CS read CS papers. They take the papers too literally. Computer science studies computation as an abstract concept. The work may be motivated by what happens in the industry, but it's not supposed to produce anything immediately applicable. Papers may include fake justifications and fake applications, because populist politicians decided long ago that all publicly funded research must have practical real-world applications. But you should not take them at face value. Academic CS values abstract results over concrete results, because real-world systems change too rapidly. Real-world results tend to become obsolete too quickly to be relevant in the time scales the academia is supposed to operate. If you are not in academic CS, you should be careful when reading the papers that you understand the context. Most of the time, you are not in the target audience. Even when there is something relevant in the paper, it's probably not the main result, but an idea related to it. And if you start investigating where that idea came from, it probably builds on many earlier results that seemed obscure and practically irrelevant on their own. Peer reviewers usually spend a few hours on a single review (though there is a lot of variation between fields). A week would be so expensive that most established academics would have to stop teaching and doing research and become full-time reviewers.

deleted 2y ago
[deleted]
withinboredom 2y ago
> Academic CS values abstract results over concrete results, because real-world systems change too rapidly. Real-world results tend to become obsolete too quickly to be relevant in the time scales the academia is supposed to operate. This isn't true. When I'm implementing a paper, I usually go for JUST implementing what they describe, usually by hand. Like if it is a new SQL syntax, I will write a custom recursive descent parser, and hand-roll the query planner, for just the new stuff and hard code some other parts, just as a demonstration. I'm not interested in the industry application part, I'm interested in replicating their work. Once I can replicate it, assuming it is correct, then I will factor the work into a production system. It's this first part that I am frustrated with, not the full implementation in production software.
_gabe_ 2y ago
I’m sorry, what? The best computer scientists I know, Djikstra, Tony Hoare, Turing, Knuth, and more, all developed practical algorithms and concepts that are still used today! Like the parent commenter said, it’s really not difficult to provide code that works and supports your conclusion. I’ve also read computer science papers that bury the lede. It’s apparent when you try to replicate them that the author was disappointed that the work didn’t support their hypothesis, so they made it look good. It’s sad because a true scientific pursuit finds the knowledge valuable whether it works or not! They just need to state it instead of hide that fact. Your whole comment reads as some sort of weird gate-keeping where people without the “proper” education could never fully understand a “true” paper. We can, and we do. There’s a reason we learn from truly great computer scientists like I listed above in college, and not the people posting unreproducible work.
- jltsiren 2y ago
  Early computer scientists discovered many fundamental ideas. Since then, the field has become more specialized and technical. My comment was not gatekeeping and more in line with "if you are a mechanical engineer, don't expect to get much value from papers in theoretical physics". "Proper" education or otherwise spending a few years focused on the topic may help, but it's far from foolproof. The "if you start investigating where that idea came from" part came from my personal experiences. There have been times when I've categorized a paper, on a topic I'm supposed to be an expert, as interesting but practically irrelevant. Only to later find that it became a building block for a major practical result. If you read a research paper and think you understood it well enough to judge it, you are probably wrong. Even if you are an expert on the topic.
  - withinboredom 2y ago
    > If you read a research paper and think you understood it well enough to judge it, you are probably wrong. Even if you are an expert on the topi That sounds like utter nonsense to me, but I go through about 5-10 papers per year. > There have been times when I've categorized a paper, on a topic I'm supposed to be an expert, as interesting but practically irrelevant. Only to later find that it became a building block for a major practical result. This is usually why I am investigating a paper. I want to understand what they did because I see how it can be used practically, if what they wrote is correct. I'm working on the wpaxos paper right now. There's even some example code, totally missing some aspects of the paxos algorithm (it also doesn't appear to run in its original form). There's a whole section in the paper about transactions, marked as "//todo" in the code. The flexible quorum implementation is subtly incorrect as well. Anyway, it's a pretty good paper, and I'm fairly certain a large part of it is probably driving cosmosdb (since they work at Microsoft now and I recognize a number of possibilities that echo what cosmosdb can do, along with some same constraints and it would explain the weird billing; but this is a total guess and I have no proof of it). Anyway, your comment is something I totally agree with. Most people lack the required knowledge, creativity, and insight into most papers to truly understand what the value is in the research. For fun, I am researching magnetic resonance through an iphone's magnetometer to detect stalkers. If it works, I'll have an app (assuming apple doesn't have any issues with it). I wish I could write a paper on it, but I am nowhere near academia these days; which is a whole different shit the academics have done to themselves to make it so only academics can contribute to public knowledge research. The whole publishing papers thing is broken IMHO, and no, I don't have a solution.

deleted 2y ago

[deleted]

withinboredom 2y ago

> Academic CS values abstract results over concrete results, because real-world systems change too rapidly. Real-world results tend to become obsolete too quickly to be relevant in the time scales the academia is supposed to operate. This isn't true. When I'm implementing a paper, I usually go for JUST implementing what they describe, usually by hand. Like if it is a new SQL syntax, I will write a custom recursive descent parser, and hand-roll the query planner, for just the new stuff and hard code some other parts, just as a demonstration. I'm not interested in the industry application part, I'm interested in replicating their work. Once I can replicate it, assuming it is correct, then I will factor the work into a production system. It's this first part that I am frustrated with, not the full implementation in production software.

_gabe_ 2y ago

I’m sorry, what? The best computer scientists I know, Djikstra, Tony Hoare, Turing, Knuth, and more, all developed practical algorithms and concepts that are still used today! Like the parent commenter said, it’s really not difficult to provide code that works and supports your conclusion. I’ve also read computer science papers that bury the lede. It’s apparent when you try to replicate them that the author was disappointed that the work didn’t support their hypothesis, so they made it look good. It’s sad because a true scientific pursuit finds the knowledge valuable whether it works or not! They just need to state it instead of hide that fact. Your whole comment reads as some sort of weird gate-keeping where people without the “proper” education could never fully understand a “true” paper. We can, and we do. There’s a reason we learn from truly great computer scientists like I listed above in college, and not the people posting unreproducible work.

jltsiren 2y ago
Early computer scientists discovered many fundamental ideas. Since then, the field has become more specialized and technical. My comment was not gatekeeping and more in line with "if you are a mechanical engineer, don't expect to get much value from papers in theoretical physics". "Proper" education or otherwise spending a few years focused on the topic may help, but it's far from foolproof. The "if you start investigating where that idea came from" part came from my personal experiences. There have been times when I've categorized a paper, on a topic I'm supposed to be an expert, as interesting but practically irrelevant. Only to later find that it became a building block for a major practical result. If you read a research paper and think you understood it well enough to judge it, you are probably wrong. Even if you are an expert on the topic.
- withinboredom 2y ago
  > If you read a research paper and think you understood it well enough to judge it, you are probably wrong. Even if you are an expert on the topi That sounds like utter nonsense to me, but I go through about 5-10 papers per year. > There have been times when I've categorized a paper, on a topic I'm supposed to be an expert, as interesting but practically irrelevant. Only to later find that it became a building block for a major practical result. This is usually why I am investigating a paper. I want to understand what they did because I see how it can be used practically, if what they wrote is correct. I'm working on the wpaxos paper right now. There's even some example code, totally missing some aspects of the paxos algorithm (it also doesn't appear to run in its original form). There's a whole section in the paper about transactions, marked as "//todo" in the code. The flexible quorum implementation is subtly incorrect as well. Anyway, it's a pretty good paper, and I'm fairly certain a large part of it is probably driving cosmosdb (since they work at Microsoft now and I recognize a number of possibilities that echo what cosmosdb can do, along with some same constraints and it would explain the weird billing; but this is a total guess and I have no proof of it). Anyway, your comment is something I totally agree with. Most people lack the required knowledge, creativity, and insight into most papers to truly understand what the value is in the research. For fun, I am researching magnetic resonance through an iphone's magnetometer to detect stalkers. If it works, I'll have an app (assuming apple doesn't have any issues with it). I wish I could write a paper on it, but I am nowhere near academia these days; which is a whole different shit the academics have done to themselves to make it so only academics can contribute to public knowledge research. The whole publishing papers thing is broken IMHO, and no, I don't have a solution.