6 ms·

Best Paper Awards in Computer Science over the past 25 years

159 points by lazyjeff 5y ago

knuthsat 5y ago
It's just amazing how DL made most work completely irrelevant. Stuff before 2010 in natural language processing is ridiculous. Dynamic programming algorithms, beam search, dependency parsing (grammar) algorithms (going from O(n^3) to O(n) with cost-sensitive algorithms), a huge focus on lexical analysis, part-of-speech, graphical models (maximum entropy, conditional random fields, etc.). Today all of these algorithms are completely irrelevant. No one needs part-of-speech anymore, or dependency (grammar) trees, or cost-sensitive reinforcement learning reductions. I remember being so inspired by all of the work and learned a lot, but it's quite funny how Lindy works.
- g42gregory 5y ago
  It felt like that 5-7 yrs ago. But now, we have 1,000s of variation of Transformers, Diffusion Models, Energy Models, Patches, Sequence models for Reinforcement Learning (compete with beam search inside!), GNNs, and others to choose from. I feel that all of the Linguistics is likely to come right back and get integrated into the various Deep Learning frameworks.
  - deleted 5y ago
    [deleted]
  - deleted 5y ago
    [deleted]
- canjobear 5y ago
  I wouldn't say they're irrelevant. POS tagging and dependency parsing are still useful for data science type applications, if not for full-scale natural language understanding. Maximum entropy and log-linear models aren't gone, they've just reappeared in a new form where the features come from neural nets.
- ackbar03 5y ago
  I can't help feeling that research prior to deep learning was more rigorous and impressive though. When I read papers from before they tend to be filled with statistical modeling and proofs and were somewhat intimidating. Now it seems like it's a lot of "oh we made this model and it works".
- nextos 5y ago
  You are partly wrong. Graphical models are orthogonal to DL. We are starting to learn how to mix both. E.g HMM + DL = Deep Markov model. It has the advantages of both, structure and large numbers of parameters. Some SOTA NLP models follow this approach.
alexhawke 5y ago
No papers from SIGARCH? :(
- xpuente 5y ago
  Computer architecture is useless /s
- KineticLensman 5y ago
  Or from SIGBOVIK? [0] :( I commend to you the classic paper "Lowestcase and Uppestcase letters: Advances in Derp Learning", in the 2021 proceedings. [0] http://www.sigbovik.org/ http://www.sigbovik.org/
lazyjeff 5y ago
Surprised to see this here, because I don't remember reposting this today (but I did post a couple days ago after midnight and nothing happened), so maybe it's some @dang magic. Anyways, I recently updated this collection over the new year (with a bit of help from two students). It's not meant to be a comprehensive list of all best paper awards, but a fairly representative one across the main conferences in computer science. I guess I think of it as the Oscars (Academy Awards) of Computer Science Papers. It originated because award announcements and conference websites disappear quickly because each year a different organizer manages the website, so this information is lost forever. And I wanted a way to look back and see what people thought was the best paper in that year to see if those paper indeed made an impact. Those who are more quantitative may be interested in this aggregated list of which institutions produce the most best paper awards: https://jeffhuang.com/best_paper_awards/institutions.html https://jeffhuang.com/best_paper_awards/institutions.html And the interesting thing that came out of it is the number of best paper awards is highly correlated with the US News ranking of computer science departments (which is purely based on subjective surveys of department chairs and graduate directors): https://drafty.cs.brown.edu/csopenrankings/ https://drafty.cs.brown.edu/csopenrankings/
- macintux 5y ago
  There's a mechanism for moderators to re-share posts they feel are interesting but didn't attract attention.
- bts 5y ago
  Thank you very much for maintaining this list! Have you considered adding ICFP in addition to PLDI for programming language papers?
  - lazyjeff 5y ago
    Thanks for the note, but I'm afraid not. At this point, it's taking a lot of time for the data entry each year, plus conferences are now almost all awarding 3-6 best papers rather than 1, so the annual work has expanded considerably. Also it's a lot harder now to find the older awardees as various websites have disappeared, and former PC chairs often have forgotten the outcome.
Kyragem 5y ago
bawolff 5y ago
CHI seems to have quite a gamut of paper subjects.
- jasonhong 5y ago
  Yes, CHI is quite broad as a field. It started out primarily as computer scientists and psychologists, grew to include design, and has since then expanded to include elements of anthropology, art, material sciences, ICT4D (tech for developing countries), and more. One thing I've never quite figured out, though, is that other fields in CS tend to splinter into new conferences when the main one gets too big (or perhaps its just systems conferences). In contrast, CHI seems to just get bigger and broader every year.
  - bawolff 5y ago
    Its just fascinating that one conference includes all of these: Coupling Simulation and Hardware for Interactive Circuit Debugging Designing Menstrual Technologies with Adolescents Increasing Electrical Muscle Stimulation's Dexterity by means of Back of the Hand Actuation
  - toomim 5y ago
    True. Whereas most Computer Science communities gather to solve a particular problem (like comprehending natural language), HCI contributions tend to be the problem definitions themselves. A common CHI paper would read "we studied a set of users X and Y and found that they tend to run into problems Z. We designed a solution W to address Z, and suggest some other work that could be done to address Z better." The conference thus contains many papers proposing radically different ways of seeing the issues that users face in computing along with radically different ways of addressing them. If these new-found problem definitions happen to gather enough attention for a sustained period of time, then it warrants a new conference of researchers ready to address those problems. But I don't think this happens often enough in CHI for it to break into sub-conferences very often. (Also, hi Jason!)
Ar-Curunir 5y ago
Only semi-serious suggestion: for security I would add USENIX Security, EUROCRYPT and CRYPTO.
maxwell86 5y ago
No supercomputing ? :( No parallelism ? :(
AnthonBerg 5y ago
My personal awards for the best computer science papers go to Conor McBride and Per Martin-Löf.
compsciphd 5y ago
they left out mobicom papers before 2009 (one of them is mine :) so I'm a bit biased), yes the title changed, but mobicom obviously considers it equivalent https://beta.sigmobile.org/articles/mobicom-best-paper-award https://beta.sigmobile.org/articles/mobicom-best-paper-award
- lazyjeff 5y ago
  Sorry about that (but congratulations anyways). But we have an old but explicit criteria that Best Student Papers are not included, as the Best Paper Awards we list have to consider every accepted paper as a candidate.
  - compsciphd 5y ago
    I'm wondering what percentage of papers at these conferences don't have a student as the primary author. My guess is primarily, only the ones that come from industry?
m0ksha 5y ago
Nice work. The historical view of where research is going is great to see.

159 points by lazyjeff 5y ago

It's just amazing how DL made most work completely irrelevant. Stuff before 2010 in natural language processing is ridiculous. Dynamic programming algorithms, beam search, dependency parsing (grammar) algorithms (going from O(n^3) to O(n) with cost-sensitive algorithms), a huge focus on lexical analysis, part-of-speech, graphical models (maximum entropy, conditional random fields, etc.). Today all of these algorithms are completely irrelevant. No one needs part-of-speech anymore, or dependency (grammar) trees, or cost-sensitive reinforcement learning reductions. I remember being so inspired by all of the work and learned a lot, but it's quite funny how Lindy works.

g42gregory 5y ago
It felt like that 5-7 yrs ago. But now, we have 1,000s of variation of Transformers, Diffusion Models, Energy Models, Patches, Sequence models for Reinforcement Learning (compete with beam search inside!), GNNs, and others to choose from. I feel that all of the Linguistics is likely to come right back and get integrated into the various Deep Learning frameworks.
- deleted 5y ago
  [deleted]
- deleted 5y ago
  [deleted]
canjobear 5y ago
I wouldn't say they're irrelevant. POS tagging and dependency parsing are still useful for data science type applications, if not for full-scale natural language understanding. Maximum entropy and log-linear models aren't gone, they've just reappeared in a new form where the features come from neural nets.
ackbar03 5y ago
I can't help feeling that research prior to deep learning was more rigorous and impressive though. When I read papers from before they tend to be filled with statistical modeling and proofs and were somewhat intimidating. Now it seems like it's a lot of "oh we made this model and it works".
nextos 5y ago
You are partly wrong. Graphical models are orthogonal to DL. We are starting to learn how to mix both. E.g HMM + DL = Deep Markov model. It has the advantages of both, structure and large numbers of parameters. Some SOTA NLP models follow this approach.

alexhawke 5y ago

No papers from SIGARCH? :(

xpuente 5y ago
Computer architecture is useless /s
KineticLensman 5y ago
Or from SIGBOVIK? [0] :( I commend to you the classic paper "Lowestcase and Uppestcase letters: Advances in Derp Learning", in the 2021 proceedings. [0] http://www.sigbovik.org/ http://www.sigbovik.org/

lazyjeff 5y ago

Surprised to see this here, because I don't remember reposting this today (but I did post a couple days ago after midnight and nothing happened), so maybe it's some @dang magic. Anyways, I recently updated this collection over the new year (with a bit of help from two students). It's not meant to be a comprehensive list of all best paper awards, but a fairly representative one across the main conferences in computer science. I guess I think of it as the Oscars (Academy Awards) of Computer Science Papers. It originated because award announcements and conference websites disappear quickly because each year a different organizer manages the website, so this information is lost forever. And I wanted a way to look back and see what people thought was the best paper in that year to see if those paper indeed made an impact. Those who are more quantitative may be interested in this aggregated list of which institutions produce the most best paper awards: https://jeffhuang.com/best_paper_awards/institutions.html https://jeffhuang.com/best_paper_awards/institutions.html And the interesting thing that came out of it is the number of best paper awards is highly correlated with the US News ranking of computer science departments (which is purely based on subjective surveys of department chairs and graduate directors): https://drafty.cs.brown.edu/csopenrankings/ https://drafty.cs.brown.edu/csopenrankings/

macintux 5y ago
There's a mechanism for moderators to re-share posts they feel are interesting but didn't attract attention.
bts 5y ago
Thank you very much for maintaining this list! Have you considered adding ICFP in addition to PLDI for programming language papers?
- lazyjeff 5y ago
  Thanks for the note, but I'm afraid not. At this point, it's taking a lot of time for the data entry each year, plus conferences are now almost all awarding 3-6 best papers rather than 1, so the annual work has expanded considerably. Also it's a lot harder now to find the older awardees as various websites have disappeared, and former PC chairs often have forgotten the outcome.

Kyragem 5y ago

bawolff 5y ago

CHI seems to have quite a gamut of paper subjects.

jasonhong 5y ago
Yes, CHI is quite broad as a field. It started out primarily as computer scientists and psychologists, grew to include design, and has since then expanded to include elements of anthropology, art, material sciences, ICT4D (tech for developing countries), and more. One thing I've never quite figured out, though, is that other fields in CS tend to splinter into new conferences when the main one gets too big (or perhaps its just systems conferences). In contrast, CHI seems to just get bigger and broader every year.
- bawolff 5y ago
  Its just fascinating that one conference includes all of these: Coupling Simulation and Hardware for Interactive Circuit Debugging Designing Menstrual Technologies with Adolescents Increasing Electrical Muscle Stimulation's Dexterity by means of Back of the Hand Actuation
- toomim 5y ago
  True. Whereas most Computer Science communities gather to solve a particular problem (like comprehending natural language), HCI contributions tend to be the problem definitions themselves. A common CHI paper would read "we studied a set of users X and Y and found that they tend to run into problems Z. We designed a solution W to address Z, and suggest some other work that could be done to address Z better." The conference thus contains many papers proposing radically different ways of seeing the issues that users face in computing along with radically different ways of addressing them. If these new-found problem definitions happen to gather enough attention for a sustained period of time, then it warrants a new conference of researchers ready to address those problems. But I don't think this happens often enough in CHI for it to break into sub-conferences very often. (Also, hi Jason!)

Ar-Curunir 5y ago

Only semi-serious suggestion: for security I would add USENIX Security, EUROCRYPT and CRYPTO.

maxwell86 5y ago

No supercomputing ? :( No parallelism ? :(

AnthonBerg 5y ago

My personal awards for the best computer science papers go to Conor McBride and Per Martin-Löf.

compsciphd 5y ago

they left out mobicom papers before 2009 (one of them is mine :) so I'm a bit biased), yes the title changed, but mobicom obviously considers it equivalent https://beta.sigmobile.org/articles/mobicom-best-paper-award https://beta.sigmobile.org/articles/mobicom-best-paper-award

lazyjeff 5y ago
Sorry about that (but congratulations anyways). But we have an old but explicit criteria that Best Student Papers are not included, as the Best Paper Awards we list have to consider every accepted paper as a candidate.
- compsciphd 5y ago
  I'm wondering what percentage of papers at these conferences don't have a student as the primary author. My guess is primarily, only the ones that come from industry?

m0ksha 5y ago

Nice work. The historical view of where research is going is great to see.