4 ms·

I think there is an enormous survivorship bias here. Research advances come in alternating of periods torrents and droughts, not in a steady trickle. Hamming, S

I think there is an enormous survivorship bias here. Research advances come in alternating of periods torrents and droughts, not in a steady trickle. Hamming, Shannon, and their contemporaries worked at a time of revolution in computer science. Let's put deep learning aside for a second. Can anyone here name a single academic in any other subfield of computer science who had a string of groundbreaking research works where every one of those works is from 1980 or after? I can't. And that's almost half a century that we're working with. Many people who defended their PhDs in the 1980s are already retired. This applies to plenty of other fields as well. Medicine, for instance: There were surgeons in the 1960s and 1970s who have half a dozen new surgical techniques to their name. (If you want some examples, look up Michael DeBakey, Mark Coventry, and Thomas Starzl.) You will not find a single academic surgeon today who comes even close to that, and at least in my opinion it's not because modern surgeons lack drive or creativity or the ambition to tackle big problems. Hamming is probably right that the role of luck in the sense of serendipity is overblown, but he ignores role of luck in the sense of structural factors that are out of your control. Because he and his colleagues largely operated in a context where all those structural factors were completely optimized. If you decide to start a PhD in a field 10 years before that field is destined to hit a multi-decade rut, if your advisor fails to make tenure partway through your PhD and your lab is shut down, if you get your first academic appointment in an environment where research funding is trending down and regulatory burdens are trending up, these are all factors that are by and large out of your control that can significantly alter the trajectory of your career.

linguae 3y ago
Indeed. The fact of the matter is that today’s environment for researchers is completely different from what it was like in 1986 when this talk was given, though I mention it not to take anything away from this talk. The days of places like Xerox PARC and Bell Labs where researchers had considerable freedom and autonomy are over and have been for quite some time now. Many industry labs promote business-driven research instead of purely curiosity-driven work and expect their researchers to produce a regular flow of research results that can be productized, lest they be shown the door. Academia these days isn’t exactly a bastion of freedom, either, with its “publish-or-perish” pressures needed to secure tenure and remain in good standing, as well as the need to raise grant money, both of which require pleasing external reviewers. Thus, successful researchers under this environment must find a way to manage the inevitable uncertainties of research while also producing enough output to make their evaluators happy. I find these “productivity” metrics stifling, but if I want to continue as a researcher, it’s either play the game successfully or find another way to make a living. I’ve been thinking long and hard about this for years. Perhaps researchers who value freedom of inquiry and freedom from “publish-or-perish” pressures could work independently, perhaps being funded by fellowships (like the MacArthur Grant), from part-time work, or from a side business.
dblhumbucker 3y ago
I disagree with this sentiment. Although I'm not familiar with today's groundbreaking research, I think dismissing an entire branch of research simply because it goes against one's argument is a little funny. However, Hamming specifically states in his lecture that this could be applied to engineers starting companies, and there are numerous examples of founders who have started multiple successful businesses in this area. In the lecture series, Hamming explicitly emphasizes the importance of dedicating time to predicting what the next big step in one's field would be. He dedicated half of every Friday to trying to predict the future of computer science. One of the major points he drives home is that the next breakthrough in any field will not be what he or anyone else has worked on in the past. That is why it is worth striving to work on the right things.
- Calavar 3y ago
  > I think dismissing an entire branch of research simply because it goes against one's argument is a little funny I don't think it goes against my argument at all. Deep learning is a place where many structural factors are currently optimized, so it has a glut of prominent researchers. The big names in deep learning like Hinton, LeCunn, Bengio took their first post PhD appointments 30 to 40 years ago. All three put out some good work between 1985 and 2010, but all three became an order of magnitude more productive after 2010. They went from being members of a mostly irrelevant and overlooked research niche (neural networks were considered a dead end in ML) to being heads of research at multi-billion dollar companies. If it is really all about the individual, why weren't they that productive from the start? What changed? Well, the number of GPU cores hit an inflection point. Technologies for GP-GPU programming like cuda reached maturity. As I said, there is truth to what Hamming said too. The 1920s were primed for a revolution in physics, but there were thousands of physicists contemporary to Einstein, de Broglie, Heisenberg, and Bohr who didn't do anything of note. So what set the ones who won Nobel prizes apart from the rest? That's where Hamming's advice comes in. In other words, I can be convinced that Einstein or de Broglie would have reached the tops of their fields in any time or place and in nearly any field. But you cannot convince me that they would have made the history books if they were marine biologists in the 1990s. It doesn't matter how hard you think about where the next revolution in your field will be if there is no revolution to be had.
deleted 3y ago
[deleted]
asdfman123 3y ago
Related: I’m annoyed by programmers who implore others to study “worthwhile things.” I’m guessing most of us were obsessed with computers in our teens, and the job market just happened to align with one of our hobbies. Kids who are fascinated with classical piano or performance art don’t stumble into lucrative careers unless they’re in the top 0.1%.
- Fomite 3y ago
  It's also something that can be turned on programmers. One of the things I'm told the most by computational students wanting to join my group is that they want to do "something worthwhile" instead of going to work for Facebook.
pbadams 3y ago
> Let's put deep learning aside for a second. Can anyone here name a single academic in any other subfield of computer science who had a string of groundbreaking research works where every one of those works is from 1980 or after? Shafi Goldwasser (Probabilistic encryption, zero-knowledge proofs, etc.) got her PhD in 1984. Leslie Lamport is just a bit before your deadline, his 'Time, Clocks' paper came out in 1978, but the bulk of his work was after 1980, including paxos. While there's definitely some truth to your Kuhnian view of 'times of revolution' in a field, I think it's hard to apply that to recent progress because it may just be that it's not clear which research works were groundbreaking without the benefit of hindsight. To me, the revolutionary period of CS research is still ongoing.
- pclmulqdq 3y ago
  Almost every breakthrough in distributed systems is more recent. LZW is 1980s. Most data storage systems used today had their guts invented in the last 40 years, with only the top layer being older than that. If you look at "Computer science" as the narrowly-defined field of data structure and algorithm design in a vacuum, maybe things slowed down after 1980, but that's because problems with different constraints just became more interesting.
kragen 3y ago
i agree! hamming even talks about this a bit in his talk your chances of doing great research are slim if you're born in kenya in the 01940s, no matter how much drive and creativity you have, but that's not because 'research advances' have 'droughts', it's because you're not being allowed to do great research, even if plenty of research advances are happening out of your reach (unless you're richard leakey) that's what's happened to modern surgeons, nuclear engineers, aeronautical engineers, chemists, etc. the usa and eu today have mostly become the kenya of the 01940s, where only the occasional richard leakey is allowed to excel, and thus we have entered the so-called great stagnation but restrictions are much looser in computer science than in most fields. we still have: - oleg kiselyov (sxml, probabilistic programming, relational programming and thence the reasoned schemer, type checking as small-step abstract evaluation, tagless staged interpreters, stream processing without runtime overhead) - dan bernstein (twisted edwards curves including curve25519, breaking aes with cache timing attacks, nacl, tweetnacl, qmail, salsa, poly1305, striking down usa export controls, organizing the pqc conferences) - fabrice bellard (qemu, ffmpeg, tcc, lzexe, and bellard's formula, aside from nncp, which is deep learning) - jeff dean and sanjay ghemawat (epi info, leveldb, tensorflow, bigtable, mapreduce, spanner, 'the google cluster architecture', lots of internal google stuff, and again a bunch of deep learning stuff) - raph levien (io, advogato, the gnome canvas, libart) - graydon hoare (monotone and thence git, rust), and - rob pike (blit, utf-8, much of plan9, sawzall, and golang) - wouter van oortmerssen (cube/sauerbraten, flatbuffers, amiga e, false and thus more or less the field of esolangs, lobster, bla with first-class environments, aardappel, fisheye quake) i've left out the people i personally know well
jltsiren 3y ago
> Let's put deep learning aside for a second. Can anyone here name a single academic in any other subfield of computer science who had a string of groundbreaking research works where every one of those works is from 1980 or after? That's because of the nature of the field. CS researchers are kind of like NPCs or bureaucrats. The technical stuff we do is not interesting to the general public, but it enables other people to do more interesting stuff. You mention deep learning as an exception. I, as a researcher in another CS subfield, cannot name a single person who has done fundamental work in it. The "tragedy" of CS is that it's too relevant in the short term. If someone makes a breakthrough, other people will probably commercialize it in a decade or two. Afterwards, history books won't remember the person who discovered the thing but the company that commercialized it or the product launched by the company.
- 7speter 3y ago
  > The technical stuff we do is not interesting to the general public, but it enables other people to do more interesting stuff. People upthread said a lot of scientific research is useless, but my understanding was that the research eventually builds on top of each other in a lot of cases?
  - caddemon 3y ago
    Deep in the trenches a lot of research these days is actually just pretty fucking useless. There are many, many results that unfortunately do not replicate in even slightly different situations. It's a very inefficient way to be searching the scientific space so to speak, and it's a more modern phenomenon. We've scaled (or tried to) science in IMO a really bad way. At least in neurobiology I'm very convinced of that.
Upvoter33 3y ago
"Can anyone here name a single academic in any other subfield of computer science who had a string of groundbreaking research works where every one of those works is from 1980" Hennessy and Patterson for their work on RISC Patterson for RAID Dean and Ghemawat for MapReduce That's without thinking. There have been a lot of very important works since the 1980s.
psychomugs 3y ago
The survivorship bias here reflects the positive-results bias in research itself. One neither wants to nor can publish negative results, despite their potential to be more valuable than massaged and hacked positive results. Not only is this unfair to unlucky research paths, but it also breeds a social media-like positive feedback loop wherein the individual researcher feels alone in failure.
- anonymouskimmer 3y ago
  > wherein the individual researcher feels alone in failure. When thanks to them not publishing their failure, there will probably be a few others who fail the exact same way on the exact same research idea.
di4na 3y ago
Lemire. SPJ. John Hughes. Neil Mitchell. And i did not even try hard, these just instantly came to mind...

linguae 3y ago

Indeed. The fact of the matter is that today’s environment for researchers is completely different from what it was like in 1986 when this talk was given, though I mention it not to take anything away from this talk. The days of places like Xerox PARC and Bell Labs where researchers had considerable freedom and autonomy are over and have been for quite some time now. Many industry labs promote business-driven research instead of purely curiosity-driven work and expect their researchers to produce a regular flow of research results that can be productized, lest they be shown the door. Academia these days isn’t exactly a bastion of freedom, either, with its “publish-or-perish” pressures needed to secure tenure and remain in good standing, as well as the need to raise grant money, both of which require pleasing external reviewers. Thus, successful researchers under this environment must find a way to manage the inevitable uncertainties of research while also producing enough output to make their evaluators happy. I find these “productivity” metrics stifling, but if I want to continue as a researcher, it’s either play the game successfully or find another way to make a living. I’ve been thinking long and hard about this for years. Perhaps researchers who value freedom of inquiry and freedom from “publish-or-perish” pressures could work independently, perhaps being funded by fellowships (like the MacArthur Grant), from part-time work, or from a side business.

dblhumbucker 3y ago

I disagree with this sentiment. Although I'm not familiar with today's groundbreaking research, I think dismissing an entire branch of research simply because it goes against one's argument is a little funny. However, Hamming specifically states in his lecture that this could be applied to engineers starting companies, and there are numerous examples of founders who have started multiple successful businesses in this area. In the lecture series, Hamming explicitly emphasizes the importance of dedicating time to predicting what the next big step in one's field would be. He dedicated half of every Friday to trying to predict the future of computer science. One of the major points he drives home is that the next breakthrough in any field will not be what he or anyone else has worked on in the past. That is why it is worth striving to work on the right things.

Calavar 3y ago
> I think dismissing an entire branch of research simply because it goes against one's argument is a little funny I don't think it goes against my argument at all. Deep learning is a place where many structural factors are currently optimized, so it has a glut of prominent researchers. The big names in deep learning like Hinton, LeCunn, Bengio took their first post PhD appointments 30 to 40 years ago. All three put out some good work between 1985 and 2010, but all three became an order of magnitude more productive after 2010. They went from being members of a mostly irrelevant and overlooked research niche (neural networks were considered a dead end in ML) to being heads of research at multi-billion dollar companies. If it is really all about the individual, why weren't they that productive from the start? What changed? Well, the number of GPU cores hit an inflection point. Technologies for GP-GPU programming like cuda reached maturity. As I said, there is truth to what Hamming said too. The 1920s were primed for a revolution in physics, but there were thousands of physicists contemporary to Einstein, de Broglie, Heisenberg, and Bohr who didn't do anything of note. So what set the ones who won Nobel prizes apart from the rest? That's where Hamming's advice comes in. In other words, I can be convinced that Einstein or de Broglie would have reached the tops of their fields in any time or place and in nearly any field. But you cannot convince me that they would have made the history books if they were marine biologists in the 1990s. It doesn't matter how hard you think about where the next revolution in your field will be if there is no revolution to be had.

deleted 3y ago

[deleted]

asdfman123 3y ago

Related: I’m annoyed by programmers who implore others to study “worthwhile things.” I’m guessing most of us were obsessed with computers in our teens, and the job market just happened to align with one of our hobbies. Kids who are fascinated with classical piano or performance art don’t stumble into lucrative careers unless they’re in the top 0.1%.

Fomite 3y ago
It's also something that can be turned on programmers. One of the things I'm told the most by computational students wanting to join my group is that they want to do "something worthwhile" instead of going to work for Facebook.

pbadams 3y ago

> Let's put deep learning aside for a second. Can anyone here name a single academic in any other subfield of computer science who had a string of groundbreaking research works where every one of those works is from 1980 or after? Shafi Goldwasser (Probabilistic encryption, zero-knowledge proofs, etc.) got her PhD in 1984. Leslie Lamport is just a bit before your deadline, his 'Time, Clocks' paper came out in 1978, but the bulk of his work was after 1980, including paxos. While there's definitely some truth to your Kuhnian view of 'times of revolution' in a field, I think it's hard to apply that to recent progress because it may just be that it's not clear which research works were groundbreaking without the benefit of hindsight. To me, the revolutionary period of CS research is still ongoing.

pclmulqdq 3y ago
Almost every breakthrough in distributed systems is more recent. LZW is 1980s. Most data storage systems used today had their guts invented in the last 40 years, with only the top layer being older than that. If you look at "Computer science" as the narrowly-defined field of data structure and algorithm design in a vacuum, maybe things slowed down after 1980, but that's because problems with different constraints just became more interesting.

kragen 3y ago

i agree! hamming even talks about this a bit in his talk your chances of doing great research are slim if you're born in kenya in the 01940s, no matter how much drive and creativity you have, but that's not because 'research advances' have 'droughts', it's because you're not being allowed to do great research, even if plenty of research advances are happening out of your reach (unless you're richard leakey) that's what's happened to modern surgeons, nuclear engineers, aeronautical engineers, chemists, etc. the usa and eu today have mostly become the kenya of the 01940s, where only the occasional richard leakey is allowed to excel, and thus we have entered the so-called great stagnation but restrictions are much looser in computer science than in most fields. we still have: - oleg kiselyov (sxml, probabilistic programming, relational programming and thence the reasoned schemer, type checking as small-step abstract evaluation, tagless staged interpreters, stream processing without runtime overhead) - dan bernstein (twisted edwards curves including curve25519, breaking aes with cache timing attacks, nacl, tweetnacl, qmail, salsa, poly1305, striking down usa export controls, organizing the pqc conferences) - fabrice bellard (qemu, ffmpeg, tcc, lzexe, and bellard's formula, aside from nncp, which is deep learning) - jeff dean and sanjay ghemawat (epi info, leveldb, tensorflow, bigtable, mapreduce, spanner, 'the google cluster architecture', lots of internal google stuff, and again a bunch of deep learning stuff) - raph levien (io, advogato, the gnome canvas, libart) - graydon hoare (monotone and thence git, rust), and - rob pike (blit, utf-8, much of plan9, sawzall, and golang) - wouter van oortmerssen (cube/sauerbraten, flatbuffers, amiga e, false and thus more or less the field of esolangs, lobster, bla with first-class environments, aardappel, fisheye quake) i've left out the people i personally know well

jltsiren 3y ago

> Let's put deep learning aside for a second. Can anyone here name a single academic in any other subfield of computer science who had a string of groundbreaking research works where every one of those works is from 1980 or after? That's because of the nature of the field. CS researchers are kind of like NPCs or bureaucrats. The technical stuff we do is not interesting to the general public, but it enables other people to do more interesting stuff. You mention deep learning as an exception. I, as a researcher in another CS subfield, cannot name a single person who has done fundamental work in it. The "tragedy" of CS is that it's too relevant in the short term. If someone makes a breakthrough, other people will probably commercialize it in a decade or two. Afterwards, history books won't remember the person who discovered the thing but the company that commercialized it or the product launched by the company.

7speter 3y ago
> The technical stuff we do is not interesting to the general public, but it enables other people to do more interesting stuff. People upthread said a lot of scientific research is useless, but my understanding was that the research eventually builds on top of each other in a lot of cases?
- caddemon 3y ago
  Deep in the trenches a lot of research these days is actually just pretty fucking useless. There are many, many results that unfortunately do not replicate in even slightly different situations. It's a very inefficient way to be searching the scientific space so to speak, and it's a more modern phenomenon. We've scaled (or tried to) science in IMO a really bad way. At least in neurobiology I'm very convinced of that.

Upvoter33 3y ago

"Can anyone here name a single academic in any other subfield of computer science who had a string of groundbreaking research works where every one of those works is from 1980" Hennessy and Patterson for their work on RISC Patterson for RAID Dean and Ghemawat for MapReduce That's without thinking. There have been a lot of very important works since the 1980s.

psychomugs 3y ago

The survivorship bias here reflects the positive-results bias in research itself. One neither wants to nor can publish negative results, despite their potential to be more valuable than massaged and hacked positive results. Not only is this unfair to unlucky research paths, but it also breeds a social media-like positive feedback loop wherein the individual researcher feels alone in failure.

anonymouskimmer 3y ago
> wherein the individual researcher feels alone in failure. When thanks to them not publishing their failure, there will probably be a few others who fail the exact same way on the exact same research idea.

di4na 3y ago

Lemire. SPJ. John Hughes. Neil Mitchell. And i did not even try hard, these just instantly came to mind...