6 ms·
This pattern can be seen in golden age vs current CS papers: the former[0] contain multiple ideas per paper, derived from a short[1] list of references; the lat
by 082349872349872 4y ago
This pattern can be seen in golden age vs current CS papers: the former[0] contain multiple ideas per paper, derived from a short[1] list of references; the latter often spend pages upon a single idea, derived from a huge list of references.
[0] at least: the classics which still have attention drawn to them today
[1] to be fair, they didn't have much literature from which to draw — maybe a fairer comparison would be "references as percentage of literature extant"?
- delusional 4y ago[0] is extremely important here, maybe too important to be left in a footnote. When you look back at history it's all too easy to only recall a select few gems and discard the rest. It's possible that the vast majority of research published today will go by the wayside as dead ends or insignificant, while the stuff that ends up having world changing impact will also contain "multiple ideas" and a "short list of references". I don't have any proof to disagree with you, not even an anecdote, but be cautious of making claims such as these without data. Now will almost always seem uninteresting and pedestrian while the past will seem mysterious and interesting.
- zdw 4y agoI tend to agree with this - one time I tried reading through all the Internet RFCs starting at the first one and I would say that probably 80% of the ideas in first 1000 were abandoned quickly, and after that that the % of abandoned went up to 90-95 percent, with a high quantity of vendors trying to push their own solution a standard. Very pareto principle/sturgeons law there.
- maicro 4y agoWith respect, in case you or anyone else wasn't aware - this is an example of "survivorship bias": https://en.wikipedia.org/wiki/Survivorship_bias https://en.wikipedia.org/wiki/Survivorship_bias .
- DiggyJohnson 4y agoThis is a weak claim unless you provide outside evidence of survivorship bias. Just because a bias could exist isn’t evidence that it does exist in and of itself. This feels like an example of our trigger happy use of biases to dismiss opinions we disagree with. What perspective, experience, or evidence makes you think survivorship bias is at play here? In the comment you’re replying to, for example, the rate of un-abandoned RFC started ~20%, and decreased to ~5-10% over the period of GP’s review. That seems lije a very significant change to me, which is hidden by the surprise that the original number wasn’t higher.
- maicro 4y agoFair argument. I didn't put too much rigorous thought into my post, and really didn't mean offence, but to roughly explain my thinking: - First, I wasn't trying to dismiss any opinions by bringing up survivorship bias, I brought it up because I think it's valuable to have a term to describe the general and probably bi-directional relationship between how notable something is and how well it's known many years after creation - I'd argue that the original comment's footnote [0] talking about "classic" papers is explicitly bringing survivorship, if not the survivorship bias, into the conversation. Honestly, at this point I've kinda lost track of what I wanted to clarify here, so - I apologize for the confusion and, if interpreted as such, the offensive posting. Have a good day.
- PartiallyTyped 4y agoIsn’t what you describe simply the consequence of picking low hanging fruits? it’s only seminal papers that involve multiple ideas together, and even those have a lot of mathematics involved to make them stand up to scrutiny tests. I’d even go as far as to argue that if you took any of the highly cited papers of the last 5 years in ML, 20 years or so back, they’d be even more groundbreaking than that time’s seminal work.
- bjornsing 4y agoI think you see the same in physics. The extreme example is probably Einstein’s “On the electrodynamics of moving bodies” [1], where he laid out the special theory of relativity. It has some incredibly original ideas, and not a single reference. :) 1. https://www.fourmilab.ch/etexts/einstein/specrel/specrel.pdf https://www.fourmilab.ch/etexts/einstein/specrel/specrel.pdf
- pnathan 4y agoI noticed the same thing when doing my Master's (graduated 2012, work in embedded debuggers), for which I read a heck of a lot of early papers, both the better and the worse (Sturgeon's Law is very real). My SWAG is that a lot of early CS papers were being written by people doing very serious programming; they often did a few years in industry (or they were doing very focused Physics coding) before coming back for the PhD. Too, they were not worried about working within the dominant operating system framework - such things didn't quite exist. So they could give things a fresh go by the simple virtue of starting out on a new computer. I think the straight BS->PhD pipeline has introduced an "academic" bias, in the bad sense of the word. I also think that the dominance of Linux, Windows, etc have put blinkers on our research. In any case, you can trace the evolution I describe in the journal https://onlinelibrary.wiley.com/journal/1097024x https://onlinelibrary.wiley.com/journal/1097024x which has been around for decades.
- RealityVoid 4y ago> work in embedded debuggers That is extremely interesting. I always thought that the debugging tools we have are cool... but we could do _so much more_ to observe the systems we develop. And the scripting languages in most of the embedded debuggers I had worked with sucked.
- pnathan 4y agoIn general, our debugging tools today are just terrifyingly crude. Some improvements have been made since my thesis in the industry state but overall... yuck. To get us to the state of the art debugging research in the 90s: - FOL scripting trap on every variable, function, etc. essentially you need this "monitor" or supervisor behind the scenes observing the total state of the system _as it exists in source code, not in hex_. - reversible programs that can go forward and backward in program state & history. Not sure if the 90s research had a full "tree" of possible executions you could walk or if it was linear. - if you attach something like a Lamport state matrix with the a transport layer you can move to a parallel/concurrent debugger OTel and friends are essentially trying to build points 2 and 3 there, but its very mushy, since you're not actually capturing a full system state as part of the system, but whatever the coders chose to put in the system capture. I remember in 2012, 2013 or so, Green River came out with a single core reversible debugger. gdb had one too in '09, or so. Some cool information in an old SO thread: https://stackoverflow.com/q/1470434/26227 https://stackoverflow.com/q/1470434/26227
- kragen 4y agoan easy way to read golden-age papers (say, 01952–01972) that don't have attention drawn to them today is to look at the other papers in the same issue of the journal that had some well-known paper in it when i've done this i've found that most of the papers present trivial and wrongheaded approaches, just like most papers today, but they're still a lot more pleasant to read because they're much better written; however, it's much slower going because the terminology is pretty alien they also have much shorter lists of references but of course they didn't have bibtex
- nindalf 4y agoThis pattern can be seen in golden age vs current kitchen utensils: the former[0] perform admirably today; the latter often are so poorly constructed that they break or have to be thrown away within a couple of years.[1] [0] at least: the utensils which still have attention drawn to them today [1] to be fair: it’s probably survivorship bias.
- godelski 4y agoThis may be a confounding effect where there's pressure to publish faster. But I'd also say that this same pressure can push towards consolidation of ideas as ideas on the boundaries are often more difficult and may take longer to compete.
- asdff 4y agoThe reference lists were short because you were limited to journal issues held in your university library that you happened to pull off the shelf. There was no google scholar to look for each and every paper ever published containing your keyword.