4 ms·
Why is this the most cited paper in AI and not the original 1943 paper who started it all?
by netdevphoenix 1y ago
Why is this the most cited paper in AI and not the original 1943 paper who started it all?
- zaptrem 1y agoTransformers are what made ML infinitely scalable and caused a huge amount of progress in very few years since everyone could just go scale things. However, idk how many of those papers actually even cite the transformer paper?
- tankenmate 1y agoI just checked Google Scholar, not perfect but good for an indicative; "A logical calculus of the ideas immanent in nervous activity" [WS McCulloch, W Pitts - The bulletin of mathematical biophysics, 1943] has ~33,000 citations, and "Attention is all you need" [A Vaswani, N Shazeer, et al, Advances in Neural Information Processing Systems, 2017] has ~180,000 citations.
- netdevphoenix 1y agoAs I understand, the transformer architecture is built on deep learning. Would you say that transformers made a bigger progress RELATIVE to the progress made by deep learning? AFAIK, before the first wave of AI powered apps that were visible to users appeared thanks to deep learning in the early 10s. Users went from nothing to fancy AI features, the question is likely subjective but is the jump from nothing to fancy AI features the same as the jump from fancy AI features to GenAI in relative terms? We can't forget that new tech builds upon older tech hence merits need to be relative
- tankenmate 1y agoProbably because of the modern "publish or perish" mantra led to an exponential growth in publications, and "newer is better" means that newer impactful papers get cited more than older impactful publications. But that thesis is probably a paper in itself (of the meta analysis navel gazing variety).
- daemonologist 1y agoBecause the MCP neuron is taken as common knowledge and people do not feel the need to explicitly reference it (and haven't for some time), and the pace of publishing has increased in recent years.