3 ms·
Ask HN: What are some of the best papers on AI, ML, DL and their applications?
I was reading the Deep Double Descent paper by OpenAI: https://arxiv.org/abs/1912.02292 https://arxiv.org/abs/1912.02292. The writing is so lucid. Even the structure of the paper is non-conventional. They have a results section before the Related Works section to give sneak peak of what the paper is about. More like telling a story. Even before starting the Introduction there is a self explanatory image spanning the entier two columns.
Can you link to more papers that are written in such style and are easy to read if you have the background knowledge?
- ftxbro 3y agoA Mathematical Theory of Communication by Claude Shannon (https://people.math.harvard.edu/~ctm/home/text/others/shannon/entropy/entropy.pdf https://people.math.harvard.edu/~ctm/home/text/others/shanno...)
- nimrody 3y agoThis one is about compression, reliable communication. While interesting and well-written, I don't think it matches the original request for "papers on AI, ML, ...".
- ftxbro 3y ago> "I don't think it matches the original request for "papers on AI, ML, ..."." I feel like most who understand this paper and who also understand AI and ML will disagree.
- quickthrower2 3y agoI had a good quick skim. It has a language model (ok trigrams!) and the cross entropy formula and reasoning for it. On my reading list for sure! We did some information theory at uni but don’t recall all of this stuff, maybe I just forgot.
- ResearchCode 3y agoResearch paper on AI doesn't necessarily mean a meme title and chasing a dubious "sota" status on some benchmark. I would say foundational work is more worth reading.
- idorosen 3y agoThe paper introducing support vector machines by Cortes & Vapnik was written exceptionally well in my opinion. It tells a part of the story of 60 years of pattern recognition (ML) succinctly from Fisher in 1936 to 1992. https://link.springer.com/content/pdf/10.1007/bf00994018.pdf https://link.springer.com/content/pdf/10.1007/bf00994018.pdf
- flor1s 3y agoI'm not sure if it's truly a timeless paper, but "Attention is all you need" by Vaswani et al. has been super influential in recent years. Also "Deep Unsupervised Learning using Nonequilibrium Thermodynamics" by Sohl-Dickstein et al. (about Diffusion Models) and "NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis" by Mildenhall et al. were hugely influential to me.
- gverri 3y agohttps://paperswithcode.com https://paperswithcode.com