8 ms·

30papers.com – Ilya's 30 essential ML papers, in a beginner friendly format

652 points by notmcrowley 3mo ago

notmcrowley 3mo ago
Author here. First year CS student at Trinity College Dublin. I Built this because when I was getting into reading research papers I ended up burning a ton of my Claude usage asking questions other people have probably already asked. The website is just a side project and definitely a WIP. Happy to answer questions or take PRs on GitHub.
- groby_b 3mo ago
  I think it'd be interesting to hear what you think the goal of the site is. Is it just rehosting the list, plus a reformatted copy of the papers? I was hoping you'd have at least annotated them with what you'd learned?
  - notmcrowley 3mo ago
    Hey thanks for checking out the website. I did not expect this to get as much attention as it did. I was honestly just planning on having it as a small side project for my friends and some others who would like to get into this kind of stuff. I will definitely annotate them in the future if that is something that people would appreciate. I currently have something like this done for a few papers on my X account.
    - groby_b 3mo ago
      I mean, you don't have to annotate them :) Would I love to read annotations? Sure - but it's a boatload of work for you, and if it's just meant as a repo for you and your friends, no need to do a ton of work just because an Internet rando asked. But given the sudden wide audience, a quick "here's what this is for" at the top might be helpful.
    - fuzzythinker 3mo ago
      Yes, I expected annotations, so please add. I know you named the site 30 papers, but I don't think anyone will hold you to it if you add more papers to it as you read and annotate more, or allow others to do so.
  - gjvc 3mo ago
    > I think it'd be interesting to hear what you think the goal of the site is. why do you care? this is a disingenuous question.
- gowld 3mo ago
  An option to disable animation and show the paper links in a simple list would be helpful.
  - jodacola 3mo ago
    Agree on the animation. As an aside, I've seen folks mention respecting reduced animation hints and such in the past and was always curious about this because I've never had any negative experiences with animations... until now! Something about the animations on this site did my brain in while scrolling through the papers, and now I "get it."
  - anomaloustho 3mo ago
    The problem is the background is often times doing a wave motion across the screen. Then the foreground content is doing an in/out undulation on top. So you’re seeing an undulating in/out in every possible direction + the background. And the foreground animations are all at the same time. So it’s not that we’re emphasizing any one thing. We’re emphasizing all of it. The key with animations is in what they’re trying to draw attention to, the character of the movement, and the timing of it. You usually don’t want everything to equally animate at once. I would: • Use background movement that also isn’t a “wave” • Stagger the timing of foreground animations so the main content is emphasized, followed by a pause, followed by the sidebars • Change the nature of the animations so they’re not doing the same essentially thing “zoom and pan” - so have the center zoom and pan, but do something different for the sides.
- mjg2 3mo ago
  Thanks for sharing this. It appears your README.md's first paragraph is truncated. It ends with "Carmack which reportedly contains..." What were you intending to say next?
- phtrivier 3mo ago
  Hi ! The "artistic direction" is a bit original, but that's your thing, and you're free to present it anyway you want, of course ! I think the biggest problem people / I had when reading the list was the "based on a rumoured list of papers that Ilya Sutskever gave to John Carmack." Where did you get the "rumoured list" from ? Why should a reader trust the rumours ? That seems to be a pretty big appeal to authority, and it's okay if it's only "word of mouth" (as most papers seems legit), but it's really weird not to give a source, or a backstory, or references, etc... Especially since you claim to only have 27 ;)
  - notmcrowley 3mo ago
    The list came from a post on X by ex-OpenAI employee Andrew Carr. There are multiple lists floating around which is why I said rumoured. I found this to be a more credible source than others though given Sutskever's connection to OpenAI so I used it.
- fl0id 3mo ago
  What is the easily accessible part? Just that they are on one website? At first I thought there would be more explanation or other addons.
  - notmcrowley 3mo ago
    There are summaries for each paper on the landing page. Also as you read the papers, key/difficult words are highlighted and you can click on them to get a simple definition quickly. A few people have asked for write ups on what my takeaways from each paper were so I am currently working on that. If there are any suggestions that anyone has for things that would help them with the papers I will definitely add them. My main goal is to make this as easy as possible for people to use
lostmsu 3mo ago
Main page UX is terrible. If you go for quirky, fine, but I would not want to use it.
- solarengineer 3mo ago
  Indeed. I scoffed at your comment and went to the website. After scrolling a bit, I find myself having a mild headache and slight dizziness. I would request the author to consider something that does not distract us from this educational and informative website ( I have bookmarked it ).
- soperj 3mo ago
  Yes, normally wouldn't ever say anything, but I could even read the text things were just flying around. (On firefox)
- mattmatheus 3mo ago
  Indeed. It's very bad.
quibono 3mo ago
I was confused for a minute, I thought this was "top 30 papers by Ilya" and was then wondering why "Quantifying the Rise and Fall of Complexity in Closed Systems: The Coffee Automaton" is on the list. > In additition, even though I have read the vast majority of the papers featured on the website, I have not read through each of the website's versions end to end. Website's versions, as in - the actual text or the "explanations"? Either way this is a big red flag.
imenani 3mo ago
Nice presentation of the list! I'd recommend watching a few of his talks/podcasts before during reading these to get the overview and how all the bits in these works tie together. https://www.dwarkesh.com/p/ilya-sutskever https://www.dwarkesh.com/p/ilya-sutskever https://simons.berkeley.edu/talks/ilya-sutskever-openai-2023-08-14 https://simons.berkeley.edu/talks/ilya-sutskever-openai-2023... https://www.dwarkesh.com/p/ilya-sutskever-2 https://www.dwarkesh.com/p/ilya-sutskever-2
prideout 3mo ago
Kolmogorov Complexity looks interesting. It seems to formalize Occam’s Razor and the notion that intelligence = compression.
- Lerc 3mo ago
  I wouldn't say so about Occam's Razor which is a heuristic. The relationship between compression and intelligence, while not equal is definitely there. It looks like 3Blue1Brown is going to be doing some videos on this aspect.
  - nextaccountic 3mo ago
    there's a way to connect kolmogorov complexity and occam's razor, which is solomonoff induction
- Destructotor 3mo ago
  If you find this interesting, you should look into Solomonoff induction. It combines Kolmogorov complexity with Bayes rule to provide a general framework for inductive inference, and naturally formalizes Occam's razor.
renyicircle 3mo ago
The formatting of the articles on this website is bad. I've opened the first one and all the LaTeX formulas are messed up. The subscripts and superscripts are all flattened rendering the math hard to comprehend. Did the author actually try to read any of the articles? >∏ plocal(x|z) = i p(xi|z,xWindowAround(i)) Images and tables are not rendered at all. What is the point of this? Just keep the links to arxiv and leave it at that, otherwise render the articles properly
- deleted 3mo ago
  [deleted]
omneity 3mo ago
I thought the actual 30 papers have never been disclosed. Do you have a source tying the recommendations back to Ilya, or did you come up with this list?
- renyicircle 3mo ago
  This list was made by some guy on twitter. https://x.com/keshavchan/status/1787861946173186062 https://x.com/keshavchan/status/1787861946173186062 It's unknown whether it has anything to do with Ilya Sutskever.
- ayhanfuat 3mo ago
  I think someone on Twitter made it up. It was also 40 papers, not 30. https://dallasinnovates.com/exclusive-qa-john-carmacks-different-path-to-artificial-general-intelligence/ https://dallasinnovates.com/exclusive-qa-john-carmacks-diffe...
- notmcrowley 3mo ago
  The list I got was from ex-OpenAI employee Andrew Carr on X. I believe he said in his post however that the list he uploaded is not the full list they were provided at OpenAI however.
aperrien 3mo ago
Is there a way to download them all in one go?
- gooob 3mo ago
  https://colah.github.io/posts/2015-08-Understanding-LSTMs/ https://colah.github.io/posts/2015-08-Understanding-LSTMs/ https://papers.nips.cc/paper/2012/hash/c399862d3b9d6b76c8436e924a68c45b-Abstract.html https://papers.nips.cc/paper/2012/hash/c399862d3b9d6b76c8436... https://papers.nips.cc/paper_files/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf https://papers.nips.cc/paper_files/paper/2012/file/c399862d3... https://arxiv.org/pdf/1512.03385 https://arxiv.org/pdf/1512.03385 https://arxiv.org/pdf/1511.07122 https://arxiv.org/pdf/1511.07122 https://arxiv.org/pdf/1603.05027 https://arxiv.org/pdf/1603.05027 https://arxiv.org/pdf/1409.2329 https://arxiv.org/pdf/1409.2329 https://arxiv.org/pdf/1512.02595 https://arxiv.org/pdf/1512.02595 https://arxiv.org/pdf/1409.0473 https://arxiv.org/pdf/1409.0473 https://arxiv.org/pdf/1506.03134 https://arxiv.org/pdf/1506.03134 https://arxiv.org/pdf/1706.03762 https://arxiv.org/pdf/1706.03762 https://nlp.seas.harvard.edu/annotated-transformer/ https://nlp.seas.harvard.edu/annotated-transformer/ https://arxiv.org/pdf/1410.5401 https://arxiv.org/pdf/1410.5401 https://arxiv.org/pdf/1706.01427 https://arxiv.org/pdf/1706.01427 https://arxiv.org/pdf/1806.01822 https://arxiv.org/pdf/1806.01822 https://arxiv.org/pdf/1704.01212 https://arxiv.org/pdf/1704.01212 https://arxiv.org/pdf/2001.08361 https://arxiv.org/pdf/2001.08361 https://arxiv.org/pdf/1811.06965 https://arxiv.org/pdf/1811.06965 https://www.cs.toronto.edu/~hinton/absps/colt93.pdf https://www.cs.toronto.edu/~hinton/absps/colt93.pdf https://arxiv.org/pdf/math/0406077 https://arxiv.org/pdf/math/0406077 https://scottaaronson.blog/?p=762 https://scottaaronson.blog/?p=762 https://arxiv.org/pdf/1405.6903 https://arxiv.org/pdf/1405.6903 https://onlinelibrary.wiley.com/doi/10.1002/047174882X.ch14 https://onlinelibrary.wiley.com/doi/10.1002/047174882X.ch14 https://github.com/Bladefidz/information-theory/blob/master/books/Elements%20of%20Information%20Theory.pdf https://github.com/Bladefidz/information-theory/blob/master/... https://arxiv.org/pdf/1611.02731 https://arxiv.org/pdf/1611.02731 https://www.vetta.org/documents/Machine_Super_Intelligence.pdf https://www.vetta.org/documents/Machine_Super_Intelligence.p... https://karpathy.github.io/2015/05/21/rnn-effectiveness/ https://karpathy.github.io/2015/05/21/rnn-effectiveness/ https://cs231n.github.io/ https://cs231n.github.io/
clintonc 3mo ago
I wish this were organized according to suggested/logical reading order. For example, the paper introducing the attention mechanism probably ought to precede "attention is all you need".
- eirikbakke 3mo ago
  Second this! And if the papers are in "logical reading order", it would be very useful if this is stated on top!
brachkow 3mo ago
> "beginner friendly format" > looks inside > math
IceDane 3mo ago
Why on earth would you deliberately choose to do whatever the fuck it is you did with the scroll and the animations for each paper when scrolling through the landing page? What are those animations supposed to be? I use firefox but I also visited on chrome, and the page is even more broken there. Scroll doesn't "take" unless I scroll hard enough, otherwise it bounces back. But on chrome, at least, it seems like the animation for each paper is clearer - it's supposed to be animating the scale of the paper as you scroll to it.. but it seems that your background animation is lagging everything so much it just doesn't work.
- elictronic 3mo ago
  Myspace and 5th grader PowerPoint presentations had a vibe coded child.
david_shi 3mo ago
Is this meant to be read in order?
throwaw12 3mo ago
Where did you get the list? AFAIK, list was never shared
jawarner 3mo ago
Noting the theory papers on Kolmorogov complexity. For those not familiar, Ilya argues that the reason why neural networks generalize -- why they work at all -- is because they are effectively finding a simple description of their training data, converging down onto the limit of the Kolmorogov complexity. [1] [1] https://www.youtube.com/watch?v=AKMuA_TVz3A https://www.youtube.com/watch?v=AKMuA_TVz3A
- niksmather 3mo ago
  That's true of all statistical models, it's not some magic property of neural networks.
cute_boi 3mo ago
No need stupid moving texts. CS231n: Convolutional Neural Networks for Visual Recognition - https://cs231n.github.io/ https://cs231n.github.io/ The Unreasonable Effectiveness of Recurrent Neural Networks - https://karpathy.github.io/2015/05/21/rnn-effectiveness/ https://karpathy.github.io/2015/05/21/rnn-effectiveness/ Understanding LSTM Networks - https://colah.github.io/posts/2015-08-Understanding-LSTMs/ https://colah.github.io/posts/2015-08-Understanding-LSTMs/ ImageNet Classification with Deep Convolutional Neural Networks - https://papers.nips.cc/paper/2012/hash/c399862d3b9d6b76c8436e924a68c45b-Abstract.html https://papers.nips.cc/paper/2012/hash/c399862d3b9d6b76c8436... Deep Residual Learning for Image Recognition - https://arxiv.org/abs/1512.03385 https://arxiv.org/abs/1512.03385 Multi-Scale Context Aggregation by Dilated Convolutions - https://arxiv.org/abs/1511.07122 https://arxiv.org/abs/1511.07122 Identity Mappings in Deep Residual Networks - https://arxiv.org/abs/1603.05027 https://arxiv.org/abs/1603.05027 Recurrent Neural Network Regularization - https://arxiv.org/abs/1409.2329 https://arxiv.org/abs/1409.2329 Deep Speech 2: End-to-End Speech Recognition in English and Mandarin - https://arxiv.org/abs/1512.02595 https://arxiv.org/abs/1512.02595 Order Matters: Sequence to Sequence for Sets - https://arxiv.org/abs/1511.06391 https://arxiv.org/abs/1511.06391 Neural Machine Translation by Jointly Learning to Align and Translate - https://arxiv.org/abs/1409.0473 https://arxiv.org/abs/1409.0473 Pointer Networks - https://arxiv.org/abs/1506.03134 https://arxiv.org/abs/1506.03134 Attention Is All You Need - https://arxiv.org/abs/1706.03762 https://arxiv.org/abs/1706.03762 The Annotated Transformer - https://nlp.seas.harvard.edu/annotated-transformer/ https://nlp.seas.harvard.edu/annotated-transformer/ Neural Turing Machines - https://arxiv.org/abs/1410.5401 https://arxiv.org/abs/1410.5401 A Simple Neural Network Module for Relational Reasoning - https://arxiv.org/abs/1706.01427 https://arxiv.org/abs/1706.01427 Relational Recurrent Neural Networks - https://arxiv.org/abs/1806.01822 https://arxiv.org/abs/1806.01822 Neural Message Passing for Quantum Chemistry - https://arxiv.org/abs/1704.01212 https://arxiv.org/abs/1704.01212 Scaling Laws for Neural Language Models - https://arxiv.org/abs/2001.08361 https://arxiv.org/abs/2001.08361 GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism - https://arxiv.org/abs/1811.06965 https://arxiv.org/abs/1811.06965 Keeping Neural Networks Simple by Minimizing the Description Length of the Weights - https://www.cs.toronto.edu/~hinton/absps/colt93.pdf https://www.cs.toronto.edu/~hinton/absps/colt93.pdf A Tutorial Introduction to the Minimum Description Length Principle - https://arxiv.org/abs/math/0406077 https://arxiv.org/abs/math/0406077 The First Law of Complexodynamics - https://scottaaronson.blog/?p=762 https://scottaaronson.blog/?p=762 Quantifying the Rise and Fall of Complexity in Closed Systems: The Coffee Automaton - https://arxiv.org/abs/1405.6903 https://arxiv.org/abs/1405.6903 Kolmogorov Complexity - https://onlinelibrary.wiley.com/doi/book/10.1002/047174882X https://onlinelibrary.wiley.com/doi/book/10.1002/047174882X Variational Lossy Autoencoder - https://arxiv.org/abs/1611.02731 https://arxiv.org/abs/1611.02731 Machine Super Intelligence - https://www.vetta.org/documents/Machine_Super_Intelligence.pdf https://www.vetta.org/documents/Machine_Super_Intelligence.p...
HAL3000 3mo ago
Someone posts on X, "These are Ilya’s 30 papers", gives no source, doesn't say where he got it from, and isn't connected to either Ilya or Carmack (Ilya gave him the list). Then someone vibe codes a barely usable website based on that, and it lands on the HN front page? Is this correct?
- starcast2026 3mo ago
  [dead]
- dominotw 3mo ago
  they kind of mention the source on their website though " rumoured list of papers that Ilya Sutskever gave to John Carmack. " there is aslo manning book called illya list https://www.manning.com/books/sutskevers-list https://www.manning.com/books/sutskevers-list
- youniverse 3mo ago
  Compiled resources for nerds are catnip. Hit that bookmark/upvote button to never get to it :)
  - CobrastanJorji 3mo ago
    I wish this was more wrong.
  - dominotw 3mo ago
    attention is all you need
  - schmookeeg 3mo ago
    I feel seen. Straight to the "stuff to read later" bookmark pile/mausoleum :)
    - NDlurker 3mo ago
      One time my brother upgraded his RAM so he could keep more tabs open. I think he was up to a couple hundred.
janpmz 3mo ago
After seeing this for the first time, I've build PdfToMp3 to listen to these papers. It has now evolved into ListenDock. Fun fact: PdfToMp3 existed before NotebookLM and I already had "overviews", but I called them teacher explanations. Here is an example of a "Teacher Explanation" of the paper "Quantifying the Rise and Fall of Complexity in Closed Systems: The Coffee Automaton" https://listendock.com/e/quantifying_the_rise_and_fall_of_complexity_in_closed_systems_the_coffee_automaton_wvlvm https://listendock.com/e/quantifying_the_rise_and_fall_of_co...
- janpmz 3mo ago
  Why do I get downvoted whenever I post something here? Do you think its too spammy? Because its AI? Do I have a downvote bot following me?
  - jimhi 3mo ago
    I haven’t looked at your other comments but the answer is your comment isn’t valuable. Text to speech summarizing is a dime a dozen. Your audience here prefers reading a blog and is already annoyed by ai vs written by a human content so what you are offering is the opposite of what they want.
    - janpmz 3mo ago
      Ok, thanks for the viewpoint, that makes sense. I use AI summaries every day and find it very valuable. But I also see the trend e.g. on Reddit, that people are very dismissive of ai content.
jackp96 3mo ago
So the styling and animation work looks really cool (when isolated), but they distract from the content itself, IMO. I think it'd work better if you featured the animated background effect toward the top of the page and shifted toward static graphics (or much subtler animations) as the user scrolls. And I don't think the zoom-out effect on the listing cards has the intended effect; I found myself wanting to get a better look at the papers and was a little disappointed/annoyed when they got smaller and harder to see as I pulled them into view. The colors/shadows/layout all looks really nice, but I feel like the animations (as-is) ultimately detract from the experience rather than add to it. Thanks for sharing, though!
eachro 3mo ago
Anyone got a list for the agentic LLM age?
lwarfield 3mo ago
For beginners I'd recommend the Welch Labs Illustrated Guide To AI if your not well versed in reading papers. Its a beautiful book that I've enjoyed going through. I'd recommend going through these papers after reading that to get a deep understanding.
- mauz 3mo ago
  Bought because of this comment. Thanks!
lwarfield 3mo ago
Its interesting seeing how many of these researchers became the heads of frontier labs!
nimchimpsky 3mo ago
[dead]
notmcrowley 3mo ago
Hey guys, I really appreciate all of the attention this post has received. I honestly thought it was going to be just a small project to help some of my friends get into reading research papers. A large number of people complained about how intense some of the backgrounds/animations were (I might have been a bit too focused on making something that looked cool over usability). In response I have added toggles for both the movement on the page and the backgrounds for the papers. Other people mentioned that they would have liked some more personalised reflections on each paper. I currently have already done some of these for the more popular papers on my X @notmcrowley . I would have no problem adding these to the site if people think it will help. I feel the need to warn that I have not been formally educated on ML or AI so any interpretation will just be mine and may not necessarily be the correct one. (If anyone with more experience would like to contribute to this feel free to reach out).
- SirHackalot 3mo ago
  Please add them on the site for those of us who have never had Twitter and don’t plan to open one ever. Thanks for this compilation, I am — like your friends — trying to get into reading research papers and this is right up my alley right now.
- EagnaIonat 3mo ago
  Even with the motion/background button toggles you are still left with tiny fonts that make it hard to read. It actually made me check my browser wasn't set to zoom out. But then using zoom changes nothing, which breaks accessibility. Also why does the header need to take up 3/4 of the screen? [edit] Clicking on a paper doesn't even bring the paper up. I have to hit another click to get to it.
amemi 3mo ago
Possibly the original X tweet that popularized this list? 2024, 876k views https://x.com/keshavchan/status/1787861946173186062 https://x.com/keshavchan/status/1787861946173186062 In my opinion, whether it was actually by Ilya or not is not worthy of debate. Many of them are widely recognized for being good pedagogical resources (e.g. annotated transformer, unreasonable effectiveness of RNNs, understanding LSTM networks), and others are landmark papers which anyone interested in the field would benefit from reading: - Krizhevsky et al. (2012) introduced AlexNet - Bahdanau et al. (2014) introduced attention - He et al. (2015) introduced ResNet - Vaswani et al. (2017) introduced the Transformer Other papers are more specialized. Of them, I think Kaplan et al. (2020) by OpenAI is probably most important.
- gekoxyz 3mo ago
  Even if Ilya didn't really create this list I have a very good opinion about every paper on this page that I've read (most of them) so I think it's a great resource. Lately during my off time I want to do something related to AI research (which I am already doing full time atm so I need something light) and I am for sure going to read through this.
anonymzz 3mo ago
Sutskever's List : https://www.manning.com/books/sutskevers-list https://www.manning.com/books/sutskevers-list
theahura 3mo ago
always a fun day when someone rediscovers these. Lucky 10000 in case folks are interested, i wrote up a ~layman's review of each paper over the course of two years a while back. Several of those reviews ended up doing reasonably well on hn. Full analysis of the ~23 docs that were papers and not massive books https://12gramsofcarbon.com/p/ilyas-30-papers-to-carmack-table https://12gramsofcarbon.com/p/ilyas-30-papers-to-carmack-tab...
nitin7 3mo ago
How is this a beginner friendly format?
mmiao 3mo ago
the format is not friendly at all...
nomilk 3mo ago
This is a beautiful way to present extremely high quality information. I sometimes lament the unpleasant friction involved in finding and reading academic papers (the overly formal style is a necessary evil, but the irritating paywalls, followed by inevitable searches for '%{title} filetype:pdf' feel like unnecessary ones).
anentropic 3mo ago
Has anyone tried just asking John Carmack...?
bookofjoe 3mo ago
Delete the two options in the upper right hand corner of the homepage and Bob's your uncle.
upmind 3mo ago
I wonder how up to date these papers are because I imagine this list came out >6 years ago
- dahart 3mo ago
  It’s very easy to check & see there’s at least one paper from 2023. Also it takes time to know which papers are influential. But better to contribute than speculate… what are the seminal ML papers from the last 6 years that should be on the list?
joeclark77 3mo ago
Thanks for the effort; I have a little feedback: - Very few of these are labeled with a clear reference to source and date (think APA style). - They don't seem to be in order chronologically. I would think "essential" papers would be read from the earlier "foundational" works up to the more recent ones that work on them. - The oldest ones seem to date to the 2010s. I think it would be hard to jump in at such a recent point. The basics of machine learning come from the 1960s-70s. If you want to improve it I would recommend coming up with a sequential "reading list" including a few classic papers, some intermediate advancements (frequently referenced), and then a few new, cutting edge articles.
mfaulk 3mo ago
I read most of those papers a few years ago and wrote up an overview and summaries: https://mfaulk.github.io/2024/06/19/ilya-papers.html https://mfaulk.github.io/2024/06/19/ilya-papers.html
hwj 3mo ago
The "ML" here seems to stand for "Machine Learning" (not "Meta Language" as in SML or OCaml).
_giorgio_ 3mo ago
Ilya has never made the list public. The list is simply made up, as confirmed by John Carmack. """ John Carmack @ID_AA_Carmack I rather expected @ilyasut to have made a public post by now after all the discussion of the AI reading list he gave me. A canonical list of references from a leading figure would be appreciated by many. I would be curious myself about what he would add from the last three years. """ https://x.com/ID_AA_Carmack/status/1622673143469858816 https://x.com/ID_AA_Carmack/status/1622673143469858816

652 points by notmcrowley 3mo ago

Author here. First year CS student at Trinity College Dublin. I Built this because when I was getting into reading research papers I ended up burning a ton of my Claude usage asking questions other people have probably already asked. The website is just a side project and definitely a WIP. Happy to answer questions or take PRs on GitHub.

groby_b 3mo ago
I think it'd be interesting to hear what you think the goal of the site is. Is it just rehosting the list, plus a reformatted copy of the papers? I was hoping you'd have at least annotated them with what you'd learned?
- notmcrowley 3mo ago
  Hey thanks for checking out the website. I did not expect this to get as much attention as it did. I was honestly just planning on having it as a small side project for my friends and some others who would like to get into this kind of stuff. I will definitely annotate them in the future if that is something that people would appreciate. I currently have something like this done for a few papers on my X account.
  - groby_b 3mo ago
    I mean, you don't have to annotate them :) Would I love to read annotations? Sure - but it's a boatload of work for you, and if it's just meant as a repo for you and your friends, no need to do a ton of work just because an Internet rando asked. But given the sudden wide audience, a quick "here's what this is for" at the top might be helpful.
  - fuzzythinker 3mo ago
    Yes, I expected annotations, so please add. I know you named the site 30 papers, but I don't think anyone will hold you to it if you add more papers to it as you read and annotate more, or allow others to do so.
- gjvc 3mo ago
  > I think it'd be interesting to hear what you think the goal of the site is. why do you care? this is a disingenuous question.
gowld 3mo ago
An option to disable animation and show the paper links in a simple list would be helpful.
- jodacola 3mo ago
  Agree on the animation. As an aside, I've seen folks mention respecting reduced animation hints and such in the past and was always curious about this because I've never had any negative experiences with animations... until now! Something about the animations on this site did my brain in while scrolling through the papers, and now I "get it."
- anomaloustho 3mo ago
  The problem is the background is often times doing a wave motion across the screen. Then the foreground content is doing an in/out undulation on top. So you’re seeing an undulating in/out in every possible direction + the background. And the foreground animations are all at the same time. So it’s not that we’re emphasizing any one thing. We’re emphasizing all of it. The key with animations is in what they’re trying to draw attention to, the character of the movement, and the timing of it. You usually don’t want everything to equally animate at once. I would: • Use background movement that also isn’t a “wave” • Stagger the timing of foreground animations so the main content is emphasized, followed by a pause, followed by the sidebars • Change the nature of the animations so they’re not doing the same essentially thing “zoom and pan” - so have the center zoom and pan, but do something different for the sides.
mjg2 3mo ago
Thanks for sharing this. It appears your README.md's first paragraph is truncated. It ends with "Carmack which reportedly contains..." What were you intending to say next?
phtrivier 3mo ago
Hi ! The "artistic direction" is a bit original, but that's your thing, and you're free to present it anyway you want, of course ! I think the biggest problem people / I had when reading the list was the "based on a rumoured list of papers that Ilya Sutskever gave to John Carmack." Where did you get the "rumoured list" from ? Why should a reader trust the rumours ? That seems to be a pretty big appeal to authority, and it's okay if it's only "word of mouth" (as most papers seems legit), but it's really weird not to give a source, or a backstory, or references, etc... Especially since you claim to only have 27 ;)
- notmcrowley 3mo ago
  The list came from a post on X by ex-OpenAI employee Andrew Carr. There are multiple lists floating around which is why I said rumoured. I found this to be a more credible source than others though given Sutskever's connection to OpenAI so I used it.
fl0id 3mo ago
What is the easily accessible part? Just that they are on one website? At first I thought there would be more explanation or other addons.
- notmcrowley 3mo ago
  There are summaries for each paper on the landing page. Also as you read the papers, key/difficult words are highlighted and you can click on them to get a simple definition quickly. A few people have asked for write ups on what my takeaways from each paper were so I am currently working on that. If there are any suggestions that anyone has for things that would help them with the papers I will definitely add them. My main goal is to make this as easy as possible for people to use

lostmsu 3mo ago

Main page UX is terrible. If you go for quirky, fine, but I would not want to use it.

solarengineer 3mo ago
Indeed. I scoffed at your comment and went to the website. After scrolling a bit, I find myself having a mild headache and slight dizziness. I would request the author to consider something that does not distract us from this educational and informative website ( I have bookmarked it ).
soperj 3mo ago
Yes, normally wouldn't ever say anything, but I could even read the text things were just flying around. (On firefox)
mattmatheus 3mo ago
Indeed. It's very bad.

quibono 3mo ago

I was confused for a minute, I thought this was "top 30 papers by Ilya" and was then wondering why "Quantifying the Rise and Fall of Complexity in Closed Systems: The Coffee Automaton" is on the list. > In additition, even though I have read the vast majority of the papers featured on the website, I have not read through each of the website's versions end to end. Website's versions, as in - the actual text or the "explanations"? Either way this is a big red flag.

imenani 3mo ago

Nice presentation of the list! I'd recommend watching a few of his talks/podcasts before during reading these to get the overview and how all the bits in these works tie together. https://www.dwarkesh.com/p/ilya-sutskever https://www.dwarkesh.com/p/ilya-sutskever https://simons.berkeley.edu/talks/ilya-sutskever-openai-2023-08-14 https://simons.berkeley.edu/talks/ilya-sutskever-openai-2023... https://www.dwarkesh.com/p/ilya-sutskever-2 https://www.dwarkesh.com/p/ilya-sutskever-2

prideout 3mo ago

Kolmogorov Complexity looks interesting. It seems to formalize Occam’s Razor and the notion that intelligence = compression.

Lerc 3mo ago
I wouldn't say so about Occam's Razor which is a heuristic. The relationship between compression and intelligence, while not equal is definitely there. It looks like 3Blue1Brown is going to be doing some videos on this aspect.
- nextaccountic 3mo ago
  there's a way to connect kolmogorov complexity and occam's razor, which is solomonoff induction
Destructotor 3mo ago
If you find this interesting, you should look into Solomonoff induction. It combines Kolmogorov complexity with Bayes rule to provide a general framework for inductive inference, and naturally formalizes Occam's razor.

renyicircle 3mo ago

The formatting of the articles on this website is bad. I've opened the first one and all the LaTeX formulas are messed up. The subscripts and superscripts are all flattened rendering the math hard to comprehend. Did the author actually try to read any of the articles? >∏ plocal(x|z) = i p(xi|z,xWindowAround(i)) Images and tables are not rendered at all. What is the point of this? Just keep the links to arxiv and leave it at that, otherwise render the articles properly

deleted 3mo ago
[deleted]

omneity 3mo ago

I thought the actual 30 papers have never been disclosed. Do you have a source tying the recommendations back to Ilya, or did you come up with this list?

renyicircle 3mo ago
This list was made by some guy on twitter. https://x.com/keshavchan/status/1787861946173186062 https://x.com/keshavchan/status/1787861946173186062 It's unknown whether it has anything to do with Ilya Sutskever.
ayhanfuat 3mo ago
I think someone on Twitter made it up. It was also 40 papers, not 30. https://dallasinnovates.com/exclusive-qa-john-carmacks-different-path-to-artificial-general-intelligence/ https://dallasinnovates.com/exclusive-qa-john-carmacks-diffe...
notmcrowley 3mo ago
The list I got was from ex-OpenAI employee Andrew Carr on X. I believe he said in his post however that the list he uploaded is not the full list they were provided at OpenAI however.

aperrien 3mo ago

Is there a way to download them all in one go?

gooob 3mo ago
https://colah.github.io/posts/2015-08-Understanding-LSTMs/ https://colah.github.io/posts/2015-08-Understanding-LSTMs/ https://papers.nips.cc/paper/2012/hash/c399862d3b9d6b76c8436e924a68c45b-Abstract.html https://papers.nips.cc/paper/2012/hash/c399862d3b9d6b76c8436... https://papers.nips.cc/paper_files/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf https://papers.nips.cc/paper_files/paper/2012/file/c399862d3... https://arxiv.org/pdf/1512.03385 https://arxiv.org/pdf/1512.03385 https://arxiv.org/pdf/1511.07122 https://arxiv.org/pdf/1511.07122 https://arxiv.org/pdf/1603.05027 https://arxiv.org/pdf/1603.05027 https://arxiv.org/pdf/1409.2329 https://arxiv.org/pdf/1409.2329 https://arxiv.org/pdf/1512.02595 https://arxiv.org/pdf/1512.02595 https://arxiv.org/pdf/1409.0473 https://arxiv.org/pdf/1409.0473 https://arxiv.org/pdf/1506.03134 https://arxiv.org/pdf/1506.03134 https://arxiv.org/pdf/1706.03762 https://arxiv.org/pdf/1706.03762 https://nlp.seas.harvard.edu/annotated-transformer/ https://nlp.seas.harvard.edu/annotated-transformer/ https://arxiv.org/pdf/1410.5401 https://arxiv.org/pdf/1410.5401 https://arxiv.org/pdf/1706.01427 https://arxiv.org/pdf/1706.01427 https://arxiv.org/pdf/1806.01822 https://arxiv.org/pdf/1806.01822 https://arxiv.org/pdf/1704.01212 https://arxiv.org/pdf/1704.01212 https://arxiv.org/pdf/2001.08361 https://arxiv.org/pdf/2001.08361 https://arxiv.org/pdf/1811.06965 https://arxiv.org/pdf/1811.06965 https://www.cs.toronto.edu/~hinton/absps/colt93.pdf https://www.cs.toronto.edu/~hinton/absps/colt93.pdf https://arxiv.org/pdf/math/0406077 https://arxiv.org/pdf/math/0406077 https://scottaaronson.blog/?p=762 https://scottaaronson.blog/?p=762 https://arxiv.org/pdf/1405.6903 https://arxiv.org/pdf/1405.6903 https://onlinelibrary.wiley.com/doi/10.1002/047174882X.ch14 https://onlinelibrary.wiley.com/doi/10.1002/047174882X.ch14 https://github.com/Bladefidz/information-theory/blob/master/books/Elements%20of%20Information%20Theory.pdf https://github.com/Bladefidz/information-theory/blob/master/... https://arxiv.org/pdf/1611.02731 https://arxiv.org/pdf/1611.02731 https://www.vetta.org/documents/Machine_Super_Intelligence.pdf https://www.vetta.org/documents/Machine_Super_Intelligence.p... https://karpathy.github.io/2015/05/21/rnn-effectiveness/ https://karpathy.github.io/2015/05/21/rnn-effectiveness/ https://cs231n.github.io/ https://cs231n.github.io/

clintonc 3mo ago

I wish this were organized according to suggested/logical reading order. For example, the paper introducing the attention mechanism probably ought to precede "attention is all you need".

eirikbakke 3mo ago
Second this! And if the papers are in "logical reading order", it would be very useful if this is stated on top!

brachkow 3mo ago

> "beginner friendly format" > looks inside > math

IceDane 3mo ago

Why on earth would you deliberately choose to do whatever the fuck it is you did with the scroll and the animations for each paper when scrolling through the landing page? What are those animations supposed to be? I use firefox but I also visited on chrome, and the page is even more broken there. Scroll doesn't "take" unless I scroll hard enough, otherwise it bounces back. But on chrome, at least, it seems like the animation for each paper is clearer - it's supposed to be animating the scale of the paper as you scroll to it.. but it seems that your background animation is lagging everything so much it just doesn't work.

elictronic 3mo ago
Myspace and 5th grader PowerPoint presentations had a vibe coded child.

david_shi 3mo ago

Is this meant to be read in order?

throwaw12 3mo ago

Where did you get the list? AFAIK, list was never shared

jawarner 3mo ago

Noting the theory papers on Kolmorogov complexity. For those not familiar, Ilya argues that the reason why neural networks generalize -- why they work at all -- is because they are effectively finding a simple description of their training data, converging down onto the limit of the Kolmorogov complexity. [1] [1] https://www.youtube.com/watch?v=AKMuA_TVz3A https://www.youtube.com/watch?v=AKMuA_TVz3A

niksmather 3mo ago
That's true of all statistical models, it's not some magic property of neural networks.

cute_boi 3mo ago

No need stupid moving texts. CS231n: Convolutional Neural Networks for Visual Recognition - https://cs231n.github.io/ https://cs231n.github.io/ The Unreasonable Effectiveness of Recurrent Neural Networks - https://karpathy.github.io/2015/05/21/rnn-effectiveness/ https://karpathy.github.io/2015/05/21/rnn-effectiveness/ Understanding LSTM Networks - https://colah.github.io/posts/2015-08-Understanding-LSTMs/ https://colah.github.io/posts/2015-08-Understanding-LSTMs/ ImageNet Classification with Deep Convolutional Neural Networks - https://papers.nips.cc/paper/2012/hash/c399862d3b9d6b76c8436e924a68c45b-Abstract.html https://papers.nips.cc/paper/2012/hash/c399862d3b9d6b76c8436... Deep Residual Learning for Image Recognition - https://arxiv.org/abs/1512.03385 https://arxiv.org/abs/1512.03385 Multi-Scale Context Aggregation by Dilated Convolutions - https://arxiv.org/abs/1511.07122 https://arxiv.org/abs/1511.07122 Identity Mappings in Deep Residual Networks - https://arxiv.org/abs/1603.05027 https://arxiv.org/abs/1603.05027 Recurrent Neural Network Regularization - https://arxiv.org/abs/1409.2329 https://arxiv.org/abs/1409.2329 Deep Speech 2: End-to-End Speech Recognition in English and Mandarin - https://arxiv.org/abs/1512.02595 https://arxiv.org/abs/1512.02595 Order Matters: Sequence to Sequence for Sets - https://arxiv.org/abs/1511.06391 https://arxiv.org/abs/1511.06391 Neural Machine Translation by Jointly Learning to Align and Translate - https://arxiv.org/abs/1409.0473 https://arxiv.org/abs/1409.0473 Pointer Networks - https://arxiv.org/abs/1506.03134 https://arxiv.org/abs/1506.03134 Attention Is All You Need - https://arxiv.org/abs/1706.03762 https://arxiv.org/abs/1706.03762 The Annotated Transformer - https://nlp.seas.harvard.edu/annotated-transformer/ https://nlp.seas.harvard.edu/annotated-transformer/ Neural Turing Machines - https://arxiv.org/abs/1410.5401 https://arxiv.org/abs/1410.5401 A Simple Neural Network Module for Relational Reasoning - https://arxiv.org/abs/1706.01427 https://arxiv.org/abs/1706.01427 Relational Recurrent Neural Networks - https://arxiv.org/abs/1806.01822 https://arxiv.org/abs/1806.01822 Neural Message Passing for Quantum Chemistry - https://arxiv.org/abs/1704.01212 https://arxiv.org/abs/1704.01212 Scaling Laws for Neural Language Models - https://arxiv.org/abs/2001.08361 https://arxiv.org/abs/2001.08361 GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism - https://arxiv.org/abs/1811.06965 https://arxiv.org/abs/1811.06965 Keeping Neural Networks Simple by Minimizing the Description Length of the Weights - https://www.cs.toronto.edu/~hinton/absps/colt93.pdf https://www.cs.toronto.edu/~hinton/absps/colt93.pdf A Tutorial Introduction to the Minimum Description Length Principle - https://arxiv.org/abs/math/0406077 https://arxiv.org/abs/math/0406077 The First Law of Complexodynamics - https://scottaaronson.blog/?p=762 https://scottaaronson.blog/?p=762 Quantifying the Rise and Fall of Complexity in Closed Systems: The Coffee Automaton - https://arxiv.org/abs/1405.6903 https://arxiv.org/abs/1405.6903 Kolmogorov Complexity - https://onlinelibrary.wiley.com/doi/book/10.1002/047174882X https://onlinelibrary.wiley.com/doi/book/10.1002/047174882X Variational Lossy Autoencoder - https://arxiv.org/abs/1611.02731 https://arxiv.org/abs/1611.02731 Machine Super Intelligence - https://www.vetta.org/documents/Machine_Super_Intelligence.pdf https://www.vetta.org/documents/Machine_Super_Intelligence.p...

HAL3000 3mo ago

Someone posts on X, "These are Ilya’s 30 papers", gives no source, doesn't say where he got it from, and isn't connected to either Ilya or Carmack (Ilya gave him the list). Then someone vibe codes a barely usable website based on that, and it lands on the HN front page? Is this correct?

starcast2026 3mo ago
[dead]
dominotw 3mo ago
they kind of mention the source on their website though " rumoured list of papers that Ilya Sutskever gave to John Carmack. " there is aslo manning book called illya list https://www.manning.com/books/sutskevers-list https://www.manning.com/books/sutskevers-list
youniverse 3mo ago
Compiled resources for nerds are catnip. Hit that bookmark/upvote button to never get to it :)
- CobrastanJorji 3mo ago
  I wish this was more wrong.
- dominotw 3mo ago
  attention is all you need
- schmookeeg 3mo ago
  I feel seen. Straight to the "stuff to read later" bookmark pile/mausoleum :)
  - NDlurker 3mo ago
    One time my brother upgraded his RAM so he could keep more tabs open. I think he was up to a couple hundred.

janpmz 3mo ago

After seeing this for the first time, I've build PdfToMp3 to listen to these papers. It has now evolved into ListenDock. Fun fact: PdfToMp3 existed before NotebookLM and I already had "overviews", but I called them teacher explanations. Here is an example of a "Teacher Explanation" of the paper "Quantifying the Rise and Fall of Complexity in Closed Systems: The Coffee Automaton" https://listendock.com/e/quantifying_the_rise_and_fall_of_complexity_in_closed_systems_the_coffee_automaton_wvlvm https://listendock.com/e/quantifying_the_rise_and_fall_of_co...

janpmz 3mo ago
Why do I get downvoted whenever I post something here? Do you think its too spammy? Because its AI? Do I have a downvote bot following me?
- jimhi 3mo ago
  I haven’t looked at your other comments but the answer is your comment isn’t valuable. Text to speech summarizing is a dime a dozen. Your audience here prefers reading a blog and is already annoyed by ai vs written by a human content so what you are offering is the opposite of what they want.
  - janpmz 3mo ago
    Ok, thanks for the viewpoint, that makes sense. I use AI summaries every day and find it very valuable. But I also see the trend e.g. on Reddit, that people are very dismissive of ai content.

jackp96 3mo ago

So the styling and animation work looks really cool (when isolated), but they distract from the content itself, IMO. I think it'd work better if you featured the animated background effect toward the top of the page and shifted toward static graphics (or much subtler animations) as the user scrolls. And I don't think the zoom-out effect on the listing cards has the intended effect; I found myself wanting to get a better look at the papers and was a little disappointed/annoyed when they got smaller and harder to see as I pulled them into view. The colors/shadows/layout all looks really nice, but I feel like the animations (as-is) ultimately detract from the experience rather than add to it. Thanks for sharing, though!

eachro 3mo ago

Anyone got a list for the agentic LLM age?

lwarfield 3mo ago

For beginners I'd recommend the Welch Labs Illustrated Guide To AI if your not well versed in reading papers. Its a beautiful book that I've enjoyed going through. I'd recommend going through these papers after reading that to get a deep understanding.

mauz 3mo ago
Bought because of this comment. Thanks!

lwarfield 3mo ago

Its interesting seeing how many of these researchers became the heads of frontier labs!

nimchimpsky 3mo ago

[dead]

notmcrowley 3mo ago

Hey guys, I really appreciate all of the attention this post has received. I honestly thought it was going to be just a small project to help some of my friends get into reading research papers. A large number of people complained about how intense some of the backgrounds/animations were (I might have been a bit too focused on making something that looked cool over usability). In response I have added toggles for both the movement on the page and the backgrounds for the papers. Other people mentioned that they would have liked some more personalised reflections on each paper. I currently have already done some of these for the more popular papers on my X @notmcrowley . I would have no problem adding these to the site if people think it will help. I feel the need to warn that I have not been formally educated on ML or AI so any interpretation will just be mine and may not necessarily be the correct one. (If anyone with more experience would like to contribute to this feel free to reach out).

SirHackalot 3mo ago
Please add them on the site for those of us who have never had Twitter and don’t plan to open one ever. Thanks for this compilation, I am — like your friends — trying to get into reading research papers and this is right up my alley right now.
EagnaIonat 3mo ago
Even with the motion/background button toggles you are still left with tiny fonts that make it hard to read. It actually made me check my browser wasn't set to zoom out. But then using zoom changes nothing, which breaks accessibility. Also why does the header need to take up 3/4 of the screen? [edit] Clicking on a paper doesn't even bring the paper up. I have to hit another click to get to it.

amemi 3mo ago

Possibly the original X tweet that popularized this list? 2024, 876k views https://x.com/keshavchan/status/1787861946173186062 https://x.com/keshavchan/status/1787861946173186062 In my opinion, whether it was actually by Ilya or not is not worthy of debate. Many of them are widely recognized for being good pedagogical resources (e.g. annotated transformer, unreasonable effectiveness of RNNs, understanding LSTM networks), and others are landmark papers which anyone interested in the field would benefit from reading: - Krizhevsky et al. (2012) introduced AlexNet - Bahdanau et al. (2014) introduced attention - He et al. (2015) introduced ResNet - Vaswani et al. (2017) introduced the Transformer Other papers are more specialized. Of them, I think Kaplan et al. (2020) by OpenAI is probably most important.

gekoxyz 3mo ago
Even if Ilya didn't really create this list I have a very good opinion about every paper on this page that I've read (most of them) so I think it's a great resource. Lately during my off time I want to do something related to AI research (which I am already doing full time atm so I need something light) and I am for sure going to read through this.

anonymzz 3mo ago

Sutskever's List : https://www.manning.com/books/sutskevers-list https://www.manning.com/books/sutskevers-list

theahura 3mo ago

always a fun day when someone rediscovers these. Lucky 10000 in case folks are interested, i wrote up a ~layman's review of each paper over the course of two years a while back. Several of those reviews ended up doing reasonably well on hn. Full analysis of the ~23 docs that were papers and not massive books https://12gramsofcarbon.com/p/ilyas-30-papers-to-carmack-table https://12gramsofcarbon.com/p/ilyas-30-papers-to-carmack-tab...

nitin7 3mo ago

How is this a beginner friendly format?

mmiao 3mo ago

the format is not friendly at all...

nomilk 3mo ago

This is a beautiful way to present extremely high quality information. I sometimes lament the unpleasant friction involved in finding and reading academic papers (the overly formal style is a necessary evil, but the irritating paywalls, followed by inevitable searches for '%{title} filetype:pdf' feel like unnecessary ones).

anentropic 3mo ago

Has anyone tried just asking John Carmack...?

bookofjoe 3mo ago

Delete the two options in the upper right hand corner of the homepage and Bob's your uncle.

upmind 3mo ago

I wonder how up to date these papers are because I imagine this list came out >6 years ago

dahart 3mo ago
It’s very easy to check & see there’s at least one paper from 2023. Also it takes time to know which papers are influential. But better to contribute than speculate… what are the seminal ML papers from the last 6 years that should be on the list?

joeclark77 3mo ago

Thanks for the effort; I have a little feedback: - Very few of these are labeled with a clear reference to source and date (think APA style). - They don't seem to be in order chronologically. I would think "essential" papers would be read from the earlier "foundational" works up to the more recent ones that work on them. - The oldest ones seem to date to the 2010s. I think it would be hard to jump in at such a recent point. The basics of machine learning come from the 1960s-70s. If you want to improve it I would recommend coming up with a sequential "reading list" including a few classic papers, some intermediate advancements (frequently referenced), and then a few new, cutting edge articles.

mfaulk 3mo ago

I read most of those papers a few years ago and wrote up an overview and summaries: https://mfaulk.github.io/2024/06/19/ilya-papers.html https://mfaulk.github.io/2024/06/19/ilya-papers.html

hwj 3mo ago

The "ML" here seems to stand for "Machine Learning" (not "Meta Language" as in SML or OCaml).

_giorgio_ 3mo ago

Ilya has never made the list public. The list is simply made up, as confirmed by John Carmack. """ John Carmack @ID_AA_Carmack I rather expected @ilyasut to have made a public post by now after all the discussion of the AI reading list he gave me. A canonical list of references from a leading figure would be appreciated by many. I would be curious myself about what he would add from the last three years. """ https://x.com/ID_AA_Carmack/status/1622673143469858816 https://x.com/ID_AA_Carmack/status/1622673143469858816