4 ms·
1. Bibliography Blues The papers have bibliographies with citations in text. The knowledge you need to read a given paper usually lies within 3 degrees of bibl
by barfbagginus 2y ago
1. Bibliography Blues
The papers have bibliographies with citations in text. The knowledge you need to read a given paper usually lies within 3 degrees of bibliographical separation.
You can also use reverse citation search on Google scholar to find newer works that cite a given work. This can uncover a review or textbook, which often makes the concept much simpler to digest. And it can be a more advanced approach which gives you an alternative view or lets you abandon the original paper altogether.
2. Back to tha basics
For math, it would help if you have college level calculus, differential equations, linear algebra, and statistics. If you don't have that, be ready to assign yourself math readings and homework. It would also really help if you had analysis, topology, abstract algebra, and category theory under the belt, just so you can see the mathematical forest for the trees... But that's not really so necessary
You can often find the author's preferred reference for a given work in the bibliography, and use that.
For computer science, it helps to be able to read algorithm pseudocodes, follow complexity and correctness proofs, and understand performance experiments and charts. Skimming Knuth's "Art of Computer Programming" should be very helpful.
You'll need some algorithmic/statistical learning theory as well, since authors often use these concepts to prove that a system can learn. Though TBH, I usually skip these proofs and focus on the performance details. I usually find a higher performance paper before I need to care about the learning proofs. Still it helps to expect them and have an idea of how they work, if only to know when you can safely ignore them.
3. Narrative Arcs
Many AI results build on keystone papers that define eras. We can build on these papers by understanding their philosophy and deeper implications, even if we don't really understand their math. There's a story that you will tell yourself, and it may lead to interesting research or business results.
For example, if you read "Attention is all you need", you set the stage for LLMs. then "LLMs are in context learners", followed by the "chain of thought", "reflexion", and "toolformer" paper will give you a great sense of how LLMs are emerging as more than just stochastic parrots, ready to be developed and deployed in a lot of ways that overcome their issues. But looking at how bad the numbers can be even with our best efforts, you also get the sense of how much work these systems will need to actually be good, and the sense that corner cutting will lead to a lot of pretty awful systems.
It's a fun time to be a computer scientist or software engineer! Lots of cool papers to read!
- smeej 2y agoWow! I'm glad there are people like you out there to be excited about it! I glazed over before I even got to the end of your comment, with the dazzled sort of awestruck glaze where I realize I will never do even a tiny fraction of the work required to know things about this topic you probably take totally for granted. Thanks for taking the time to expose me to your enthusiasm!