This is really cool. I worked on the AI Index Report and had to bug karpathy to get his copy of arxiv papers to do analysis. (He's been slowly collecting papers

This is really cool. I worked on the AI Index Report and had to bug karpathy to get his copy of arxiv papers to do analysis. (He's been slowly collecting papers for arxiv sanity for years). Getting them all from the arxiv API would have taken months. This will enable tons of useful stats gathering about the fields represented on arxiv. Hopefully it will also lead to new scientific insights as well!