4 ms·
Ask HN: Is there a data set for GitHub repos associated with academic papers?
Codes are often included in academic publications however I haven't seen a list of repos anywhere that is connected to doi numbers from papers or zenodo. Does this already exist somewhere or should I need to create it?
- mks_shuffle 3y agoFor ML/DL papers you can check https://paperswithcode.com/ https://paperswithcode.com/
- 3abiton 3y agoDo they release yearly/periodic bundled datasets or you will have to use their api to make dataset?
- sargstuff 3y ago20+ tools to help you mine and analyze GitHub and Git data[0] "Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations’ COCI: a multidisciplinary comparison of coverage via citations" (2020)[1] "Analyzing the GitHub Repositories of Research Papers"(2020)[2] HN comments on "OpenAlex: The Promising Alternative to Microsoft Academic Graph"[3] has additional related links "The Lens"[4]; Resource reference links to related knowledge graph sites(2022)[5] ----- [0] : https://livablesoftware.com/tools-mine-analyze-github-git-software-data/ https://livablesoftware.com/tools-mine-analyze-github-git-so... [1] : https://link.springer.com/article/10.1007/s11192-020-03690-4 https://link.springer.com/article/10.1007/s11192-020-03690-4 [2] : "Analyzing the GitHub Repositories of Research Papers" : https://livablesoftware.com/tools-mine-analyze-github-git-software-data/ https://livablesoftware.com/tools-mine-analyze-github-git-so... [3] : https://news.ycombinator.com/item?id=31271477 https://news.ycombinator.com/item?id=31271477 [4] : The Lens : https://about.lens.org/the-lens-scholarly-metarecord-strategy-beyond-microsoft-academic-graph/ https://about.lens.org/the-lens-scholarly-metarecord-strateg... [5] : "The Microsoft Academic Knowledge Graph enhanced: Author name disambiguation, publication classification, and embeddings " : https://direct.mit.edu/qss/article/3/1/51/109628/The-Microsoft-Academic-Knowledge-Graph-enhanced https://direct.mit.edu/qss/article/3/1/51/109628/The-Microso... ----- post publication edit addition #1: If have github reference, can try and use [i] to find related papers. [i] https://aarontay.medium.com/3-new-tools-to-try-for-literature-mapping-connected-papers-inciteful-and-litmaps-a399f27622a https://aarontay.medium.com/3-new-tools-to-try-for-literatur...
- Amir6 3y agoGoogle the following; site:arxive.org filetype:pdf “GitHub.com”
- kingkongjaffa 3y agohas 0 results: https://www.google.com/search?hl=en&q=site%3Aarxive.org%20filetype%3Apdf%20%22GitHub.com%22 https://www.google.com/search?hl=en&q=site%3Aarxive.org%20fi... https://www.google.com/search?hl=en&q=site%3Ahttps%3A%2F%2Farxiv.org%2F%20%22github.com%2F%22#ip=1 https://www.google.com/search?hl=en&q=site%3Ahttps%3A%2F%2Fa... seems to be mostly papers about github itself.
- sargstuff 3y ago1) find the doi number [1a][1b] 2) find sources that cite the doi number -> google scholar[2][3] 3) filter for 'github' ----- [1a]resolve a doi name : https://dx.doi.org/ https://dx.doi.org/ [1b]find a doi number : https://answers.lib.iup.edu/faq/31945 https://answers.lib.iup.edu/faq/31945 [2] : https://scholar.google.com/ https://scholar.google.com/ [3] : google with "site:http://doi.org/ http://doi.org/" [4] : finding a doi in document page : https://stackoverflow.com/questions/27910/finding-a-doi-in-a-document-or-page https://stackoverflow.com/questions/27910/finding-a-doi-in-a...
- aaron695 3y ago[dead]
- ebfe1 3y agoOne idea: You can play with the github dataset in ClickHouse playground. This is just a quick sql query to look for the DOI number pattern mentioned in any comment on repositories: https://play.clickhouse.com/play?user=play#c2VsZWN0IHJlcG9fbmFtZSxldmVudF90eXBlLGJvZHkgZnJvbSBnaXRodWJfZXZlbnRzIHdoZXJlIGV2ZW50X3R5cGUgaW4gKCdJc3N1ZUNvbW1lbnRFdmVudCcsJ0lzc3Vlc0V2ZW50JywnUHVsbFJlcXVlc3RFdmVudCcsJ1B1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudEV2ZW50JykgYW5kIG1hdGNoIChib2R5LCcuKjEwXC5cZHs0LDl9XC9bLVwuXzsoKVwvOkEtWjAtOV0rLionKSBsaW1pdCAxMA== https://play.clickhouse.com/play?user=play#c2VsZWN0IHJlcG9fb... ``` select repo_name,event_type,body from github_events where event_type in ('IssueCommentEvent','IssuesEvent','PullRequestEvent','PullRequestReviewCommentEvent') and match (body,'.10\.\d{4,9}\/[-\._;()\/:A-Z0-9]+.') limit 10 ``` perhaps you can extend on that :)