4 ms·
Ash HN: Who is cloning my GitHub repo?
I have several github repos that frequently get cloned according to the Insights > Traffic page.
These repos are of no special interest to anyone, at least not on a level that will make regular cloning beneficial.
My best guess was that these are crawlers that regularly clone all repos on github, but the clones aren't evenly spaced so it doesn't look automatic.
Are these git clone-crawlers after all?
- version_five 3y agoI've seen the same thing. Presumably there's lots of misc crawling and scraping going on so you wouldn't expect it to regular, it's not just one org doint periodic updates.
- cratermoon 3y agoLLM training models
- puttycat 3y agoWhy not crawl directly from the github frontend though?
- deleted 3y ago[deleted]
- cratermoon 3y agoGood question. I'm making the assumption that cloning an entire repo with all the history and configuration is much easier and generates less total request traffic than hitting the front end or even using the API.
- Daeraxa 3y agoThere are places that archive public repos for preservation reasons - like https://archive.softwareheritage.org/ https://archive.softwareheritage.org/
- enescakir 3y agoDo you have any CI pipelines?