3 ms·
How do I create a mirror of Github?
by mamon 6y ago
How do I create a mirror of Github?
- sdesol 6y agoYou can't, unless your question is "how do I mirror a github repo?" I'm currently indexing thousands of GitHub repos for analytics reasons, and if you don't play nice (fetch infrequently), GitHub will throttle/terminate your connection.
- threeseed 6y agoGitHub isn't the first company to do this. You just need to use one of the many scraping proxy services which will give you a HTTP proxy that cycles through millions of IP addresses. It's impossible to block since they are largely residential IPs controlled by malware. It definitely works but you are encouraging them which isn't a nice feeling.