3 ms·
What? That's crazy! Does Github block indexing?
by oblio 5y ago
What? That's crazy! Does Github block indexing?
- beembeem 5y agohttps://github.com/robots.txt https://github.com/robots.txt I don't see anything here about wiki specifically but maybe one of the rules hits wiki pages?
- crazysim 5y agoThey've moved from robots.txt to blocking by headers.
- bckygldstn 5y agoThere's more details on https://github-wiki-see.page/ https://github-wiki-see.page/ and https://github.com/github/feedback/discussions/4992#discussioncomment-1448177 https://github.com/github/feedback/discussions/4992#discussi... > we have also introduced an x-robots-tag: none in the http response header of Wiki pages > Abusive behavior in Wikis had a negative impact on our search engine ranking > GitHub is currently permitting a select criteria of GitHub Wikis to be indexed