3 ms·

Github for some reason disables search engine indexing of wikis, not sure about discussions.

shagie 4y ago
The problem that they have (and other user content created sites) if indexing was on is that SEO spammers use it to dump lots of links to sites. That you don't see this is an indication that it works and dissuading those spammers. Consider all the nearly forgotten repos out there that you occasionally stumble over and that there are no "work from home and get $30/hour - click here" or a dozen forms of "somespam.example.com" links peppering it. Compare that with the old Wordpress blogs out there that haven't disabled comments and yet still have search indexing (or not, but spammers don't always dig far enough to see if there's a nofollow attribute on links). So, for the most part, GitHub isn't indexed by search engines - and its across the board for the entirety of the site. Spammers realize that it is pointless to attempt in any of the forms that they could on other user created content sites and so expend no effort at all to post spam links. Github discussions is searchable within GitHub in that repo. For example https://github.com/laravel/framework/discussions?discussions_q=Faker https://github.com/laravel/framework/discussions?discussions... The wiki is fairly useful - and again, if you are searching for content in GitHub you can search wikis ( https://docs.github.com/en/search-github/searching-on-github/searching-wikis https://docs.github.com/en/search-github/searching-on-github... ) and get only content within GitHub (without having to sift through Google search results) with arguably a more powerful query language than Google's search bar ( https://docs.github.com/en/search-github/getting-started-with-searching-on-github/understanding-the-search-syntax https://docs.github.com/en/search-github/getting-started-wit... ) Personally, if I am looking for content for a FOSS that is hosted on GitHub I use GitHub's search for discussions and wikis first before trying to search for content in google.
- trinovantes 4y ago
  But this doesn't explain why Issues are indexable on Google What's the difference between a spammer creating their own repo and creating a bunch of issues with spam links vs their own repo and creating a bunch of wiki pages with spam links?
  - shagie 4y ago
    The difference is likely the ability for the community to flag and moderate a comment rather than an entire wiki and repository. That said, the wiki workflow, permissions, and limitations in formatting may push people who are after a more durable store of information to create a GitHub pages site which is then managed with the same workflow (forks, pull requests, and reviews) as code, along with the ability for search engines to index the GitHub pages site.

The problem that they have (and other user content created sites) if indexing was on is that SEO spammers use it to dump lots of links to sites. That you don't see this is an indication that it works and dissuading those spammers. Consider all the nearly forgotten repos out there that you occasionally stumble over and that there are no "work from home and get $30/hour - click here" or a dozen forms of "somespam.example.com" links peppering it. Compare that with the old Wordpress blogs out there that haven't disabled comments and yet still have search indexing (or not, but spammers don't always dig far enough to see if there's a nofollow attribute on links). So, for the most part, GitHub isn't indexed by search engines - and its across the board for the entirety of the site. Spammers realize that it is pointless to attempt in any of the forms that they could on other user created content sites and so expend no effort at all to post spam links. Github discussions is searchable within GitHub in that repo. For example https://github.com/laravel/framework/discussions?discussions_q=Faker https://github.com/laravel/framework/discussions?discussions... The wiki is fairly useful - and again, if you are searching for content in GitHub you can search wikis ( https://docs.github.com/en/search-github/searching-on-github/searching-wikis https://docs.github.com/en/search-github/searching-on-github... ) and get only content within GitHub (without having to sift through Google search results) with arguably a more powerful query language than Google's search bar ( https://docs.github.com/en/search-github/getting-started-with-searching-on-github/understanding-the-search-syntax https://docs.github.com/en/search-github/getting-started-wit... ) Personally, if I am looking for content for a FOSS that is hosted on GitHub I use GitHub's search for discussions and wikis first before trying to search for content in google.

trinovantes 4y ago
But this doesn't explain why Issues are indexable on Google What's the difference between a spammer creating their own repo and creating a bunch of issues with spam links vs their own repo and creating a bunch of wiki pages with spam links?
- shagie 4y ago
  The difference is likely the ability for the community to flag and moderate a comment rather than an entire wiki and repository. That said, the wiki workflow, permissions, and limitations in formatting may push people who are after a more durable store of information to create a GitHub pages site which is then managed with the same workflow (forks, pull requests, and reviews) as code, along with the ability for search engines to index the GitHub pages site.