3 ms·

> "and those user profiles are visible to Google" googlebot adheres to robots.txt, right? in which case couldn't self-hosted gitlab admins add a robots.txt en

• jefftk 6y ago
  That requires the spammers to notice that it's blocked in robots.txt, which seems optimistic
• dspillett 6y ago
  Most bots aren't going to bother checking if you've done that, so while they'll not get the expected benefit you'll still get the spam.
• pitay 6y ago
  There is also adding the rel="nofollow ugc" to user submitted links that removes the benefits of linking for spammers. https://support.google.com/webmasters/answer/96569 https://support.google.com/webmasters/answer/96569