> Is removing content the same as disallowing it in a robots.txt? No. > Because they have an interesting article [1] disallowed in their robots.txt that was a

> Is removing content the same as disallowing it in a robots.txt? No. > Because they have an interesting article [1] disallowed in their robots.txt that was added a while after it was published. Oddly, it's a different seemingly benign story for me: Disallow: /sections/health-shots/2013/03/11/173816690/new-voices-for-the-voiceless-synthetic-speech-gets-an-upgrade IDK why they include a seemingly random story in their robots.txt. I don't think the intention is to hide the story, though.