3 ms·
It's an easy fix if Google cared. Have an online tool that validates if the robots.txt is correct, and send out an announcement that files that don't meet spec
by asdfman123 7y ago
It's an easy fix if Google cared. Have an online tool that validates if the robots.txt is correct, and send out an announcement that files that don't meet spec will be penalized in terms of SEO.
- DEADBEEFC0FFEE 7y agoCould also make the spec easier to be compliant.
- pbhjpbhj 7y agoIIRC Google do automated checks on robots.txt and report in webmaster tools if you did something that looks crazy. https://qph.fs.quoracdn.net/main-qimg-002d1f819e1bfbbd14fa2de937b489d0 https://qph.fs.quoracdn.net/main-qimg-002d1f819e1bfbbd14fa2d... shows some of the interface, but I seem to recall getting notified in webmaster tools when I messed up the robots.txt on a particular site.
- hombre_fatal 7y agoThat just punishes users by sinking relevant results for reasons users couldn’t possibly care about.
- PunksATawnyFill 7y agoI enjoy the hypocrisy of Google punishing sites that aren't "mobile-friendly," and then deliberately disabling ZOOMING on their own mobile sites.