3 ms·
Show HN: Robots.txt Checker (CoffeeScript, Node) w. Bookmarklet
- alphadown 15y agoHow about a robot.txt generator. Maybe it shows the user all the directories of the website (reads the sitemap) and then allows the user to disallow specific documents and directories in a point and click fashion. The generator then writes it in the proper syntax. Yea? Nay?