3 ms·
No robots.txt is basically the same as the default contents of a robots.txt saying "All bots welcome"
by joshmanders 4y ago
No robots.txt is basically the same as the default contents of a robots.txt saying "All bots welcome"
- vertis 4y agoNo robots.txt is unpredictable because it depends on the implementation of the bot.