2 ms·
I'm pretty sure the Bing Crawler does respect robots.txt. The data Bing collected didn't come from spidering Google.
by stanleydrew 16y ago
I'm pretty sure the Bing Crawler does respect robots.txt. The data Bing collected didn't come from spidering Google.