3 ms·
> maybe their robots file only allows google? It's pretty easy to check: https://forums.developer.ebay.com/robots.txt https://forums.developer.ebay.com/robots.
by thomasahle 9y ago
> maybe their robots file only allows google?
It's pretty easy to check: https://forums.developer.ebay.com/robots.txt https://forums.developer.ebay.com/robots.txt
Sitemap: /sitemap.xml
User-Agent: sitebot
Disallow: /
User-Agent: *
Disallow: /accounts/
Disallow: /users/
Disallow: /revisions/
Disallow: /search
Disallow: /*sort=newest
Disallow: /*sort=hottest
Disallow: /*sort=votes
Disallow: /commands/
Disallow: /badges/
Disallow: /comments
Disallow: /storage/
Doesn't seem like anything disallows https://forums.developer.ebay.com/questions/20235/listing-items-in-store-subcategory-by-name-no-cate.html https://forums.developer.ebay.com/questions/20235/listing-it...