2 ms·
Maybe because their robots.txt disallows that? https://www.youtube.com/robots.txt https://www.youtube.com/robots.txt User-agent: * ... Disallow: /
by pbalau 11mo ago
Maybe because their robots.txt disallows that?
https://www.youtube.com/robots.txt https://www.youtube.com/robots.txt
User-agent: *
...
Disallow: /feeds/videos.xml
...