3 ms·Look at robots.txt first. If the website owner says there DO NoT CRAWL, uou cannot use that data.by eb0la 10y agoLook at robots.txt first. If the website owner says there DO NoT CRAWL, uou cannot use that data.