maltz

3 ms·

1. 1.▲
   Mastering Web Scraping in Python: Scaling to Distributed Crawling(zenrows.com)

   7 points by maltz 5y ago|0 comments
2. 2.▲
   by maltz 5y ago

   You should probably take a look at this. https://www.zenrows.com/blog/stealth-web-scraping-in-python-...
3. 3.▲
   by maltz 5y ago

   YouTube has "var ytInitialData" & "var ytInitialPlayerResponse" params hardcoded in HTML. No need to run JS!
4. 4.▲
   by maltz 5y ago

   Yep, seems so. https://www.zenrows.com/pricing
5. 5.▲
   by maltz 5y ago

   If pages are constructed client-side, the content you are looking for is either hardcoded as JSON in the HTML or loaded via XHR request. Scrape that.
6. 6.▲
   by maltz 5y ago

   Playwright. It can be easily used with JS, Python, Go, Java, etc.
7. 7.▲
   by maltz 5y ago

   It's part of a series of blog posts that talks explicitly about crawling. There are indeed other links that do better explaining advanced extraction techniques. Extraction => https://www.zenrows.com/blog/masteri
8. 8.▲
   Mastering Web Scraping in Python: Crawling from Scratch(zenrows.com)

   233 points by maltz 5y ago|81 comments
9. 9.▲
   Stealth Web Scraping in Python: Avoid Blocking Like a Ninja(zenrows.com)

   2 points by maltz 5y ago|0 comments
10. 10.▲
    Mastering Web Scraping in Python: From Zero to Hero(zenrows.com)

    6 points by maltz 5y ago|0 comments