4 ms·

I found puppeteer very nice to script against if you need a real headless browser: https://github.com/puppeteer/puppeteer https://github.com/puppeteer/puppetee

• dorianmariefr 5y ago
  I like to use capybara https://github.com/teamcapybara/capybara https://github.com/teamcapybara/capybara
• simonw 5y ago
  I've just started switching from Puppeteer to Playwright - pretty much the exact same functionality, but in a more actively maintained, tighter package (and with great language bindings for JavaScript, Python, .NET and Java. I wrote a bit about that here: https://simonwillison.net/2022/Mar/10/shot-scraper/#how-it-works-playwright https://simonwillison.net/2022/Mar/10/shot-scraper/#how-it-w...
• ydant 5y ago
  [Playwright](https://playwright.dev/ https://playwright.dev/) (Node / Python) is my current preferred - mainly because I seem to have less reliability issues with the browser starting/stopping cleanly (although it's never perfect with any of the tools I've tried).
• lysecret 5y ago
  I used selenium. Really like it and very well maintained.