3 ms·
Show HN: A Node.js script powered by Puppeteer for undetectable web scraping
This is a Node.js script that leverages Puppeteer with extra settings to create a web crawler that avoids detection. This tool allows you to scrape websites while minimizing the risk of being blocked or identified as a bot.
- dotancohen 3y agoInteresting. What makes this project more undetectable than other approaches? Will this bypass Cloudflare security checks? Can it screenshot? Can it close pop-up dialogues (sign in, sign up, cookies) before screenshoting?
- darkotodoric 3y agoThis is a demonstration of how you can avoid detection from the "coding side". Of course, a big part of crawling is a proxy list that you use and how aggressive your crawler is. If you don't know how to code this script will not help you.