3 ms·
Show HN: caniscrape – Analyze anti-bot protections before scraping (CLI and Web)
I built this after hitting an infinite loop pagination trap on Newegg and wasting hours building a scraper that never had a chance.
caniscrape analyzes a URL and tells you:
- What protections are active (WAF, CAPTCHA, rate limits, TLS fingerprinting, honeypots)
- Difficulty score (0-10)
- What tools/approach you'll need
Install: pip install caniscrape
Usage: caniscrape https://example.com https://example.com
Web version: https://caniscrape.org https://caniscrape.org
NOTE: Very hard sites like Amazon or YouTube will show as 0 because their bot protections are essentially invisible, I will fix this in a future update.
NOTE FOR WEB VERSION:
The web version uses a cloud IP which is blacklisted by major sites. I will add proxy rotations to defeat this but for now, the website is only good for easy/medium sites.