3 ms·
CDP is great for testing. But one of the most basic checks for bot detection is checking for CDP(webdriver). Its always going to be a cat and mouse game. You'll
by Tsarp 1y ago
CDP is great for testing. But one of the most basic checks for bot detection is checking for CDP(webdriver). Its always going to be a cat and mouse game. You'll see a bunch of solutions captch solvers etc, But they usually are only good for a few weeks.
- Tsarp 1y agoWith stuff like https://www.cloudflare.com/en-in/application-services/products/turnstile/ https://www.cloudflare.com/en-in/application-services/produc... and https://blog.cloudflare.com/ai-labyrinth/ https://blog.cloudflare.com/ai-labyrinth/ big money going on both sides last thing you want is to shadow detected as a bot. Its all ok if you are scraping to top rated SEO slop which is usually static sites but for anything beyond it wont work well eventually. Quite a few issues on browerbase, crawl4ai and similar repos around being detected as a bot.
- yencabulator 1y agoThere's no reason why the same cat and mouse game wouldn't apply to this browser as a whole.
- Tsarp 1y agoTrue, but its orders of magnitude lesser when webdriver flag is an extremely basic bot check that is now considered 101.
- yencabulator 1y agoIt sounds like you're thinking of window.navigator.webdriver, which is a WebDriver thing not part of Chrome DevTools Protocol. With CDP, as far as I can tell the detection mechanisms are more about the heuristics of e.g. how fast a form is filled -- which this AI stuff will trigger immediately too. (And even if CDP had an explicit marker somewhere, surely patching that out is easier than piling up enough patches to "make a new browser".)
- Tsarp 1y agoDont you need to navigator.webdriver === true for CDP to drive automation? Maybe I need to update my understanding on this. THis is usually a dead giveaway
- yencabulator 1y agoI see mentions that (unpatched) webdriver is easy to detect but detecting CDP only works by heuristics on timing etc.