4 ms·
> Firstly, I'm aware of human-based captcha-defeating systems. You describe captchas that "cannot be outsourced to a mechanical turk-based API". I'm wasn't awar
by dsacco 9y ago
> Firstly, I'm aware of human-based captcha-defeating systems. You describe captchas that "cannot be outsourced to a mechanical turk-based API". I'm wasn't aware such systems existed, that sounds scary. Do you have any examples?
Google's latest captcha specification (and similarly sophisticated systems) must be completed in a small window of time, change rapidly, have a varying number of "rounds", and are extremely antagonistic to being reloaded in e.g. a frame.
Practically speaking, you can't consistently outsource that to a third party API that uses humans to click and verify the images.
> Secondly, I'm a little confused by your mention that #1 will simply raise the latency of the request to 10-20 seconds instead of completely blocking it.
For captcha systems which can be defeated by humans, they will tolerate an extended period of time before they're solved. This means the amount of time it takes your crawler to send the data to the API and receive the challenge response, then send it to the captcha on the page will not cause the test to fail. It will, however, introduce a significant amount of latency in your requests.
> Thirdly, in the case where a captcha (eg, ReCAPTCHA) can be defeated by a captcha-filling service... well, such services are extremely cheap, and there's no per-account fee or overhead. It sounds plausible that one could simply sign up for the captcha-filling service multiple times, and submit captchas to defeat in parallel. Could this work?
Sure, but what are you gaining? These things are usually priced on a per-captcha basis. So you could have multiple accounts, but you'll be paying the same as just sending multiple requests to the API, and the concurrent requests are routed via the API's backend to different human operators for a solution.
- yorwba 9y ago> Practically speaking, you can't consistently outsource that to a third party API that uses humans to click and verify the images. It sounds like you can outsource it, but you essentially need a remote desktop connection for the captcha solver to interact with the page. The overhead might be a bit much, but it doesn't seem impossible.
- exikyut 9y ago> Google's latest captcha specification (and similarly sophisticated systems) must be completed in a small window of time, change rapidly, have a varying number of "rounds", and are extremely antagonistic to being reloaded in e.g. a frame. Hmmm. I've observed the timing thing myself - I'll sometimes hit the captcha first while I fill the rest of a form out, then it'll expire, and I'll need to do it again. What do you mean by "change rapidly" and "varying number of "rounds""? Regarding avoiding loading the content in a frame - what about making the initial iframe load happen on the machine of the person filling the captcha? Why can't this work? -- > Practically speaking, you can't consistently outsource that to a third party API that uses humans to click and verify the images. I'm both very curious and somewhat surprised this is the case. -- >> It sounds plausible that one could simply sign up for the captcha-filling service multiple times, and submit captchas to defeat in parallel. Could this work? > Sure, but what are you gaining? ... you'll be paying the same as just sending multiple requests to the API, and the concurrent requests are routed via the API's backend to different human operators for a solution. This is what I'd strongly expect, but I very much wonder if captcha-filling services ratelimit the number of filled captchas per account. In theory (now I think about it) ratelimiting would hurt the service provider so this might be unlikely, but where I want to play with massive parallelism I do wonder if sharding hundreds of requests across multiple accounts would be useful or not. Sharding across multiple providers would probably increase parallel throughput by a small amount though. Thanks for the feedback!