3 ms·

Yes, it's a real problem for anyone offering any sort of search capabilities. Like, about 0.5% of the traffic to my search engine is human. I'm not aware of a

Yes, it's a real problem for anyone offering any sort of search capabilities. Like, about 0.5% of the traffic to my search engine is human. I'm not aware of any search engine that doesn't have similar stats.

hskalin 3y ago
Off topic: how do you determine what percentage of search is coming from humans?
- marginalia_nu 3y ago
  Well about 99% of the search requests I got back when I was using cloudflare couldn't get past their bot-mitigation, and of what made it through, at least half looked very automated.
  - superkuh 3y ago
    I'm a human and I can't get past cloudflare "bot mitigation" with my browser. Bot mitigation actually just means your browser executing the latest bleeding edge javascript functions to make sure your behavior is monetizable.
    - marginalia_nu 3y ago
      No that's not actually true at all. The website always worked with text-only browsers, cloudflare or not. Thoroughly tested with the likes of w3m and dillo. Virtually all of the traffic that was intercepted claimed to be modern Chrome or Safari or similar, which should be capable of "executing the latest bleeding edge javascript functions". The primary reason why anyone gets shit from bot mitigation is IP reputation, this is far more important (and effective) than looking at browser characteristics.
crtasm 3y ago
Github could require captcha for non-logged in users, I suppose.
- dylan604 3y ago
  I'm a human, yet I am unable to get past Steam's captcha. It is not the only site that I cannot prove to not be a robot. I'm guessing the number of collateral damage is worth it to them. I'm not a big gamer, and wouldn't be a big source of revenue for them anyway.
  - Kuinox 3y ago
    Steam has a captcha ?
    - kuchenbecker 3y ago
      No Dylan604 is a suspected Cylon /s. Many companies will black-hole you and force infinite captias despite solving them correctly to waste resources.

hskalin 3y ago

Off topic: how do you determine what percentage of search is coming from humans?

marginalia_nu 3y ago
Well about 99% of the search requests I got back when I was using cloudflare couldn't get past their bot-mitigation, and of what made it through, at least half looked very automated.
- superkuh 3y ago
  I'm a human and I can't get past cloudflare "bot mitigation" with my browser. Bot mitigation actually just means your browser executing the latest bleeding edge javascript functions to make sure your behavior is monetizable.
  - marginalia_nu 3y ago
    No that's not actually true at all. The website always worked with text-only browsers, cloudflare or not. Thoroughly tested with the likes of w3m and dillo. Virtually all of the traffic that was intercepted claimed to be modern Chrome or Safari or similar, which should be capable of "executing the latest bleeding edge javascript functions". The primary reason why anyone gets shit from bot mitigation is IP reputation, this is far more important (and effective) than looking at browser characteristics.

crtasm 3y ago

Github could require captcha for non-logged in users, I suppose.

dylan604 3y ago
I'm a human, yet I am unable to get past Steam's captcha. It is not the only site that I cannot prove to not be a robot. I'm guessing the number of collateral damage is worth it to them. I'm not a big gamer, and wouldn't be a big source of revenue for them anyway.
- Kuinox 3y ago
  Steam has a captcha ?
  - kuchenbecker 3y ago
    No Dylan604 is a suspected Cylon /s. Many companies will black-hole you and force infinite captias despite solving them correctly to waste resources.