3 ms·

> To fight scrapers, we show some values as images that look like text You are fighting screen readers more than anything; as well as legitimate plugins, form

> To fight scrapers, we show some values as images that look like text You are fighting screen readers more than anything; as well as legitimate plugins, form autofills, etc. If this is for captcha, you are fighting all the users as well. > And we insert random (non-visible) html and css classes in our site to screw with em, and use randomized css classnames. Legitimate browser plugins, etc. I'd just use electron or selenium with `nth-child`, `:visible`, `[class*="…"]`, etc. What you effectively doing is wasting time on useless stuff. This is even more useless than trying to prevent copying of DVDs or pirating games.

elchief 10y ago
Except traffic from known scrapers (or what appear to be) is down 20% Sure, xpath and css selector experts can figure it out, but that's not everyone
- comboy 10y ago
  I don't understand, why only 20%? If the traffic is from known scrapers why can't you just render "scrap off" ie easily get rid of them? And traffic from good scrapers is of course pretty much impossible to measure so you don't know how big percentage of scrapers you got rid of in total.
  - tedunangst 10y ago
    If the scraper gets back nothing, they know they've been spotted and will make adjustments. Easy to check for automatically. If you alter the page to feed them garbage, it takes longer to notice.
  - elchief 10y ago
    It's easy for someone viewing some logs to say "ok this is very likely automated scraping", though it can be harder to automate detecting this. In the same way that porn is obvious to a human, but not a computer.
    - elmigranto 10y ago
      But it is not magic, though, and surely can be automated. In fact, I would argue that your time might be better spent on this, instead of randomizing CSS classes. If you end up building something worthwile, that could be a great product too! (Look at all those CDNs / Anti-DDoS platforms, sounds like they could've been started this exact way.)
BrandonMarc 10y ago
> What you effectively doing is wasting time on useless stuff. This is even more useless than trying to prevent copying of DVDs or pirating games. Can you be so sure? The Union blockade of the Confederacy had plenty of holes, and smugglers / privateers / blockade-runners made good money getting through (when they survived) ... but that doesn't mean the blockade wasn't effective all the same at weakening the Confederate military and economy.
- elmigranto 10y ago
  Do you really not see the difference between military blockade and randomizing CSS classes?
  - BrandonMarc 10y ago
    Honestly, no. This one time I was pissed off at Egypt for undercutting me in cotton prices, so I tried to set up a blockade to prevent merchant ships going in and out of Cairo. ... and it would have worked, too, except for my naval vessels were all CSS classes. I even tried to name them cleverly, a la "USS hero unit" or "USS datatable table-consensed span9", but my plan was foiled.
- emodendroket 10y ago
  Only one guy has to beat it for it to be widely disseminated, though.

elchief 10y ago

Except traffic from known scrapers (or what appear to be) is down 20% Sure, xpath and css selector experts can figure it out, but that's not everyone

comboy 10y ago
I don't understand, why only 20%? If the traffic is from known scrapers why can't you just render "scrap off" ie easily get rid of them? And traffic from good scrapers is of course pretty much impossible to measure so you don't know how big percentage of scrapers you got rid of in total.
- tedunangst 10y ago
  If the scraper gets back nothing, they know they've been spotted and will make adjustments. Easy to check for automatically. If you alter the page to feed them garbage, it takes longer to notice.
- elchief 10y ago
  It's easy for someone viewing some logs to say "ok this is very likely automated scraping", though it can be harder to automate detecting this. In the same way that porn is obvious to a human, but not a computer.
  - elmigranto 10y ago
    But it is not magic, though, and surely can be automated. In fact, I would argue that your time might be better spent on this, instead of randomizing CSS classes. If you end up building something worthwile, that could be a great product too! (Look at all those CDNs / Anti-DDoS platforms, sounds like they could've been started this exact way.)

BrandonMarc 10y ago

> What you effectively doing is wasting time on useless stuff. This is even more useless than trying to prevent copying of DVDs or pirating games. Can you be so sure? The Union blockade of the Confederacy had plenty of holes, and smugglers / privateers / blockade-runners made good money getting through (when they survived) ... but that doesn't mean the blockade wasn't effective all the same at weakening the Confederate military and economy.

elmigranto 10y ago
Do you really not see the difference between military blockade and randomizing CSS classes?
- BrandonMarc 10y ago
  Honestly, no. This one time I was pissed off at Egypt for undercutting me in cotton prices, so I tried to set up a blockade to prevent merchant ships going in and out of Cairo. ... and it would have worked, too, except for my naval vessels were all CSS classes. I even tried to name them cleverly, a la "USS hero unit" or "USS datatable table-consensed span9", but my plan was foiled.
emodendroket 10y ago
Only one guy has to beat it for it to be widely disseminated, though.