5 ms·

Yeah, I used to work for one of the major anti-bot vendors. Customers weren't clueless. Nobody buys these solutions because they're so much fun, it's a cost cen

Yeah, I used to work for one of the major anti-bot vendors. Customers weren't clueless. Nobody buys these solutions because they're so much fun, it's a cost center and they monitor their ROI quite closely. Credit card charge backs, impact to infrastructure, extra incurred cost due to underlying api's (like in the Airline industry in particular) etc are all reasons why bot mitigation is a better option than nothing for a lot of companies, even if it's not 100% effective.

dzhiurgis 5y ago
Do you know much about airline api pricing more? I’ve noticed most sites won’t let you search business fares efficiently, so I made my own for Google Flights which only worked for like 6months until they added bunch of changes that made it near impossible to scrape.
- oxymoron 5y ago
  Yeah, there’s a central service that all Flight search is connected to, irregardless of airline. The airlines are charged per search to that api, so they monitor their ”look to book” rationvery closely. That ratio remains quite stable im the absence of bots, but skyrockets with any bot activity. Hence, they know from that metric how big of a bot problem they have and how much money they are losing. Major flight search software vendors have dedicated teams for this.
  - namdnay 5y ago
    In fact the airlines are charged per book, but if and only if the look to book stays within reasonable bounds. If it rockets up, they’re on the hook for the penalties
    - oxymoron 5y ago
      Thanks for the clarification! I probably misremember some of the details.
spookthesunset 5y ago
Not to mention a lot of these bots are after scamming the company’s own customers. Breaking into accounts to commit fraudulent activity, to reach out and “recruit” people into whatever scam they are trying to run. Nobody wants to spend time trying to stop these bots. It is, however, a very necessary thing to do.
azalemeth 5y ago
You very much missed the false positive rate! I'm fed up of being classed as a bot just because I browse with uMatrix, a Linux user agent, and a ton of ad filtering and anonymisation tech. I had to try to log in to my bank about ten times today because their js-crap website didn't like me (grumble why does it even need to ask for my desktop's accelerometer data via js...) Stuff like this is a pain beyond pain. I really hope that the clients you mention know that they piss off a proportion of their users with every move they take.
- charcircuit 5y ago
  >I'm fed up of being classed as a bot just because I browse with uMatrix, a Linux user agent, and a ton of ad filtering and anonymisation tech. Have you tried not using these things? Anonymity is exactly what bots want. They want to be able to post a spam message every single second and be impossible to ban since they are anonymous. The internet can't function if people are allowed to be anonymous.
  - wumpus 5y ago
    This is not quite up there with "won't someone think about the children!!!!", but still, it's sad. Fortunately, almost all of the websites I visit with my anonymized browser aren't places that I wish to attempt to post a message. Unfortunately, I can easily run into defenses of an entire site when the problem is spam sending.
  - selfhoster11 5y ago
    > The internet can't function if people are allowed to be anonymous. You must have missed the first 20 or so years of its existence, if that's your position.
    - orf 5y ago
      It was much smaller, and spam messages where everywhere.
    - charcircuit 5y ago

dzhiurgis 5y ago

Do you know much about airline api pricing more? I’ve noticed most sites won’t let you search business fares efficiently, so I made my own for Google Flights which only worked for like 6months until they added bunch of changes that made it near impossible to scrape.

oxymoron 5y ago
Yeah, there’s a central service that all Flight search is connected to, irregardless of airline. The airlines are charged per search to that api, so they monitor their ”look to book” rationvery closely. That ratio remains quite stable im the absence of bots, but skyrockets with any bot activity. Hence, they know from that metric how big of a bot problem they have and how much money they are losing. Major flight search software vendors have dedicated teams for this.
- namdnay 5y ago
  In fact the airlines are charged per book, but if and only if the look to book stays within reasonable bounds. If it rockets up, they’re on the hook for the penalties
  - oxymoron 5y ago
    Thanks for the clarification! I probably misremember some of the details.

spookthesunset 5y ago

Not to mention a lot of these bots are after scamming the company’s own customers. Breaking into accounts to commit fraudulent activity, to reach out and “recruit” people into whatever scam they are trying to run. Nobody wants to spend time trying to stop these bots. It is, however, a very necessary thing to do.

azalemeth 5y ago

You very much missed the false positive rate! I'm fed up of being classed as a bot just because I browse with uMatrix, a Linux user agent, and a ton of ad filtering and anonymisation tech. I had to try to log in to my bank about ten times today because their js-crap website didn't like me (grumble why does it even need to ask for my desktop's accelerometer data via js...) Stuff like this is a pain beyond pain. I really hope that the clients you mention know that they piss off a proportion of their users with every move they take.

charcircuit 5y ago
>I'm fed up of being classed as a bot just because I browse with uMatrix, a Linux user agent, and a ton of ad filtering and anonymisation tech. Have you tried not using these things? Anonymity is exactly what bots want. They want to be able to post a spam message every single second and be impossible to ban since they are anonymous. The internet can't function if people are allowed to be anonymous.
- wumpus 5y ago
  This is not quite up there with "won't someone think about the children!!!!", but still, it's sad. Fortunately, almost all of the websites I visit with my anonymized browser aren't places that I wish to attempt to post a message. Unfortunately, I can easily run into defenses of an entire site when the problem is spam sending.
- selfhoster11 5y ago
  > The internet can't function if people are allowed to be anonymous. You must have missed the first 20 or so years of its existence, if that's your position.
  - orf 5y ago
    It was much smaller, and spam messages where everywhere.
  - charcircuit 5y ago