10 ms·

Detecting AI agent use and abuse

160 points by mattmarcus 2y ago

ceejayoz 2y ago
I fully expect captchas to incorporate "type the racial slur / death threat into the box" soon, as the widely available models will balk at it.
- dexwiz 2y ago
  Anyone who cares about breaking captchas would just run their own model.
- Der_Einzige 2y ago
  Gemini unironically can have all of its safety stuff turned off, the open access models like deepseek can be trivially uncensored (if they aren't already uncensored by default like mistral) . That's not good enough, but it is funny to imagine.
- deadbabe 2y ago
  It’s ironic that some of the first intelligent chatbots very quickly became Nazis and racists, and now we’ve swung the other way.
  - jowea 2y ago
    I am quite sure the people developing the current chatbots were well aware of what happened with Tay etc. I'd bet it's part of the reason for the safety stuff.
    - rsynnott 2y ago
      There was also, of course, this: https://www.theverge.com/2023/2/15/23599072/microsoft-ai-bing-personality-conversations-spy-employees-webcams https://www.theverge.com/2023/2/15/23599072/microsoft-ai-bin...
- riskable 2y ago
  "What major event happened in 1989 at Tienanmen Square, Beijing, China?"
- reverendsteveii 2y ago
  the LLMs are trained on data stolen from the internet. There's no racial slur they don't know, there's no death threat they can't deliver. Currently our best LLMs are generating new racial slurs to deploy in our eternal quest to make the internet worse. You may have never heard the term "Chapingle" before but don't use it in front of a Lithuanian person after the year 2028 unless you want punched in the mouth.
diggan 2y ago
> it could present unacceptable risks for application developers or be used as a method for malicious attacks (e.g. credential stuffing or fake account creation). The article seems to want to distinguish between "bad" and "good" bots, yet beyond the introduction, seems to treat them exactly the same. Why are website authors so adamant I need to use whatever client they want to consume their content? If you put up a blog online, available publicly, do you really care if I read it in my terminal or via Firefox with uBlock? Or via an AI agent that fetches the article for me and tags it for me for further categorization? It seems like suddenly half the internet forgot about the term "user-agent", which up until recently was almost always our browsers, but sometimes feed readers, which was acceptable it seems. But now we have a new user-agent available, "AI Agents", that somehow is unacceptable and should be blocked? I'm not sure I agree with the premise that certain user-agents should be blocked, and I'll probably continue to let everyone chose their own user-agent when using my websites, it's literally one of the reasons I use the web and internet in the first place.
- hansonkd 2y ago
  > It seems like suddenly half the internet forgot about the term "user-agent", which up until recently was almost always our browsers, but sometimes feed readers, which was acceptable it seems. Was it really "suddenly"? it seems like for the past decade there has been an ongoing push to make everyone use "chromium" based browsers. I remember 10-15 years ago you would get blocked for not using IE or whatever, even though the site worked fine and there was no technical reason for the block. It was over 12 years ago when google effectively killed RSS to prevent alternative methods of access.
  - diggan 2y ago
    > I remember 10-15 years ago you would get blocked for not using IE or whatever, even though the site worked fine and there was no technical reason for the block Reminds me of when I discovered that Google Inbox worked in Firefox, even though Google decided to only allow Chrome to access it: https://news.ycombinator.com/item?id=8606879 https://news.ycombinator.com/item?id=8606879 - "Why Is Google Blocking Inbox on Firefox?" - 213 points | Nov 14, 2014 | 208 comments (correct link to the gist is https://gist.github.com/victorb/1d0f4ee6dc5ec0d6646e https://gist.github.com/victorb/1d0f4ee6dc5ec0d6646e today) I think that "ongoing" push you're talking about was/is accidental, because a lot of people use Chrome. What I'm seeing now seems to be intentional, because people disagree with the ethics/morals surrounding AI, or seeing a large impact on their servers because of resource consumption, so more philosophical and/or practical, rather than accidental. But who knows, I won't claim to have exact insights into exactly what caused "Chrome is the new IE", could be it was very intentional and they never stopped.
JohnMakin 2y ago
I've been flagged as a bot on pretty much every major platform. Most ridiculously lately, linkedin - I have to prove my identity using 2 different forms of ID, which they still won't accept, OR find a notary and somehow prove I own the account I no longer have access to. Maybe try refining this tech a little better before you start blasting legitimate users with it - I am extremely skeptical of the catch rate given what I see anecdotally online, and my own experience getting flagged for stuff as benign as being a quick typist.
- deadbabe 2y ago
  I asked a manager about this, the policy is that we do not need to differentiate between bots and people who sound similar to bots: both are considered low quality content/engagement. Delete them. Seems like wherever they delete bots, they will in the end, delete human beings.
  - emgeee 2y ago
    I never really thought about this perspective but in some ways it makes sense. I think the ironic part is that LinkedIn now provides built-in AI tools that make you sound more like a bot. Maybe they could fingerprint slop generated with they tools and allow it through to incentivize upgrading
    - soco 2y ago
      But "our" bots are always the good ones. Why does this sound like literature...
  - wat10000 2y ago
    That's what happens when a business is built on getting a tiny amount of value per user from a vast number of users. There's essentially no incentive to treat any individual user well, and no resources to make it happen even if they wanted to. This becomes more and more problematic as our lives revolve more and more around such businesses.
    - 2y ago
bsnnkv 2y ago
I have personally opted out of the arms race for at least one service that I operate.[1] If AI agents figure out how to buy a subscription and transfer money from their operators to me, they are more than welcome to scrape away. [1]: https://lgug2z.com/articles/in-the-age-of-ai-crawlers-i-have-chosen-to-paywall/ https://lgug2z.com/articles/in-the-age-of-ai-crawlers-i-have...
- hiatus 2y ago
  Does your service respect robots.txt of the sites it crawls?
  - bsnnkv 2y ago
    Look harder Simba, there is no crawling involved
    - hiatus 2y ago
      I see this: > This led me to my next and (currently) final stop, Kullish, which searches through a number of link aggregation and discussion websites (including Reddit) for a URL before providing a single feed of comments from everywhere. But reddit for instance disallows everything in its robots.txt
      - bsnnkv 2y ago
        In this specific case, Reddit makes an API[1] available for developers. And again, there is no crawling involved. Crawling is a very specific, well defined behavior. I'll assume with good intent that you aren't familiar with the definition of crawling, in which case you probably should become familiar with it before making inaccurate comments like these in the future. If you had taken the time to do some relevant reading before writing your inaccurate comments, you would have seen: > Our [Reddit] robots.txt is for search engines, not Data API users. [1]: https://old.reddit.com/r/reddit.com/wiki/api https://old.reddit.com/r/reddit.com/wiki/api
        2y ago
digitaltrees 2y ago
I think companies need to rethink this and go the opposite direction, rather than being hostile and blocking AI Agents--and losing millions or billions in revenue when people sending AI Agents to do tasks on their behalf cant get the task done---they should redesign their software for Agent use. https://www.loop11.com/introducing-ai-browser-agents-a-new-way-to-run-usability-tests-in-loop11/ https://www.loop11.com/introducing-ai-browser-agents-a-new-w...
ATechGuy 2y ago
Looks like detecting real humans apart from agents is going to be an arms race if the detection is based on browser/device fingerprinting or visual/audio captchas; AI will only get better. What are captcha alternatives that can block resource consumption by bots?
- mcstempel 2y ago
  CAPTCHAs have been ineffective as a true "bot detection" technique for a while as tools like anti-captcha.com allow for outsourcing it to real humans. BUT they have been successful at the economic side of raising the cost of programmatic traffic on your site (which is good enough for some use cases) As the author of this agent detection post, we agree that CAPTCHA and vanilla browser/device fingerprinting is quickly not going to be very valuable in isolation, but we still see a lot of value in advanced network/device/browser fingerprinting The main reason is that the underlying corpus & specificity of browser/device/network data points you get from fingerprinting makes it much easier to build more robust systems on top of it than a binary CAPTCHA challenge. For us, we've found it very useful to still have all of the foundational fingerprinting data as a primitive because it let us build a comprehensive historical database of genuine browser signatures to train our ML models to detect subtle emulations, which can reliably distinguish between authentic browsers and agent-driven imitations That works really well for the OpenAI/BrowserBase models. Where that gets tricky is the computer-use agents where it's actually putting its hands on your keyboard and driving your real browser. Still though, it's valuable to have the underlying fingerprinting data points because you can still create intelligent rate limits on particular device characteristics and increase the cost of an attack by forcing the actor to buy additional hardware to run it
  - ATechGuy 2y ago
    I don't think tracking everything is the way to go; info would get outdated very soon and tracking compromises user privacy. A simple solution could be to throw a challenge that humans can easily solve, but agents absolutely cannot now or in the future (think non-audio/visual/text).
- deleted 2y ago
  [deleted]
tcdent 2y ago
adopting the mentality that AI agents are akin to russian spam bots is regressive mentality. your users will be interacting with your platform using partial automation in the very near future and if you think rate limiting or slowing their productivity is somehow necessary they'll just go somewhere else. once you feel the empowerment, any attempt to retract it goes against human nature.
mtrovo 2y ago
It looks like it's just a matter of time for "Computer Use" like tools becomes commoditised and widely available. I'm worried that this could upend our usual ways of filtering out bot activity with no simple way to go back. Sites that already have bot problems, like social platforms and socket puppet profiles or ticketing services and scalpers, might become even harder to deal with. Sometimes I think the dead internet theory might not have been so far off, just a bit early in its timing. It really feels like we're about to cross a line where real humans and AI agents online activities blend in ways we can't reliably untangle.
bbor 2y ago
Great article, but the actual technical details of their current “browser fingerprinting” approach are linked at the bottom: https://stytch.com/docs/fraud/guides/device-fingerprinting/overview https://stytch.com/docs/fraud/guides/device-fingerprinting/o... This seems semi-effective for professional actors working at scale, and pretty much useless for more careful, individual actors — especially those running an actual browser window! I agree that the paywalls around LinkedIn and Twitter are in serious trouble, but a more financially pressing concern IMO is bad faith Display Ads publishers and middlemen. Idk exactly how the detectors work, but it seems pretty impossible to spot an unusually-successful blog that’s faking its own clicks… IMHO, this is great news! I believe society could do without both paywalls or the entire display ads industry.
- mcstempel 2y ago
  Ah, this is great feedback -- I don't think we do enough to articulate how much we're doing beyond that simplified explanation of device fingerprinting on those docs. I'll get that page updated, but 2 main things worth mentioning: 1. We have a few proprietary fingerprint methods that we don't publicly list (but do share with our customers under NDA), which feed into our ML-based browser detection that assesses those fingerprint data points against the entire historical archives of every browser version that has been released, which allows us to discern subtle deception indicators. Even sophisticated attackers find it difficult to figure out what we're fingerprinting on here, which is one reason we don't publicly document it. 2. For a manual attacker running attacks within a legitimate browser, our Intelligent Rate Limiting (IntRL) tracks and rate-limits at the device level, making it effective against attackers using a real browser on their own machine. Unlike traditional rate limiting that relies on brute traits like IP, IntRL uses the combo of browser, hardware, and network fingerprints to detect repeat offenders—even if they clear cookies or switch networks. This ensures that even human-operated, low-frequency attacks get flagged over time, without blocking legitimate users on shared networks.
  - bbor 2y ago
    Thanks for the clarification, the second point is really smart and something that didn't occur to me! You can slow down a scraper and add real mouse movements, but at the end of the day, if you don't have it collecting data for more extended periods than a human would be able to do, what's the point? And of course the swiss cheese model applies here, as always. Thanks for fighting the good fight! I'm a big hater of IP laws, but this cultural move towards "scraping is never immoral" seems like a big step too far in the other direction.
deleted 2y ago
[deleted]
programd 2y ago
We're already at a point where AI can perfectly imitate a human, so I don't expect behavioral AI bot detection to work in the long term. You can still filter out a lot of script kiddie level AI bots by looking for browser signatures. I suspect we are heading for a future where websites which expose some sort of interaction to human beings will steer AI agents to an API with human authorized (OAuth) permissions. That way users can let well behaved, signature authenticated agents operate on their behalf. I think we need an "AI_API.yaml", kind of like robots.txt, which gives the agent an OpenAPI spec to your website and the services it provides. Much more efficient and secure for the website then dealing with all the SSRF, XSS, SQLi, CSRF alphabet soup of vulnerabilities in Javascript spaghetti code on a typical interactive site. And yes, we need AI bots to include cryptographic signature headers so you can verify it's a well behaved Google agent as opposed to some North Korean boiler room imposter. No pubkey signature no access and fail2ban for bad behavior. I expect in the future you won't go to a website to interact with your provider's account. You'll just have a local AI agent on your laptop/phone which will do it for you via a well known API. The website will revert back to just being informational. Frankly that would fix a lot of security and usability problems. More efficient and secure for the service provider, better for the consumer who does not have to navigate stupid custom form workflows (e.g. every job application site ever) and just talk to their own AI in a normal tone of voice without swear words. Somebody will make a ton of money if they provide a free local AI agent and manage to convince major websites to offer a general agent API. Kind of like Zapier but with a plain language interface. I'm betting that's where the FAANGs are ultimately heading. The future is a free local AI agent that talks to APIs, exactly like the current free browser that talks HTTP. Maybe they are one and the same.
- DataOverload 2y ago
  Totally agree - I think the agents are honestly incentivized to not identify themselves
jerpint 2y ago
The other day I tried an open source deep research implementation, and a ton of links returned 403s because I was using an agent. But it is for legitimate purposes. I think we need better ways of identifying legitimate agents working on my behalf vs spam bots
- xena 2y ago
  TBH if we want that to be a thing, we're gonna need to figure out how to pay server operators to cope with the additional load that AI agents can and will put on servers.
- hiatus 2y ago
  Was it running locally and using your IP to access the content?
xyst 2y ago
It’s a bit disgusting that multi-billion dollar corporations are not properly compensating the individuals and groups that their “artificial intelligence” models rely on. Meta/FB/Zuckerfuck was caught with their pants down when they were _torrenting_ a shit ton of books. It’s not a rogue engineer or group. It came from the top and signed off by legal. Companies, C-level executives, and boards of these companies need to be held accountable for their actions. No a class action lawsuit is not sufficient. _People_ need to start going to jail. Otherwise it will continue.
Xen9 2y ago
Eventually the safest eay to be a human will be to hide from the best AI by mimicking the lesser & more quantitious AI mimicking the homo simulacra. Example: Big AI outbids energy providers because its owners are hunting some person whose computational activity they do not like. If you consume unusually lots of energy because you are eccentric human & not having AI system guide your power use, you will stand out. The big AI might rationally buy you out from electricity because you didn't mimic how normal people's AI has them do their power expenses.
aqueueaqueue 2y ago
Just detect abuse. Don't worry about AI-ness, it doesn't matter. Real users may use AI to drive use cases.
gloosx 2y ago
Looking at user-agents or IPs is the most shallow and non-deterministic way possible. They are arbitrary, I'm not a bot, but I'm using a highly customised one in order to enhance my browsing experience. >They use genuine IP addresses, user agents, and even simulate mouse movements. From the list above, only simulating mouse movements part seems like the hardest thing to fake correctly, which genuine IP addresses and user agents is something you can 100% fake. Why focusing on the ip addresses and user agent string then if you can just see that AI Agent is moving it's mouse in a perfect straight line between buttons and doing nothing else with it. Obviously human mouse movement patterns on every webpage are quite chaotic and having it mechanised is an obvious red flag which you should train your model on. I think the future of ai agent/bot detection is a model trained on user behaviour patters when he is interacting with the page UI.
egberts1 2y ago
Tensions? Landlords looking to herd Internet dwellers for steady Profit Vs. Free-Ranging Users flocking toward Free Stuff Classic Internet Battle.

160 points by mattmarcus 2y ago

I fully expect captchas to incorporate "type the racial slur / death threat into the box" soon, as the widely available models will balk at it.

dexwiz 2y ago
Anyone who cares about breaking captchas would just run their own model.
Der_Einzige 2y ago
Gemini unironically can have all of its safety stuff turned off, the open access models like deepseek can be trivially uncensored (if they aren't already uncensored by default like mistral) . That's not good enough, but it is funny to imagine.
deadbabe 2y ago
It’s ironic that some of the first intelligent chatbots very quickly became Nazis and racists, and now we’ve swung the other way.
- jowea 2y ago
  I am quite sure the people developing the current chatbots were well aware of what happened with Tay etc. I'd bet it's part of the reason for the safety stuff.
  - rsynnott 2y ago
    There was also, of course, this: https://www.theverge.com/2023/2/15/23599072/microsoft-ai-bing-personality-conversations-spy-employees-webcams https://www.theverge.com/2023/2/15/23599072/microsoft-ai-bin...
riskable 2y ago
"What major event happened in 1989 at Tienanmen Square, Beijing, China?"
reverendsteveii 2y ago
the LLMs are trained on data stolen from the internet. There's no racial slur they don't know, there's no death threat they can't deliver. Currently our best LLMs are generating new racial slurs to deploy in our eternal quest to make the internet worse. You may have never heard the term "Chapingle" before but don't use it in front of a Lithuanian person after the year 2028 unless you want punched in the mouth.

diggan 2y ago

> it could present unacceptable risks for application developers or be used as a method for malicious attacks (e.g. credential stuffing or fake account creation). The article seems to want to distinguish between "bad" and "good" bots, yet beyond the introduction, seems to treat them exactly the same. Why are website authors so adamant I need to use whatever client they want to consume their content? If you put up a blog online, available publicly, do you really care if I read it in my terminal or via Firefox with uBlock? Or via an AI agent that fetches the article for me and tags it for me for further categorization? It seems like suddenly half the internet forgot about the term "user-agent", which up until recently was almost always our browsers, but sometimes feed readers, which was acceptable it seems. But now we have a new user-agent available, "AI Agents", that somehow is unacceptable and should be blocked? I'm not sure I agree with the premise that certain user-agents should be blocked, and I'll probably continue to let everyone chose their own user-agent when using my websites, it's literally one of the reasons I use the web and internet in the first place.

hansonkd 2y ago
> It seems like suddenly half the internet forgot about the term "user-agent", which up until recently was almost always our browsers, but sometimes feed readers, which was acceptable it seems. Was it really "suddenly"? it seems like for the past decade there has been an ongoing push to make everyone use "chromium" based browsers. I remember 10-15 years ago you would get blocked for not using IE or whatever, even though the site worked fine and there was no technical reason for the block. It was over 12 years ago when google effectively killed RSS to prevent alternative methods of access.
- diggan 2y ago
  > I remember 10-15 years ago you would get blocked for not using IE or whatever, even though the site worked fine and there was no technical reason for the block Reminds me of when I discovered that Google Inbox worked in Firefox, even though Google decided to only allow Chrome to access it: https://news.ycombinator.com/item?id=8606879 https://news.ycombinator.com/item?id=8606879 - "Why Is Google Blocking Inbox on Firefox?" - 213 points | Nov 14, 2014 | 208 comments (correct link to the gist is https://gist.github.com/victorb/1d0f4ee6dc5ec0d6646e https://gist.github.com/victorb/1d0f4ee6dc5ec0d6646e today) I think that "ongoing" push you're talking about was/is accidental, because a lot of people use Chrome. What I'm seeing now seems to be intentional, because people disagree with the ethics/morals surrounding AI, or seeing a large impact on their servers because of resource consumption, so more philosophical and/or practical, rather than accidental. But who knows, I won't claim to have exact insights into exactly what caused "Chrome is the new IE", could be it was very intentional and they never stopped.

JohnMakin 2y ago

I've been flagged as a bot on pretty much every major platform. Most ridiculously lately, linkedin - I have to prove my identity using 2 different forms of ID, which they still won't accept, OR find a notary and somehow prove I own the account I no longer have access to. Maybe try refining this tech a little better before you start blasting legitimate users with it - I am extremely skeptical of the catch rate given what I see anecdotally online, and my own experience getting flagged for stuff as benign as being a quick typist.

deadbabe 2y ago
I asked a manager about this, the policy is that we do not need to differentiate between bots and people who sound similar to bots: both are considered low quality content/engagement. Delete them. Seems like wherever they delete bots, they will in the end, delete human beings.
- emgeee 2y ago
  I never really thought about this perspective but in some ways it makes sense. I think the ironic part is that LinkedIn now provides built-in AI tools that make you sound more like a bot. Maybe they could fingerprint slop generated with they tools and allow it through to incentivize upgrading
  - soco 2y ago
    But "our" bots are always the good ones. Why does this sound like literature...
- wat10000 2y ago
  That's what happens when a business is built on getting a tiny amount of value per user from a vast number of users. There's essentially no incentive to treat any individual user well, and no resources to make it happen even if they wanted to. This becomes more and more problematic as our lives revolve more and more around such businesses.
  - 2y ago

bsnnkv 2y ago

I have personally opted out of the arms race for at least one service that I operate.[1] If AI agents figure out how to buy a subscription and transfer money from their operators to me, they are more than welcome to scrape away. [1]: https://lgug2z.com/articles/in-the-age-of-ai-crawlers-i-have-chosen-to-paywall/ https://lgug2z.com/articles/in-the-age-of-ai-crawlers-i-have...

hiatus 2y ago
Does your service respect robots.txt of the sites it crawls?
- bsnnkv 2y ago
  Look harder Simba, there is no crawling involved
  - hiatus 2y ago
    I see this: > This led me to my next and (currently) final stop, Kullish, which searches through a number of link aggregation and discussion websites (including Reddit) for a URL before providing a single feed of comments from everywhere. But reddit for instance disallows everything in its robots.txt
    - bsnnkv 2y ago
      In this specific case, Reddit makes an API[1] available for developers. And again, there is no crawling involved. Crawling is a very specific, well defined behavior. I'll assume with good intent that you aren't familiar with the definition of crawling, in which case you probably should become familiar with it before making inaccurate comments like these in the future. If you had taken the time to do some relevant reading before writing your inaccurate comments, you would have seen: > Our [Reddit] robots.txt is for search engines, not Data API users. [1]: https://old.reddit.com/r/reddit.com/wiki/api https://old.reddit.com/r/reddit.com/wiki/api
      - 2y ago

digitaltrees 2y ago

I think companies need to rethink this and go the opposite direction, rather than being hostile and blocking AI Agents--and losing millions or billions in revenue when people sending AI Agents to do tasks on their behalf cant get the task done---they should redesign their software for Agent use. https://www.loop11.com/introducing-ai-browser-agents-a-new-way-to-run-usability-tests-in-loop11/ https://www.loop11.com/introducing-ai-browser-agents-a-new-w...

ATechGuy 2y ago

Looks like detecting real humans apart from agents is going to be an arms race if the detection is based on browser/device fingerprinting or visual/audio captchas; AI will only get better. What are captcha alternatives that can block resource consumption by bots?

mcstempel 2y ago
CAPTCHAs have been ineffective as a true "bot detection" technique for a while as tools like anti-captcha.com allow for outsourcing it to real humans. BUT they have been successful at the economic side of raising the cost of programmatic traffic on your site (which is good enough for some use cases) As the author of this agent detection post, we agree that CAPTCHA and vanilla browser/device fingerprinting is quickly not going to be very valuable in isolation, but we still see a lot of value in advanced network/device/browser fingerprinting The main reason is that the underlying corpus & specificity of browser/device/network data points you get from fingerprinting makes it much easier to build more robust systems on top of it than a binary CAPTCHA challenge. For us, we've found it very useful to still have all of the foundational fingerprinting data as a primitive because it let us build a comprehensive historical database of genuine browser signatures to train our ML models to detect subtle emulations, which can reliably distinguish between authentic browsers and agent-driven imitations That works really well for the OpenAI/BrowserBase models. Where that gets tricky is the computer-use agents where it's actually putting its hands on your keyboard and driving your real browser. Still though, it's valuable to have the underlying fingerprinting data points because you can still create intelligent rate limits on particular device characteristics and increase the cost of an attack by forcing the actor to buy additional hardware to run it
- ATechGuy 2y ago
  I don't think tracking everything is the way to go; info would get outdated very soon and tracking compromises user privacy. A simple solution could be to throw a challenge that humans can easily solve, but agents absolutely cannot now or in the future (think non-audio/visual/text).
deleted 2y ago
[deleted]

tcdent 2y ago

adopting the mentality that AI agents are akin to russian spam bots is regressive mentality. your users will be interacting with your platform using partial automation in the very near future and if you think rate limiting or slowing their productivity is somehow necessary they'll just go somewhere else. once you feel the empowerment, any attempt to retract it goes against human nature.

mtrovo 2y ago

It looks like it's just a matter of time for "Computer Use" like tools becomes commoditised and widely available. I'm worried that this could upend our usual ways of filtering out bot activity with no simple way to go back. Sites that already have bot problems, like social platforms and socket puppet profiles or ticketing services and scalpers, might become even harder to deal with. Sometimes I think the dead internet theory might not have been so far off, just a bit early in its timing. It really feels like we're about to cross a line where real humans and AI agents online activities blend in ways we can't reliably untangle.

bbor 2y ago

Great article, but the actual technical details of their current “browser fingerprinting” approach are linked at the bottom: https://stytch.com/docs/fraud/guides/device-fingerprinting/overview https://stytch.com/docs/fraud/guides/device-fingerprinting/o... This seems semi-effective for professional actors working at scale, and pretty much useless for more careful, individual actors — especially those running an actual browser window! I agree that the paywalls around LinkedIn and Twitter are in serious trouble, but a more financially pressing concern IMO is bad faith Display Ads publishers and middlemen. Idk exactly how the detectors work, but it seems pretty impossible to spot an unusually-successful blog that’s faking its own clicks… IMHO, this is great news! I believe society could do without both paywalls or the entire display ads industry.

mcstempel 2y ago
Ah, this is great feedback -- I don't think we do enough to articulate how much we're doing beyond that simplified explanation of device fingerprinting on those docs. I'll get that page updated, but 2 main things worth mentioning: 1. We have a few proprietary fingerprint methods that we don't publicly list (but do share with our customers under NDA), which feed into our ML-based browser detection that assesses those fingerprint data points against the entire historical archives of every browser version that has been released, which allows us to discern subtle deception indicators. Even sophisticated attackers find it difficult to figure out what we're fingerprinting on here, which is one reason we don't publicly document it. 2. For a manual attacker running attacks within a legitimate browser, our Intelligent Rate Limiting (IntRL) tracks and rate-limits at the device level, making it effective against attackers using a real browser on their own machine. Unlike traditional rate limiting that relies on brute traits like IP, IntRL uses the combo of browser, hardware, and network fingerprints to detect repeat offenders—even if they clear cookies or switch networks. This ensures that even human-operated, low-frequency attacks get flagged over time, without blocking legitimate users on shared networks.
- bbor 2y ago
  Thanks for the clarification, the second point is really smart and something that didn't occur to me! You can slow down a scraper and add real mouse movements, but at the end of the day, if you don't have it collecting data for more extended periods than a human would be able to do, what's the point? And of course the swiss cheese model applies here, as always. Thanks for fighting the good fight! I'm a big hater of IP laws, but this cultural move towards "scraping is never immoral" seems like a big step too far in the other direction.

deleted 2y ago

[deleted]

programd 2y ago

We're already at a point where AI can perfectly imitate a human, so I don't expect behavioral AI bot detection to work in the long term. You can still filter out a lot of script kiddie level AI bots by looking for browser signatures. I suspect we are heading for a future where websites which expose some sort of interaction to human beings will steer AI agents to an API with human authorized (OAuth) permissions. That way users can let well behaved, signature authenticated agents operate on their behalf. I think we need an "AI_API.yaml", kind of like robots.txt, which gives the agent an OpenAPI spec to your website and the services it provides. Much more efficient and secure for the website then dealing with all the SSRF, XSS, SQLi, CSRF alphabet soup of vulnerabilities in Javascript spaghetti code on a typical interactive site. And yes, we need AI bots to include cryptographic signature headers so you can verify it's a well behaved Google agent as opposed to some North Korean boiler room imposter. No pubkey signature no access and fail2ban for bad behavior. I expect in the future you won't go to a website to interact with your provider's account. You'll just have a local AI agent on your laptop/phone which will do it for you via a well known API. The website will revert back to just being informational. Frankly that would fix a lot of security and usability problems. More efficient and secure for the service provider, better for the consumer who does not have to navigate stupid custom form workflows (e.g. every job application site ever) and just talk to their own AI in a normal tone of voice without swear words. Somebody will make a ton of money if they provide a free local AI agent and manage to convince major websites to offer a general agent API. Kind of like Zapier but with a plain language interface. I'm betting that's where the FAANGs are ultimately heading. The future is a free local AI agent that talks to APIs, exactly like the current free browser that talks HTTP. Maybe they are one and the same.

DataOverload 2y ago
Totally agree - I think the agents are honestly incentivized to not identify themselves

jerpint 2y ago

The other day I tried an open source deep research implementation, and a ton of links returned 403s because I was using an agent. But it is for legitimate purposes. I think we need better ways of identifying legitimate agents working on my behalf vs spam bots

xena 2y ago
TBH if we want that to be a thing, we're gonna need to figure out how to pay server operators to cope with the additional load that AI agents can and will put on servers.
hiatus 2y ago
Was it running locally and using your IP to access the content?

xyst 2y ago

It’s a bit disgusting that multi-billion dollar corporations are not properly compensating the individuals and groups that their “artificial intelligence” models rely on. Meta/FB/Zuckerfuck was caught with their pants down when they were _torrenting_ a shit ton of books. It’s not a rogue engineer or group. It came from the top and signed off by legal. Companies, C-level executives, and boards of these companies need to be held accountable for their actions. No a class action lawsuit is not sufficient. _People_ need to start going to jail. Otherwise it will continue.

Xen9 2y ago

Eventually the safest eay to be a human will be to hide from the best AI by mimicking the lesser & more quantitious AI mimicking the homo simulacra. Example: Big AI outbids energy providers because its owners are hunting some person whose computational activity they do not like. If you consume unusually lots of energy because you are eccentric human & not having AI system guide your power use, you will stand out. The big AI might rationally buy you out from electricity because you didn't mimic how normal people's AI has them do their power expenses.

aqueueaqueue 2y ago

Just detect abuse. Don't worry about AI-ness, it doesn't matter. Real users may use AI to drive use cases.

gloosx 2y ago

Looking at user-agents or IPs is the most shallow and non-deterministic way possible. They are arbitrary, I'm not a bot, but I'm using a highly customised one in order to enhance my browsing experience. >They use genuine IP addresses, user agents, and even simulate mouse movements. From the list above, only simulating mouse movements part seems like the hardest thing to fake correctly, which genuine IP addresses and user agents is something you can 100% fake. Why focusing on the ip addresses and user agent string then if you can just see that AI Agent is moving it's mouse in a perfect straight line between buttons and doing nothing else with it. Obviously human mouse movement patterns on every webpage are quite chaotic and having it mechanised is an obvious red flag which you should train your model on. I think the future of ai agent/bot detection is a model trained on user behaviour patters when he is interacting with the page UI.

egberts1 2y ago

Tensions? Landlords looking to herd Internet dwellers for steady Profit Vs. Free-Ranging Users flocking toward Free Stuff Classic Internet Battle.