3 ms·

> I wish someone made a similar command line tool that does screen scraping, is powered by AI and could simulate clicks as a real user. How much are you, perso

> I wish someone made a similar command line tool that does screen scraping, is powered by AI and could simulate clicks as a real user. How much are you, personally, willing to pay for such a tool? Because, minus the AI[1], I'm pretty certain that such tools exists at different price points. I'm not sure what the AI would do for such a tool, but I'm sure that the companies making such a thing are busy working hard to incorporate some form of AI into it. [1] What would the AI in such a tool do?

nextaccountic 3y ago
> [1] What would the AI in such a tool do? Solving captchas. In particular, not only solving the challenge itself (recognizing crosswalks, etc) but also handling input (moving the mouse, alt+tabbing, etc) as if it were a real human. I was recently reading the Cloudflare turnstile announcement https://blog.cloudflare.com/turnstile-ga https://blog.cloudflare.com/turnstile-ga and it has the following links about this https://uncaptcha.cs.umd.edu/papers/uncaptcha_woot17.pdf https://uncaptcha.cs.umd.edu/papers/uncaptcha_woot17.pdf https://www.vox.com/22436832/captchas-getting-harder-ai-artificial-intelligence https://www.vox.com/22436832/captchas-getting-harder-ai-arti... https://arxiv.org/abs/1903.01003 https://arxiv.org/abs/1903.01003 In those excerpts: > The prevalence of free speech-to-text services has made it easy for bots to solve audio CAPTCHAs as well, with a recent study showing bots can accurately solve audio CAPTCHAs in over 85% of attempts. We’re proud to state that Turnstile is WCAG 2.1 Level AA compliant, while eliminating the need for audio CAPTCHAs as well as visual ones. And > With all of our emphasis on how easy it is to pass a Turnstile challenge, you would be right to ask how it can stop a bot. If a bot can find all images with crosswalks in grainy photos faster than we can, surely it can check a box as well. Bots definitely can check a box, and they can even mimic the erratic path of human mouse movement while doing so. For Turnstile, the actual act of checking a box isn’t important, it’s the background data we’re analyzing while the box is checked that matters. We find and stop bots by running a series of in-browser tests, checking browser characteristics, native browser APIs, and asking the browser to pass lightweight tests (ex: proof-of-work tests, proof-of-space tests) to prove that it’s an actual browser. The current deployment of Turnstile checks billions of visitors every day, and we are able to identify browser abnormalities that bots exhibit while attempting to pass those tests. So, this is a cat and mouse game, the cat is using AI, the mouse is probably using AI in some form or another too.
- eddd-ddde 3y ago
  We really are gonna end up with servers and clients evolving AI challenges and solvers respectively to the point where humans won't be able to solve them anymore haha
jakozaur 3y ago
The brainer price would be $20 / month. $50 / month if it was terrific. The trick has to be really useful without too much work on my end. Combine screen scraping and simulating real users with some mix of AI. My use cases: 1. Fill any form with intelligent data (from orders, through plane tickets to tax forms). Think of ChatGPT but for forms. 2. Research any person and company using various tools (e.g. LinkedIn, Google, job boards). Do it at human speed so I won't get blocked and can observe it on a screen. I can provide a script on how to do it (e.g. check LinkedIn, this board for that, etc.) Provide me with a summary of the text. 3. I want to be able to script it and hack it to whatever I need. API and CLI.

nextaccountic 3y ago

> [1] What would the AI in such a tool do? Solving captchas. In particular, not only solving the challenge itself (recognizing crosswalks, etc) but also handling input (moving the mouse, alt+tabbing, etc) as if it were a real human. I was recently reading the Cloudflare turnstile announcement https://blog.cloudflare.com/turnstile-ga https://blog.cloudflare.com/turnstile-ga and it has the following links about this https://uncaptcha.cs.umd.edu/papers/uncaptcha_woot17.pdf https://uncaptcha.cs.umd.edu/papers/uncaptcha_woot17.pdf https://www.vox.com/22436832/captchas-getting-harder-ai-artificial-intelligence https://www.vox.com/22436832/captchas-getting-harder-ai-arti... https://arxiv.org/abs/1903.01003 https://arxiv.org/abs/1903.01003 In those excerpts: > The prevalence of free speech-to-text services has made it easy for bots to solve audio CAPTCHAs as well, with a recent study showing bots can accurately solve audio CAPTCHAs in over 85% of attempts. We’re proud to state that Turnstile is WCAG 2.1 Level AA compliant, while eliminating the need for audio CAPTCHAs as well as visual ones. And > With all of our emphasis on how easy it is to pass a Turnstile challenge, you would be right to ask how it can stop a bot. If a bot can find all images with crosswalks in grainy photos faster than we can, surely it can check a box as well. Bots definitely can check a box, and they can even mimic the erratic path of human mouse movement while doing so. For Turnstile, the actual act of checking a box isn’t important, it’s the background data we’re analyzing while the box is checked that matters. We find and stop bots by running a series of in-browser tests, checking browser characteristics, native browser APIs, and asking the browser to pass lightweight tests (ex: proof-of-work tests, proof-of-space tests) to prove that it’s an actual browser. The current deployment of Turnstile checks billions of visitors every day, and we are able to identify browser abnormalities that bots exhibit while attempting to pass those tests. So, this is a cat and mouse game, the cat is using AI, the mouse is probably using AI in some form or another too.

eddd-ddde 3y ago
We really are gonna end up with servers and clients evolving AI challenges and solvers respectively to the point where humans won't be able to solve them anymore haha

jakozaur 3y ago

The brainer price would be $20 / month. $50 / month if it was terrific. The trick has to be really useful without too much work on my end. Combine screen scraping and simulating real users with some mix of AI. My use cases: 1. Fill any form with intelligent data (from orders, through plane tickets to tax forms). Think of ChatGPT but for forms. 2. Research any person and company using various tools (e.g. LinkedIn, Google, job boards). Do it at human speed so I won't get blocked and can observe it on a screen. I can provide a script on how to do it (e.g. check LinkedIn, this board for that, etc.) Provide me with a summary of the text. 3. I want to be able to script it and hack it to whatever I need. API and CLI.