5 ms·
Show HN: An experimental AntiBot, AntiCrawl reverse proxy for the web
- ronsor 2y agoWell, this is going to be extremely resource intensive (read: your site will be a DDOS magnet), and it's also an accessibility nightmare.
- ranger_danger 2y agoCurious if you have any better suggestions
- ronsor 2y agoDon't serve your website as PNGs wrapped in a single page application. Also, if the main reason for blocking bots is to reduce server load, this solution is going to require running multiple browser instances on a server, which will require a lot of resources just to serve normal traffic. Edit: I should also mention this is going to chew through bandwidth.
- throwaway888abc 2y agoAn option can be render each div/element as separate image wrapped in aria markup
- ranger_danger 2y agoWouldn't the aria markup contain the very text they don't want bots to have?
- 1231232131231 2y agoLiterally anything other than converting websites to images. For example: captchas, proof-of-work captchas, bot detection using techniques such as tls fingerprinting, blacklisting known vps/vpn IPs, etc.
- sanchezxh 2y agoYea your right.
- _1tem 2y agoI’ve actually seriously been thinking of using WebAuthn to “authenticate” every single page load with a passkey unlocked by a biometric device only, so that I can be sure that every single page load had a meat finger on TouchID or a meat face in front of FaceID before showing the page to them. In the future I imagine that there will be biometrically secure browsers that will be required for top security applications, that can guarantee that a single physical person is actually physically present while using it.
- hamandcheese 2y agoCouldn't someone just write a software key that lies and says it's biometric? AFAIK end-to-end manufacturer attestation isn't part of the spec (yet...).
- KomoD 2y agoThere's even an emulator built in to Chrome Dev Tools -> 3 dots -> More tools -> WebAuthn -> Enable virtual authenticator environment
- _1tem 2y agoAttestation statements can be used to restrict WebAuthn use to certain hardware keys only.
- hamandcheese 2y agoBut is there a chain of trust going all the way back to the manufacturer? Or is that simply metadata which could be spoofed?
- gruez 2y agoYes. That's what attestation is for.
- dyml 2y agoPlease don’t use WebAuthn on every page load. Two reasons: the protocol is not designed to do this - and the UI/UX is not designed to support this. There are better ways. 2) it will likely not work. There are virtual/software authenticatators (available in dev tools) that could generate a valid response without a human.
- tonyponydong 2y agoPlease don’t make the web more inaccessible to assistive technology users
- ranger_danger 2y agoYou're not wrong, but tell that to the bots :) Do you have any other suggestions?
- gruez 2y ago>You're not wrong, but tell that to the bots :) Tell that to the judge when you're slapped with an ADA lawsuit because you failed to provide "reasonable accommodation" to disabled users.
- ranger_danger 2y agoI have to say, I've been making websites professionally for 25 years, and I've never heard of this being a thing even once.
- gruez 2y agoYou must not be working for juicy clients then. https://www.levelaccess.com/blog/title-iii-lawsuits-10-big-companies-sued-over-website-accessibility/ https://www.levelaccess.com/blog/title-iii-lawsuits-10-big-c...
- ranger_danger 2y agoAlways companies with less than 15 employees so I think ADA doesn't apply.
- freen 2y agoWell, I’m not going to use this for it’s intended purpose, however I AM going to try to see if I can use it to pipe webpages to a simple touchscreen e-ink display and retain interactivity.
- rmbyrro 2y agoI presume you're not interested in traffic coming from search engines, right?
- jeroenhd 2y agoSearch engines have publicly listed IP ranges + user agents, setting up a HTML whitelist for search engines you do like shouldn't be impossible. Especially now that Google no longer presents its cache to visitors.
- drdaeman 2y agoDoing the ad industry's work? :-) I strongly suspect someone from ad industry will start offering an option to serve a heavily obfuscated WASM-based rendering engine to render the website, with obligatory promises that it "protects the integrity of your content", "stops the AI crawl theft", and of course it also "lowers development costs by ensuring consistent rendering across all platforms". /s, obviously
- satvikpendem 2y agoIsn't that just Flutter (or other such WASM GUIs like Slint in Rust)? The issue is no SEO so until that's fixed, I see no way it would work. And well, if it does have SEO, then any bot can scrape it anyway.
- rmbyrro 2y agoCurious that the author works for BrowserStack, a browser-automation service
- AnnaMere 2y ago> Render Webpage as PNG for static content Why not just take a screenshot yourself and use an image tag to render the image without using puppeteers? You can add a small CSS media query for showing different sizes of images for different types of devices. Why do you even need puppeteer for above basic things?