4 ms·
Running websites with CSP is an interesting experience. You'll discover how much crap is injected into people's website view. Extensions referencing remote scri
by viraptor 6y ago
Running websites with CSP is an interesting experience. You'll discover how much crap is injected into people's website view. Extensions referencing remote scripts, weird browsers injecting their content (looking at your ucweb), website being framed for both bad (clickjack overlay) and good (translators) purposes, bots injecting their own helper JS, bots that don't disable their internal CSP reporting (looking at you cookiebot), AVs which try to patch CSP header and break it (Kaspersky).
Essentially you will find there's a whole wild west out there that you don't normally think about.
- dmix 6y agoThis sites example code seems to be taking the blacklist approach instead of whitelist, so I shouldn't break too many sites. Nor give the surprise treats you mentioned. But I agree, locking down a site with CSP is a great exercise in just how domains get loaded. Especially if you have a marketing team who gets sold on the latest service weekly.
- hn_throwaway_99 6y agoI think you may be misunderstanding parent's post. When you run a site with a strict CSP and a reportUri enabled, you will notice that you get tons of blocked scripts reported not because there is a vulnerability in your site, but because clients have installed tons of browser plugins that try to inject scripts into your site, and your CSP will block them.
- ev1 6y agoI figured this out the first time I tried instrumenting my code for latency/exception handling/etc purposes (self hosted!) .. and got terabytes of error logs consisting of terribly written shitty browser plugins or malware toolbars spraying rubbish into the console.
- chrisweekly 6y agoYeah; care to take a guess as to what % of HTTP traffic is human vs bot? (Given your comment, I suspect your guess is likely to be much closer to reality than the vast majority of even the relatively clueful people here on HN.)
- viraptor 6y agoThat will depend massively on your site structure. (With enough pages Bingbot will do a magnitude more requests than other crawlers) Overall "lower % than your tooling tells you" - for example cookiebot runs crawlers with both proper user agents and "totally a normal Chrome user" from another network. (Not sure if misconfigured or on purpose)
- Semaphor 6y agoWhen doing some stuff with automated Google Admanager Ads (that required frequent updates), we realized that google is checking every link. At least once per update. Usually more often. With no proper user agent identifying it as a bot. Our solution was to redirect all links through our server and completely block every known google IP-range during the redirect.
- chrisweekly 6y agoI meant net-wide, not for their particular endpoints.