7 ms·

> this domain was caught injecting malware on mobile devices via any site that embeds cdn.polyfill.io I've said it before, and I'll say it again: https://httpt

> this domain was caught injecting malware on mobile devices via any site that embeds cdn.polyfill.io I've said it before, and I'll say it again: https://httptoolkit.com/blog/public-cdn-risks/ https://httptoolkit.com/blog/public-cdn-risks/ You can reduce issues like this using subresource intergrity (SRI) but there are still tradeoffs (around privacy & reliability - see article above) and there is a better solution: self-host your dependencies behind a CDN service you control (just bunny/cloudflare/akamai/whatever is fine and cheap). In a tiny prototyping project, a public CDN is convenient to get started fast, sure, but if you're deploying major websites I would really strong recommend not using public CDNs, never ever ever ever (the World Economic Forum website is affected here, for example! Absolutely ridiculous).

irrational 2y ago
> self-host your dependencies I can kind of understand why people went away from this, but this is how we did it for years/decades and it just worked. Yes, doing this does require more work for you, but that's just part of the job.
- jweir 2y ago
  Own your process – at best that CDN is spying on your users.
- marcosdumay 2y ago
  > and it just worked Just to add... that is unlike the CDN thing, that will send developers into Stack Overflow looking how to set-up CORS.
- toddmorey 2y ago
  For performance reasons alone, you definitely want to host as much as possible on the same domain. In my experience from inside companies, we went from self-hosting with largely ssh access to complex deployment automation and CI/CD that made it hard to include any new resource in the build process. I get the temptation: resources linked from external domains / cdns gave the frontend teams quick access to the libraries, fonts, tools, etc. they needed. Thankfully things have changed for the better and it's much easier to include these things directly inside your project.
  - jameshart 2y ago
    There was a brief period when the frontend dev world believed the most performant way to have everyone load, say, jquery, would be for every site to load it from the same CDN URL. From a trustworthy provider like Google, of course. It turned out the browser domain sandboxing wasn’t as good as we thought, so this opened up side channel attacks, which led to browsers getting rid of cross-domain cache sharing; and of course it turns out that there’s really no such thing as a ‘trustworthy provider’ so the web dev community memory-holed that little side adventure and pivoted to npm. Which is going GREAT by the way. The advice is still out there, of course. W3schools says: > One big advantage of using the hosted jQuery from Google: > Many users already have downloaded jQuery from Google when visiting another site. As a result, it will be loaded from cache when they visit your site https://www.w3schools.com/jquery/jquery_get_started.asp https://www.w3schools.com/jquery/jquery_get_started.asp Which hasn’t been true for years, but hey.
    - josephg 2y ago
      The only thing I’d trust w3schools to teach me is SEO. How do they stay on top of Google search results with such bad, out of date articles?
      - immibis 2y ago
        Be good at a time when Google manually ranks domains, then pivot to crap when Google stops updating the ranking. Same as the site formerly known as Wikia.
  - matsemann 2y ago
    > For performance reasons alone, you definitely want to host as much as possible on the same domain. It used to be the opposite. Browsers limit the amount of concurrent requests to a domain. A way to circumvent that was to load your resources from a.example.com, b.example.com, c.example.com etc. Paying some time for extra dns resolves I guess, but could then load many more resources at the same time. Not as relevant anymore, with http2 that allows sharing connections, and more common to bundle files.
ziml77 2y ago
I've seen people reference CDNs for internal sites. I hate that because it is not only a security risk but it also means we depend on the CDN being reachable for the internal site to work. It's especially annoying because the projects I've seen it on were using NPM anyway so they could have easily pulled the dependency in through there. Hell, even without NPM it's not hard to serve these JS libraries internally since they tend to get packed into one file (+ maybe a CSS file).
simonw 2y ago
I always prefer to self-host my dependencies, but as a developer who prefer to avoid an npm-based webpack/whatever build pipeline it's often WAY harder to do that than I'd like. If you are the developer of an open source JavaScript library, please take the time to offer a downloadable version of it that works without needing to run an "npm install" and then fish the right pieces out of the node_modules folder. jQuery still offer a single minified file that I can download and use. I wish other interesting libraries would do the same! (I actually want to use ES Modules these days which makes things harder due to the way they load dependencies. I'm still trying to figure out the best way to use import maps to solve this.)
- spankalee 2y ago
  As you might know, Lit offers a single bundled file for the core library.
  - simonw 2y ago
    Yes! Love that about Lit. The problem is when I want to add other things that have their own dependency graph.
    - spankalee 2y ago
      This is why I don't think it's very workable to avoid npm. It's the package manager of the ecosystem, and performs the job of downloading dependencies well. I personally never want to go back to the pre-package-manager days for any language.
      - PaulHoule 2y ago
        One argument is that Javascript-in-the-browser has advanced a lot and there's less need for a build system. (ex. ESM module in the browser) I have some side projects that are mainly HTMX-based with some usage of libraries like D3.js and a small amount of hand-written Javascript. I don't feel that bad about using unpkg because I include signatures for my dependencies.
spankalee 2y ago
I don't think SRI would have ever worked in this case because not only do they dynamically generate the polyfill based on URL parameters and user agent, but they were updating the polyfill implementations over time.
adolph 2y ago
> the World Economic Forum website is affected here, for example! Absolutely ridiculous Dammit Jim, we’re economists, not dream weavers!
jsheard 2y ago
That was my thought too but polyfill.io does do a bit more than a traditional library CDN, their server dispatches a different file depending on the requesting user agent, so only the polyfills needed by that browser are delivered and newer ones don't need to download and parse a bunch of useless code. If you check the source code they deliver from a sufficiently modern browser then it doesn't contain any code at all (well, unless they decide to serve you the backdoored version...) https://polyfill.io/v3/polyfill.min.js https://polyfill.io/v3/polyfill.min.js OTOH doing it that way means you can't use subresource integrity, so you really have to trust whoever is running the CDN even more than usual. As mentioned in the OP, Cloudflare and Fastly both host their own mirrors of this service if you still need to care about old browsers.
ryan29 2y ago
The same concept should be applied to container based build pipelines too. Instead of pulling dependencies from a CDN or a pull through cache, build them into a container and use that until you're ready to upgrade dependencies. It's harder, but creates a clear boundary for updating dependencies. It also makes builds faster and makes old builds more reproducible since building an old version of your code becomes as simple as using the builder image from that point in time. Here's a nice example [1] using Java. 1. https://phauer.com/2019/no-fat-jar-in-docker-image/ https://phauer.com/2019/no-fat-jar-in-docker-image/
- hinkley 2y ago
  I get the impression this is a goal of Nix, but I haven't quite digested how their stuff works yet.
- gopher_space 2y ago
  > The same concept should be applied to container based build pipelines too. Instead of pulling dependencies from a CDN or a pull through cache, build them into a container and use that until you're ready to upgrade dependencies. Everything around your container wants to automatically update itself as well, and some of the changelogs are half emoji.
mmsc 2y ago
>self-host your dependencies behind a CDN service you control (just bunny/cloudflare/akamai/whatever is fine and cheap). This is not always possible, and some dependencies will even disallow it (think: third-party suppliers). Anyways, then that CDN service's BGP routes are hijacked. Then what? See "BGP Routes" on https://joshua.hu/how-I-backdoored-your-supply-chain https://joshua.hu/how-I-backdoored-your-supply-chain But in general, I agree: websites pointing to random js files on the internet with questionable domain independence and security is a minefield that is already exploding in some places.
bawolff 2y ago
The shared CDN model might have made sense back when browsers used a shared cache, but they dont even do that anymore. Static files are cheap to serve. Unless your site is getting hundreds of millions of page views, just plop the js file on your webserver. With HTTP/2 it will probably be almost the same speed if not faster than a cdn in practise.
- Cthulhu_ 2y ago
  If you have hundreds of millions of pageviews, go with a trusted party - someone you actually pay money to - like Cloudflare, Akamai, or any major hosting / cloud party. But not to increase cache hit rate (what CDNs were originally intended for), but to reduce latency and move resources to the edge.
  - bawolff 2y ago
    Does it even reduce latency that much (unless you have already squeezed latency out of everything else that you can)? Presumably your backend at this point is not ultra optimized. If you send a link header and using http/2 the browser will download the js file while your backend is doing its thing. I'm doubtful that moving js to the edge would help that much in such a situation unless the client is on the literal other side of the world. There of course comes a point where it does matter, i just think the cross over point is way later than people expect.
    - smaudet 2y ago
      > Does it even reduce latency that much Absolutely: https://wondernetwork.com/pings/ https://wondernetwork.com/pings/ Stockholm <-> Tokyo is at least 400ms here, anytime you have multi-national sites having a CDN is important. For your local city, not so much (and of course you won't even see it locally).
      - bawolff 2y ago
        I understand that ping times are different when geolocated. My point was that in fairly typical scenarios (worst cases are going to be worse) it would be hidden by backend latency since the fetch could be concurrent with link headers or http 103. Devil in details of course.
fhub 2y ago
I strongly believe that Browser Dev Tools should have an extra column in the network tab that highlights JS from third party domains that don't have SRI. Likewise in the Security tab and against the JS in the Application Tab.
TZubiri 2y ago
Another alternative is not to use dependencies that you or your company are not paying for.
evantbyrne 2y ago
I agree with this take, but it sounds like Funnull acquired the entirety of the project, so they could have published the malware through NPM as well.
larodi 2y ago
I can see the CDNs like CF / Akamai becoming soon like the internet 1.2 - with the legitimate stuff in, and all else considered gray/dark/1.0 web.
modeless 2y ago
Another downside of SRI is that it defeats streaming. The browser can't verify the checksum until the whole resource is downloaded so you don't get progressive decoding of images or streaming parsing of JS or HTML.
esprehn 2y ago
Also the folks who spec'ed ES6 modules didn't think it was a required feature to ship SRI from that start so it's still not broadly and easily supported across browsers. I requested the `with` style import attributes 8 years ago and it's still not available. :/
userbinator 2y ago
World Economic Forum website is affected here, for example! What did they say about ownership? How ironic.
sandwitches 2y ago
[flagged]
Bluestein 2y ago
Meanwhile ... "High security against CDN, WAF, CC, DDoS protection and SSL protects website owners and their visitors from all types of online threats" ... says the involved CDN's page (FUNNULL CDN).- (Sure. Except the one's they themselves generate. Or the CCP.)

irrational 2y ago

> self-host your dependencies I can kind of understand why people went away from this, but this is how we did it for years/decades and it just worked. Yes, doing this does require more work for you, but that's just part of the job.

jweir 2y ago
Own your process – at best that CDN is spying on your users.
marcosdumay 2y ago
> and it just worked Just to add... that is unlike the CDN thing, that will send developers into Stack Overflow looking how to set-up CORS.
toddmorey 2y ago
For performance reasons alone, you definitely want to host as much as possible on the same domain. In my experience from inside companies, we went from self-hosting with largely ssh access to complex deployment automation and CI/CD that made it hard to include any new resource in the build process. I get the temptation: resources linked from external domains / cdns gave the frontend teams quick access to the libraries, fonts, tools, etc. they needed. Thankfully things have changed for the better and it's much easier to include these things directly inside your project.
- jameshart 2y ago
  There was a brief period when the frontend dev world believed the most performant way to have everyone load, say, jquery, would be for every site to load it from the same CDN URL. From a trustworthy provider like Google, of course. It turned out the browser domain sandboxing wasn’t as good as we thought, so this opened up side channel attacks, which led to browsers getting rid of cross-domain cache sharing; and of course it turns out that there’s really no such thing as a ‘trustworthy provider’ so the web dev community memory-holed that little side adventure and pivoted to npm. Which is going GREAT by the way. The advice is still out there, of course. W3schools says: > One big advantage of using the hosted jQuery from Google: > Many users already have downloaded jQuery from Google when visiting another site. As a result, it will be loaded from cache when they visit your site https://www.w3schools.com/jquery/jquery_get_started.asp https://www.w3schools.com/jquery/jquery_get_started.asp Which hasn’t been true for years, but hey.
  - josephg 2y ago
    The only thing I’d trust w3schools to teach me is SEO. How do they stay on top of Google search results with such bad, out of date articles?
    - immibis 2y ago
      Be good at a time when Google manually ranks domains, then pivot to crap when Google stops updating the ranking. Same as the site formerly known as Wikia.
- matsemann 2y ago
  > For performance reasons alone, you definitely want to host as much as possible on the same domain. It used to be the opposite. Browsers limit the amount of concurrent requests to a domain. A way to circumvent that was to load your resources from a.example.com, b.example.com, c.example.com etc. Paying some time for extra dns resolves I guess, but could then load many more resources at the same time. Not as relevant anymore, with http2 that allows sharing connections, and more common to bundle files.

ziml77 2y ago

I've seen people reference CDNs for internal sites. I hate that because it is not only a security risk but it also means we depend on the CDN being reachable for the internal site to work. It's especially annoying because the projects I've seen it on were using NPM anyway so they could have easily pulled the dependency in through there. Hell, even without NPM it's not hard to serve these JS libraries internally since they tend to get packed into one file (+ maybe a CSS file).

simonw 2y ago

I always prefer to self-host my dependencies, but as a developer who prefer to avoid an npm-based webpack/whatever build pipeline it's often WAY harder to do that than I'd like. If you are the developer of an open source JavaScript library, please take the time to offer a downloadable version of it that works without needing to run an "npm install" and then fish the right pieces out of the node_modules folder. jQuery still offer a single minified file that I can download and use. I wish other interesting libraries would do the same! (I actually want to use ES Modules these days which makes things harder due to the way they load dependencies. I'm still trying to figure out the best way to use import maps to solve this.)

spankalee 2y ago
As you might know, Lit offers a single bundled file for the core library.
- simonw 2y ago
  Yes! Love that about Lit. The problem is when I want to add other things that have their own dependency graph.
  - spankalee 2y ago
    This is why I don't think it's very workable to avoid npm. It's the package manager of the ecosystem, and performs the job of downloading dependencies well. I personally never want to go back to the pre-package-manager days for any language.
    - PaulHoule 2y ago
      One argument is that Javascript-in-the-browser has advanced a lot and there's less need for a build system. (ex. ESM module in the browser) I have some side projects that are mainly HTMX-based with some usage of libraries like D3.js and a small amount of hand-written Javascript. I don't feel that bad about using unpkg because I include signatures for my dependencies.

spankalee 2y ago

I don't think SRI would have ever worked in this case because not only do they dynamically generate the polyfill based on URL parameters and user agent, but they were updating the polyfill implementations over time.

adolph 2y ago

> the World Economic Forum website is affected here, for example! Absolutely ridiculous Dammit Jim, we’re economists, not dream weavers!

jsheard 2y ago

That was my thought too but polyfill.io does do a bit more than a traditional library CDN, their server dispatches a different file depending on the requesting user agent, so only the polyfills needed by that browser are delivered and newer ones don't need to download and parse a bunch of useless code. If you check the source code they deliver from a sufficiently modern browser then it doesn't contain any code at all (well, unless they decide to serve you the backdoored version...) https://polyfill.io/v3/polyfill.min.js https://polyfill.io/v3/polyfill.min.js OTOH doing it that way means you can't use subresource integrity, so you really have to trust whoever is running the CDN even more than usual. As mentioned in the OP, Cloudflare and Fastly both host their own mirrors of this service if you still need to care about old browsers.

ryan29 2y ago

The same concept should be applied to container based build pipelines too. Instead of pulling dependencies from a CDN or a pull through cache, build them into a container and use that until you're ready to upgrade dependencies. It's harder, but creates a clear boundary for updating dependencies. It also makes builds faster and makes old builds more reproducible since building an old version of your code becomes as simple as using the builder image from that point in time. Here's a nice example [1] using Java. 1. https://phauer.com/2019/no-fat-jar-in-docker-image/ https://phauer.com/2019/no-fat-jar-in-docker-image/

hinkley 2y ago
I get the impression this is a goal of Nix, but I haven't quite digested how their stuff works yet.
gopher_space 2y ago
> The same concept should be applied to container based build pipelines too. Instead of pulling dependencies from a CDN or a pull through cache, build them into a container and use that until you're ready to upgrade dependencies. Everything around your container wants to automatically update itself as well, and some of the changelogs are half emoji.

mmsc 2y ago

>self-host your dependencies behind a CDN service you control (just bunny/cloudflare/akamai/whatever is fine and cheap). This is not always possible, and some dependencies will even disallow it (think: third-party suppliers). Anyways, then that CDN service's BGP routes are hijacked. Then what? See "BGP Routes" on https://joshua.hu/how-I-backdoored-your-supply-chain https://joshua.hu/how-I-backdoored-your-supply-chain But in general, I agree: websites pointing to random js files on the internet with questionable domain independence and security is a minefield that is already exploding in some places.

bawolff 2y ago

The shared CDN model might have made sense back when browsers used a shared cache, but they dont even do that anymore. Static files are cheap to serve. Unless your site is getting hundreds of millions of page views, just plop the js file on your webserver. With HTTP/2 it will probably be almost the same speed if not faster than a cdn in practise.

Cthulhu_ 2y ago
If you have hundreds of millions of pageviews, go with a trusted party - someone you actually pay money to - like Cloudflare, Akamai, or any major hosting / cloud party. But not to increase cache hit rate (what CDNs were originally intended for), but to reduce latency and move resources to the edge.
- bawolff 2y ago
  Does it even reduce latency that much (unless you have already squeezed latency out of everything else that you can)? Presumably your backend at this point is not ultra optimized. If you send a link header and using http/2 the browser will download the js file while your backend is doing its thing. I'm doubtful that moving js to the edge would help that much in such a situation unless the client is on the literal other side of the world. There of course comes a point where it does matter, i just think the cross over point is way later than people expect.
  - smaudet 2y ago
    > Does it even reduce latency that much Absolutely: https://wondernetwork.com/pings/ https://wondernetwork.com/pings/ Stockholm <-> Tokyo is at least 400ms here, anytime you have multi-national sites having a CDN is important. For your local city, not so much (and of course you won't even see it locally).
    - bawolff 2y ago
      I understand that ping times are different when geolocated. My point was that in fairly typical scenarios (worst cases are going to be worse) it would be hidden by backend latency since the fetch could be concurrent with link headers or http 103. Devil in details of course.

fhub 2y ago

I strongly believe that Browser Dev Tools should have an extra column in the network tab that highlights JS from third party domains that don't have SRI. Likewise in the Security tab and against the JS in the Application Tab.

TZubiri 2y ago

Another alternative is not to use dependencies that you or your company are not paying for.

evantbyrne 2y ago

I agree with this take, but it sounds like Funnull acquired the entirety of the project, so they could have published the malware through NPM as well.

larodi 2y ago

I can see the CDNs like CF / Akamai becoming soon like the internet 1.2 - with the legitimate stuff in, and all else considered gray/dark/1.0 web.

modeless 2y ago

Another downside of SRI is that it defeats streaming. The browser can't verify the checksum until the whole resource is downloaded so you don't get progressive decoding of images or streaming parsing of JS or HTML.

esprehn 2y ago

Also the folks who spec'ed ES6 modules didn't think it was a required feature to ship SRI from that start so it's still not broadly and easily supported across browsers. I requested the `with` style import attributes 8 years ago and it's still not available. :/

userbinator 2y ago

World Economic Forum website is affected here, for example! What did they say about ownership? How ironic.

sandwitches 2y ago

[flagged]

Bluestein 2y ago

Meanwhile ... "High security against CDN, WAF, CC, DDoS protection and SSL protects website owners and their visitors from all types of online threats" ... says the involved CDN's page (FUNNULL CDN).- (Sure. Except the one's they themselves generate. Or the CCP.)