6 ms·

> People take libraries like lodash – or jQuery, as we analyzed earlier – and insert the whole thing into their codebases. If a simple bundler plugin could deal

> People take libraries like lodash – or jQuery, as we analyzed earlier – and insert the whole thing into their codebases. If a simple bundler plugin could deal with getting rid of everything in lodash they aren’t using, footprint is one less thing we’d have to worry about. If you use Google CDN, why does it matter how big jQuery is? If N people use their own "smaller" M-byte copy of jQuery, browsers will have to download M*N bytes, as opposed to 0 bytes if you use the cached full version. A profound waste of bandwidth. My advice: Use Google CDN, for less common stuff use cdnjs. Don't adulterate libraries!

dpkrjb 11y ago
I believe the main problem with googles cdn is it can prevent Chinese users from visiting your site
- vosper 11y ago
  I would imagine it's China that's preventing Chinese users from visiting your site, not Google?
  - ybx 11y ago
    Yes, but the end result is the same
- alextgordon 11y ago
  I guess, but if you go to the trouble of localising your website into Chinese, then loading a self-hosted version of the library for Chinese users doesn't seem like much extra work.
- everfree 11y ago
  This can be solved through the use of a local fallback if the request to the Google CDN returns 404.
  - robocat 11y ago
    Needs 1 or more of: Handling out-of-order js file loading Duplicate loading prevention A way to deal with a file that is timing out (holding up page load) Ideally using async property for script tags AFAIK most CDN versions of popular libs don't support what you need, so you might as well load locally...
sotojuan 11y ago
Tree-shaking and ES6 modules are coming. Rollup[1] supports them and Webpack will in the future. They can't come soon enough! [1] https://github.com/rollup/rollup#a-next-generation-es6-module-bundler https://github.com/rollup/rollup#a-next-generation-es6-modul...
LewisJEllis 11y ago
Sure, if someone's not using a popular 3rd party CDN because they don't know the option exists or they don't understand the potential benefits, then yes, consideration should be given. They're not always the answer, though, and there are plenty of legitimate business and security reasons why one might prefer to host their own dependencies. Sometimes technical reasons make the potential benefit minimal, anyway. Once someone's already making one request for your application source, adding a bit more to that response (a bundled minimum-necessary-subset of some not-massive dependency) can make only a very minor performance difference. It won't be as fast as if a file containing the entire dependency was in the browser's cache (or possibly even in the JS engine's compilation cache), but it'll be significantly faster than if it wasn't.
rapind 11y ago
There are quite a few disadvantages to using a CDN like Google. - Delay for DNS resolution and the new TCP connection could be non-trivial (some tests show 300ms+). - A JS CDN is also likely tracking your traffic and you may not want them to. - No offline dev environment (on the plane). - Server run (phantom etc.) tests might run super slow if they have to pull in a remote js library. - Probably issues when it comes to apps that are meant to work offline, although I think this is solved with service workers proxying the CDN request. And if the recommendation is to continue loading large libraries from a CDN and nevermind only what you need, well there's still a memory cost. If I have 5+ SPAs running in tabs that are each fairly complex (you know, like Gmail), it does start to add up. Because of the way that tabs are sandboxed I believe this may mean 5 copies of jQuery or React or Angular or w/e.
- SapphireSun 11y ago
  I've also encountered problems with China blocking jQuery or fonts from Google.
  - Grue3 11y ago
    This is solvable by also hosting a local copy and load it if CDN is not available. Example (from my website): <script src="//ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script> <script>window.jQuery || document.write('<script src="/js/vendor/jquery-1.10.2.min.js"><\/script>')</script>
- mateo411 11y ago
  It's possible to not use the CDN when you are developing in your local environment, and use the CDN in your production environment. That way you can keep developing even when you are on an airplane.
- edejong 11y ago
  Plus the cost of parsing the JSON and merging the huge object structures these libraries provide. I missed one item on your list, which is security. Google does not provide any guarantees on the safety of the CDN provider versions of JQuery. Imagine the mayhem when a widely used jquery library is replaced by a backdoored version.
citricsquid 11y ago
There was an article in recent memory that looked at how various websites were using CDN's for delivering common libraries and if I recall correctly they concluded that the chance of getting a cache hit from a CDN is relatively low because of the different versions each website is using. The Google CDN for example offers dozens and dozens of different versions of jQuery: your visitor might have visited a site using jQuery before, but have they visited a site using the same version? Unfortunately I cannot find the article, maybe someone else will be able to provide a link. There are other concerns too, if your website depends on a Javascript library and the CDN is offline it can break your website (which has happened with Google CDN before, and many of the other popular Javascript CDN's). What if the CDN is compromised and is delivering malicious content? Not impossible.
- mutagen 11y ago
  I'm only aware of this slightly older (11/2011) article [1] on the topic. I've poked at the numbers recently in Big Query and see a wide spread for jQuery versions on popular sites. Maybe one of these days I'll get around to documenting it properly. I'd rather measure from a sampling of client side pageloads but this might be a bigger project to tackle. [1] http://statichtml.com/2011/google-ajax-libraries-caching.html http://statichtml.com/2011/google-ajax-libraries-caching.htm... [2] http://stackoverflow.com/questions/29930805/measuring-visitor-http-cache-hit-ratio-for-external-cdn-resources http://stackoverflow.com/questions/29930805/measuring-visito...
mutagen 11y ago
Version fragmentation pushes the CDN hit rate down quite a bit. Unless you're using the exact major/minor version that some popular sites are using you're going to see high miss rates, especially on mobile where it likely matters the most.
vonmoltke 11y ago
My greatest frustration with JavaScript tooling is that it assumes all sites will be deployed on networks with connections to the internet. Trying to make standalone deployment packages for closed networks has been a nightmare for me.
haberman 11y ago
How can you use a CDN if you're also using a bundler/minifier? Put another way, how can you bundle your app such that CDN-available libs are "dynamically linked" but your app and less common libs are "statically linked"?
- drhayes9 11y ago
  You make the thing you're requiring an "external", such that the bundler knows it's available somewhere but won't include it. Here's browserify's documentation about the matter: https://github.com/substack/node-browserify#external-requires https://github.com/substack/node-browserify#external-require...
  - haberman 11y ago
    The doc you linked to describes how to create a bundle of JS that will export a "require()" function exposing the bundled libs to other JavaScript. That doesn't seem related to the question of how to create a bundle that assumes that some libs have already been loaded with a previous script tag that pulled, say, jQuery from a CDN.
    - drhayes9 11y ago
      Whoops! That's what I get for speed-linking. I meant this one: https://github.com/substack/node-browserify#bexternalfile https://github.com/substack/node-browserify#bexternalfile For something like jQuery you might also need a browserify shim. And now we're edging closer into some of the stuff this essay decries.
- arohner 11y ago
  You can always upload the .js files to your own CDN. The other magic of CDNs (aside from the potential cache reuse that everyone is talking about in this thead) is that the user downloads files from a geographically closer server to them than your server. This is typically a huge speed up, and applies even if your JS files are 100% unique.
xorcist 11y ago
You might want to seek out actual numbers before you say that. More often than not, that particular cache entry is evicted because of the billion combinations of frameworks, versions and CDNs. You are very far from downloading 0 bytes even for the popular frameworks. There's also the issue of availability. If you decide to use one, make sure to have local fallback. The one time I did some remote instrumentation on this, some users actually had trouble reaching the CDN. I have no idea why, it might have been DNS failures or routing problems or whatever, but that made the whole site freeze for those affected since there was no fallback.
arohner 11y ago
Your cache isn't as warm as you think it is. The cache on mobile browsers is ~50MB, and 250MB on IE11. It's smaller on older IEs. Most regular internet users will download more than that every single day. You should always use a CDN, but that doesn't excuse large libraries. Shameless plug, but my startup, https://rasterize.io https://rasterize.io can tell you your cache hit rate for every visitor to your site.
dustingetz 11y ago
what everybody else is missing: browser still has to parse & execute the javascript, which is half of the startup time, and if you are fully a SPA without server-side rendered markup, this increases your time-to-glass by about double on web clients
Lazare 11y ago
> If you use Google CDN, why does it matter how big jQuery is? 1. Most people won't have the version you wanted to use cached from the location you were trying to use it from. Studies show hit rates on requests for libraries on "shared" CDNs are very low in real world conditions. (Also, caches are small. Oh, you wanted 2.3.4r17 of BigLib? I've only got 2.3.4r18 cached; I'll go ahead and delete that so I have room to download your new copy.) 2. We don't get have HTTP/2, which means that while the number of bytes is important, so is the number of requests. If your app can get by with JUST jQuery, that doesn't matter. But if you need jQuery, plus two plugins, and lodash, and a promise polyfill, and...then all of a sudden your performance starts to be dominated by the number of requests being made, and (again) in real world situations you may see your effective load times improve if you concatenate your libraries. The dream is "just list the libraries you need and let the browser sort it out super fast". The reality is a long way from it, and I don't think telling people "just use a CDN!" is currently good advice.
- extrapickles 11y ago
  To put some numbers to this, lets assume the user has a 1mbit connection with 300ms latency. If a library is 32kb (on the wire) it will take 256ms to transmit to the client. If its not inlined, it would take 556ms to transmit, or almost twice as long for the cold cache case. If the client has a 10mbit connection (average cheap 4g mobile connection), with the same latency, its even worse as the 32kb library now takes ~27ms to download if inlined and 327ms if not. This is now a order of magnitude slower. With a latency of 100ms (typical home internet), its still 127ms to download the cold cache vs the 27ms inline takes. Since connection bandwidth is slowly rising, it makes CDN library distribution a even worse idea as time passes. The speed of light (in copper/fiber) is the dominating factor in 100ms latency, so it will not be much improved without massive expense. It may even get to the point where inlining the library is faster than waiting for a cheap mobile device to fetch the cached copy out of flash.
paulirish 11y ago
I've worked on the Google JS library CDN for a few years now. Our data indicates that cache hit rates are very low. You'd be better off serving jQuery from your libs.min.js. In general, browser caches fill up so fast and there are so many versions of jQuery in use in the wild, that the cache never ends up being as useful as we'd hope. Sorry. :/
- erikpukinskis 11y ago
  You need "LTS" releases of libraries and clients which auto-update to the newest point release on the LTS branch before this will work. Although, that's the right direction for package management to go anyway, so we might as well.

dpkrjb 11y ago

I believe the main problem with googles cdn is it can prevent Chinese users from visiting your site

vosper 11y ago
I would imagine it's China that's preventing Chinese users from visiting your site, not Google?
- ybx 11y ago
  Yes, but the end result is the same
alextgordon 11y ago
I guess, but if you go to the trouble of localising your website into Chinese, then loading a self-hosted version of the library for Chinese users doesn't seem like much extra work.
everfree 11y ago
This can be solved through the use of a local fallback if the request to the Google CDN returns 404.
- robocat 11y ago
  Needs 1 or more of: Handling out-of-order js file loading Duplicate loading prevention A way to deal with a file that is timing out (holding up page load) Ideally using async property for script tags AFAIK most CDN versions of popular libs don't support what you need, so you might as well load locally...

sotojuan 11y ago

Tree-shaking and ES6 modules are coming. Rollup[1] supports them and Webpack will in the future. They can't come soon enough! [1] https://github.com/rollup/rollup#a-next-generation-es6-module-bundler https://github.com/rollup/rollup#a-next-generation-es6-modul...

LewisJEllis 11y ago

Sure, if someone's not using a popular 3rd party CDN because they don't know the option exists or they don't understand the potential benefits, then yes, consideration should be given. They're not always the answer, though, and there are plenty of legitimate business and security reasons why one might prefer to host their own dependencies. Sometimes technical reasons make the potential benefit minimal, anyway. Once someone's already making one request for your application source, adding a bit more to that response (a bundled minimum-necessary-subset of some not-massive dependency) can make only a very minor performance difference. It won't be as fast as if a file containing the entire dependency was in the browser's cache (or possibly even in the JS engine's compilation cache), but it'll be significantly faster than if it wasn't.

rapind 11y ago

There are quite a few disadvantages to using a CDN like Google. - Delay for DNS resolution and the new TCP connection could be non-trivial (some tests show 300ms+). - A JS CDN is also likely tracking your traffic and you may not want them to. - No offline dev environment (on the plane). - Server run (phantom etc.) tests might run super slow if they have to pull in a remote js library. - Probably issues when it comes to apps that are meant to work offline, although I think this is solved with service workers proxying the CDN request. And if the recommendation is to continue loading large libraries from a CDN and nevermind only what you need, well there's still a memory cost. If I have 5+ SPAs running in tabs that are each fairly complex (you know, like Gmail), it does start to add up. Because of the way that tabs are sandboxed I believe this may mean 5 copies of jQuery or React or Angular or w/e.

SapphireSun 11y ago
I've also encountered problems with China blocking jQuery or fonts from Google.
- Grue3 11y ago
  This is solvable by also hosting a local copy and load it if CDN is not available. Example (from my website): <script src="//ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script> <script>window.jQuery || document.write('<script src="/js/vendor/jquery-1.10.2.min.js"><\/script>')</script>
mateo411 11y ago
It's possible to not use the CDN when you are developing in your local environment, and use the CDN in your production environment. That way you can keep developing even when you are on an airplane.
edejong 11y ago
Plus the cost of parsing the JSON and merging the huge object structures these libraries provide. I missed one item on your list, which is security. Google does not provide any guarantees on the safety of the CDN provider versions of JQuery. Imagine the mayhem when a widely used jquery library is replaced by a backdoored version.

citricsquid 11y ago

There was an article in recent memory that looked at how various websites were using CDN's for delivering common libraries and if I recall correctly they concluded that the chance of getting a cache hit from a CDN is relatively low because of the different versions each website is using. The Google CDN for example offers dozens and dozens of different versions of jQuery: your visitor might have visited a site using jQuery before, but have they visited a site using the same version? Unfortunately I cannot find the article, maybe someone else will be able to provide a link. There are other concerns too, if your website depends on a Javascript library and the CDN is offline it can break your website (which has happened with Google CDN before, and many of the other popular Javascript CDN's). What if the CDN is compromised and is delivering malicious content? Not impossible.

mutagen 11y ago
I'm only aware of this slightly older (11/2011) article [1] on the topic. I've poked at the numbers recently in Big Query and see a wide spread for jQuery versions on popular sites. Maybe one of these days I'll get around to documenting it properly. I'd rather measure from a sampling of client side pageloads but this might be a bigger project to tackle. [1] http://statichtml.com/2011/google-ajax-libraries-caching.html http://statichtml.com/2011/google-ajax-libraries-caching.htm... [2] http://stackoverflow.com/questions/29930805/measuring-visitor-http-cache-hit-ratio-for-external-cdn-resources http://stackoverflow.com/questions/29930805/measuring-visito...

mutagen 11y ago

Version fragmentation pushes the CDN hit rate down quite a bit. Unless you're using the exact major/minor version that some popular sites are using you're going to see high miss rates, especially on mobile where it likely matters the most.

vonmoltke 11y ago

My greatest frustration with JavaScript tooling is that it assumes all sites will be deployed on networks with connections to the internet. Trying to make standalone deployment packages for closed networks has been a nightmare for me.

haberman 11y ago

How can you use a CDN if you're also using a bundler/minifier? Put another way, how can you bundle your app such that CDN-available libs are "dynamically linked" but your app and less common libs are "statically linked"?

drhayes9 11y ago
You make the thing you're requiring an "external", such that the bundler knows it's available somewhere but won't include it. Here's browserify's documentation about the matter: https://github.com/substack/node-browserify#external-requires https://github.com/substack/node-browserify#external-require...
- haberman 11y ago
  The doc you linked to describes how to create a bundle of JS that will export a "require()" function exposing the bundled libs to other JavaScript. That doesn't seem related to the question of how to create a bundle that assumes that some libs have already been loaded with a previous script tag that pulled, say, jQuery from a CDN.
  - drhayes9 11y ago
    Whoops! That's what I get for speed-linking. I meant this one: https://github.com/substack/node-browserify#bexternalfile https://github.com/substack/node-browserify#bexternalfile For something like jQuery you might also need a browserify shim. And now we're edging closer into some of the stuff this essay decries.
arohner 11y ago
You can always upload the .js files to your own CDN. The other magic of CDNs (aside from the potential cache reuse that everyone is talking about in this thead) is that the user downloads files from a geographically closer server to them than your server. This is typically a huge speed up, and applies even if your JS files are 100% unique.

xorcist 11y ago

You might want to seek out actual numbers before you say that. More often than not, that particular cache entry is evicted because of the billion combinations of frameworks, versions and CDNs. You are very far from downloading 0 bytes even for the popular frameworks. There's also the issue of availability. If you decide to use one, make sure to have local fallback. The one time I did some remote instrumentation on this, some users actually had trouble reaching the CDN. I have no idea why, it might have been DNS failures or routing problems or whatever, but that made the whole site freeze for those affected since there was no fallback.

arohner 11y ago

Your cache isn't as warm as you think it is. The cache on mobile browsers is ~50MB, and 250MB on IE11. It's smaller on older IEs. Most regular internet users will download more than that every single day. You should always use a CDN, but that doesn't excuse large libraries. Shameless plug, but my startup, https://rasterize.io https://rasterize.io can tell you your cache hit rate for every visitor to your site.

dustingetz 11y ago

what everybody else is missing: browser still has to parse & execute the javascript, which is half of the startup time, and if you are fully a SPA without server-side rendered markup, this increases your time-to-glass by about double on web clients

Lazare 11y ago

> If you use Google CDN, why does it matter how big jQuery is? 1. Most people won't have the version you wanted to use cached from the location you were trying to use it from. Studies show hit rates on requests for libraries on "shared" CDNs are very low in real world conditions. (Also, caches are small. Oh, you wanted 2.3.4r17 of BigLib? I've only got 2.3.4r18 cached; I'll go ahead and delete that so I have room to download your new copy.) 2. We don't get have HTTP/2, which means that while the number of bytes is important, so is the number of requests. If your app can get by with JUST jQuery, that doesn't matter. But if you need jQuery, plus two plugins, and lodash, and a promise polyfill, and...then all of a sudden your performance starts to be dominated by the number of requests being made, and (again) in real world situations you may see your effective load times improve if you concatenate your libraries. The dream is "just list the libraries you need and let the browser sort it out super fast". The reality is a long way from it, and I don't think telling people "just use a CDN!" is currently good advice.

extrapickles 11y ago
To put some numbers to this, lets assume the user has a 1mbit connection with 300ms latency. If a library is 32kb (on the wire) it will take 256ms to transmit to the client. If its not inlined, it would take 556ms to transmit, or almost twice as long for the cold cache case. If the client has a 10mbit connection (average cheap 4g mobile connection), with the same latency, its even worse as the 32kb library now takes ~27ms to download if inlined and 327ms if not. This is now a order of magnitude slower. With a latency of 100ms (typical home internet), its still 127ms to download the cold cache vs the 27ms inline takes. Since connection bandwidth is slowly rising, it makes CDN library distribution a even worse idea as time passes. The speed of light (in copper/fiber) is the dominating factor in 100ms latency, so it will not be much improved without massive expense. It may even get to the point where inlining the library is faster than waiting for a cheap mobile device to fetch the cached copy out of flash.

paulirish 11y ago

I've worked on the Google JS library CDN for a few years now. Our data indicates that cache hit rates are very low. You'd be better off serving jQuery from your libs.min.js. In general, browser caches fill up so fast and there are so many versions of jQuery in use in the wild, that the cache never ends up being as useful as we'd hope. Sorry. :/

erikpukinskis 11y ago
You need "LTS" releases of libraries and clients which auto-update to the newest point release on the LTS branch before this will work. Although, that's the right direction for package management to go anyway, so we might as well.