5 ms·

It's sad that there's still no good way to do deployment based expiration of assets without horrible hacks like sticking the asset checksum in the URL. I know

It's sad that there's still no good way to do deployment based expiration of assets without horrible hacks like sticking the asset checksum in the URL. I know that none of my assets will ever change unless a deployment occurs, and even then, most of them won't change. HTTP doesn't seem to support this use case well at all.

toomim 9y ago
What do you mean by "deployment-based"? You want things to expire each time you "git pull" on the server?
- jeremiep 9y ago
  Wait you're actually running git on production boxes? Doesn't that mean your entire build toolchain also lives on production? The last company I worked for did that and everything was much slower and fragile than it would've been had we deployed packaged artifacts instead.
  - jrochkind1 9y ago
    Eh, heroku seems to handle it fine.
    - krallja 9y ago
      No, Heroku has a separate build step before it deploys to your dynos.
  - octalmage 9y ago
    Yeah this just doesn't scale, especially if you're pulling from GitHub.
- masterleep 9y ago
  I'd like to have a way to force If-Modified type checks on cached assets after a deployment, but not if no deployment occurred.
  - jrochkind1 9y ago
    I mean, there's no way to tell remote agents which aren't in contact with you to contact you, or that the cache should be busted. There's no way to broadcast this "NOW you need an if-modified check, cause I just did a deployment" to all possible caching agents. Not even really any reasonably sane way theoretically/hypothetically, on the internet we've got. Either you've got to tell the remote agent how long to cache for without checking in, or it's got to check in regularly to see if it's cache is still good, I think that's a basic constraint that can't be gotten around. You can use an ETag of a checksum, instead of a checksum in the filename. Now a user-agent can just check with an if-modified-since and get a response quickly. But it's still got to check regularly. That or guaranteed changed URLs when content changes are about the only way it's ever going to work in the HTTP client-server architecture, I think. Maybe there's a creative way to come up with guaranteed-changed-url that isn't as inconvenient in development for you, but most people find the current practices a pretty good spot I think.
    - masterleep 9y ago
      It can be done if resources are associated, perhaps by site, or by path within a site, and if there's at least one non-cached resource. For example: Add a HTTP header on resource responses that is 1:1 with the deployment. When this header changes on any response, treat any associated resources in the cache as needing an If-Modified checkin. Then add that header to a non-cached dynamic page, like the user's home page. When the browser checks this page and sees a deployment change, it'll know to check the static assets.
dom0 9y ago
> without horrible hacks like sticking the asset checksum in the URL. What's wrong with this approach?
- masterleep 9y ago
  It requires a way more complicated toolchain because you have to use dynamic methods to generate references to the assets in HTML/CSS/JS.
  - mikeytown2 9y ago
    With that more complicated toolchain comes some added benefits though. See https://www.drupal.org/project/advagg https://www.drupal.org/project/advagg as an example that just about does it all.
  - dom0 9y ago
    You don't really need a toolchain, it can all be done lazily be the web application server.
icebraining 9y ago
You don't need to use a checksum, just add some irrelevant URL parameter like "?sitever=X" to all asset URLs, then in the packaging/build script just replace it with the current site version: find ./ -type f -exec sed -i s/?sitever=X/?sitever=$VERSION/g' {} \;
- JoshTriplett 9y ago
  That will unnecessarily break caching of resources that haven't changed between versions. (Hence, hashing the actual content.)
  - icebraining 9y ago
    I suppose if you deploy a lot, that can be wasteful.
underwater 9y ago
We're almost there with ServiceWorkers. A site should soon be able to push a new manifest that updates or invalidates the cached resources.
- masterleep 9y ago
  Great pointer on the ServiceWorkers, thanks!

toomim 9y ago

What do you mean by "deployment-based"? You want things to expire each time you "git pull" on the server?

jeremiep 9y ago
Wait you're actually running git on production boxes? Doesn't that mean your entire build toolchain also lives on production? The last company I worked for did that and everything was much slower and fragile than it would've been had we deployed packaged artifacts instead.
- jrochkind1 9y ago
  Eh, heroku seems to handle it fine.
  - krallja 9y ago
    No, Heroku has a separate build step before it deploys to your dynos.
- octalmage 9y ago
  Yeah this just doesn't scale, especially if you're pulling from GitHub.
masterleep 9y ago
I'd like to have a way to force If-Modified type checks on cached assets after a deployment, but not if no deployment occurred.
- jrochkind1 9y ago
  I mean, there's no way to tell remote agents which aren't in contact with you to contact you, or that the cache should be busted. There's no way to broadcast this "NOW you need an if-modified check, cause I just did a deployment" to all possible caching agents. Not even really any reasonably sane way theoretically/hypothetically, on the internet we've got. Either you've got to tell the remote agent how long to cache for without checking in, or it's got to check in regularly to see if it's cache is still good, I think that's a basic constraint that can't be gotten around. You can use an ETag of a checksum, instead of a checksum in the filename. Now a user-agent can just check with an if-modified-since and get a response quickly. But it's still got to check regularly. That or guaranteed changed URLs when content changes are about the only way it's ever going to work in the HTTP client-server architecture, I think. Maybe there's a creative way to come up with guaranteed-changed-url that isn't as inconvenient in development for you, but most people find the current practices a pretty good spot I think.
  - masterleep 9y ago
    It can be done if resources are associated, perhaps by site, or by path within a site, and if there's at least one non-cached resource. For example: Add a HTTP header on resource responses that is 1:1 with the deployment. When this header changes on any response, treat any associated resources in the cache as needing an If-Modified checkin. Then add that header to a non-cached dynamic page, like the user's home page. When the browser checks this page and sees a deployment change, it'll know to check the static assets.

dom0 9y ago

> without horrible hacks like sticking the asset checksum in the URL. What's wrong with this approach?

masterleep 9y ago
It requires a way more complicated toolchain because you have to use dynamic methods to generate references to the assets in HTML/CSS/JS.
- mikeytown2 9y ago
  With that more complicated toolchain comes some added benefits though. See https://www.drupal.org/project/advagg https://www.drupal.org/project/advagg as an example that just about does it all.
- dom0 9y ago
  You don't really need a toolchain, it can all be done lazily be the web application server.

icebraining 9y ago

You don't need to use a checksum, just add some irrelevant URL parameter like "?sitever=X" to all asset URLs, then in the packaging/build script just replace it with the current site version: find ./ -type f -exec sed -i s/?sitever=X/?sitever=$VERSION/g' {} \;

JoshTriplett 9y ago
That will unnecessarily break caching of resources that haven't changed between versions. (Hence, hashing the actual content.)
- icebraining 9y ago
  I suppose if you deploy a lot, that can be wasteful.

underwater 9y ago

We're almost there with ServiceWorkers. A site should soon be able to push a new manifest that updates or invalidates the cached resources.

masterleep 9y ago
Great pointer on the ServiceWorkers, thanks!