7 ms·

Quick tip for those using static (jekyll, hugo...) sites on s3: If you have cloudflare in front of it, you can turn on aggressive html caching by creating a pag

Quick tip for those using static (jekyll, hugo...) sites on s3: If you have cloudflare in front of it, you can turn on aggressive html caching by creating a page rule: example.com/* => Cache level => Cache everything https://support.cloudflare.com/hc/en-us/articles/200172256-How-do-I-cache-static-HTML- https://support.cloudflare.com/hc/en-us/articles/200172256-H... This downtime made me realize we weren't caching any html on cloudflare. I just turned it on and all our static sites are doing fine now (and our bills are smaller!). If you're fancy you can even programmatically purge the cache when you do CI deploys using the cloudflare API.

kccqzy 9y ago
Programmatically purging caches is slow and costs extra. Just use normal cache-busting techniques like a different URL after a deployment. EDIT: Oops. Misread cloudflare/CloudFront.
- hamandcheese 9y ago
  Thats fine for assets, but not at all ideal for pages.
- sgl75 9y ago
  Changing URLs is not really an option for html pages of a static site.
  - jamiesonbecker 9y ago
    Only for the root of a site, but there are better cache expiration options as noted elsewhere anyway. For inner pages, browsers do consider /your-page-url?v=1505418887 to be a completely different page from /your-page-url?v=1505418888. However, just expiring cache and/or properly setting HTTP cache control headers at most CDN's is a cleaner and more correct option.
    - sgl75 9y ago
      I'm not following. How does that help my visitor see the latest version of the page when coming from a search engine or external site?
      - jamiesonbecker 9y ago
        Exactly, it doesn't. That's why you should use the actual cache mgmt options from your CDN provider, or just wait for your (properly configured) cache control headers to expire the cache (bear in mind that intermediate caching proxies don't always obey those TTLs or dates either). I was just pointing out that some people adjust their menu links (etc) to include some sort of dynamic variable (ie /?ts=xxx). I don't recommend this sort of scheme, though, except in unusual circumstances: IMO, it's fragile, leaky, and inefficient.
- manigandham 9y ago
  Cloudflare and all other modern CDNs do this instantly and for free. Only the legacy ones take longer (usually minutes at most) and might charge a small extra.
  - jsjohnst 9y ago
    Akamai takes far longer than a few minutes if you use all their POPs. It's in the 6-12hr range.
    - deleted 9y ago
      [deleted]
    - jsizzle 9y ago
      This is not true. Purges on akamai take less than ten minutes for entire cp codes or specific objects. Deploying new cache rules is less than an hour in many cases
      - jsjohnst 9y ago
        How many POPs are you using? I can assure you, I am not wrong here and have internal knowledge of why if you are truly using all POPs it takes so long.
        notyourday 9y ago
        That's not correct. Site delivery on Akamai globally is purged within several minutes
        jsjohnst 9y ago
        Look, I know what I'm talking about. Just because you can purge faster because of a different deployment doesn't mean that's the case for everyone. "Fast Purge" is not available for every type of deployment.
- tokenizerrr 9y ago
  No it's not? It's free to just purge everything.
teej 9y ago
Won't help if your cache is empty and Cloudflare can't access S3. I originally discovered this service outage when an image in cloudflare was returning a 500 error about 40 minutes ago.
- scrollaway 9y ago
  Yes and no. In outage periods, S3 access is often degraded because throttling is greatly increased, leading to 503s. Meaning if you have that cache layer on top of it, you dodge the issue. Where it doesn't help is if Cloudflare itself is having issues (all it does is move the weak link one layer up in this case). But that is easy enough to disable (assuming the cloudflare API or dashboard is up).
sfeng 9y ago
This is also a good time to get your static files deployed onto Azure or Google Cloud as well, and use your CDN or Cloudflare to failover.
samstave 9y ago
This should be a default feature of Cloudflare going forward if they experience massive S3 issues through their customers...? Right?
- scrollaway 9y ago
  I'm not sure why it would be a default feature. HTML caching should most definitely not be enabled by default. However, I do agree that it should be possible to turn on without pagerules for these sorts of scenarios.
- shallot_router 9y ago
  Definitely not. If HTML caching were enabled by default, I would've run into all sorts of weird issues with a lot of my Cloudflare-protected sites.
pmlnr 9y ago
Or you can host a mirror at home on a raspberry pi and redirect the dns when you have to.
- deleted 9y ago
  [deleted]
- dna_polymerase 9y ago
  Because DNS record propagation is absolutely great for this. Works within seconds!\s
  - pmlnr 9y ago
    There is a TTL you can set in your record. Yes, you can set it to a second. However, the case of a static website is not really hitting me as something can't afford a few seconds downtime...
    - dna_polymerase 9y ago
      In my experience I can set the TTL to whatever value I want, last mile providers don't really give a shit about that and just cache it for how long they want.
dalanmiller 9y ago
Does this require paid subscription to Cloudflare?
- scrollaway 9y ago
  It does not! It does take up a page rule though. You can have up to three page rules per domain for free.
seanwilson 9y ago
> Quick tip for those using static (jekyll, hugo...) sites on s3: If you have cloudflare in front of it, you can turn on aggressive html caching > If you're fancy you can even programmatically purge the cache when you do CI deploys using the cloudflare API. You need to be really careful with this. I'm not sure how you've got things set up but isn't this going to lead to issues unless you have a cache busting or revalidation strategy? You can purge Cloudflare's cache but that's not going to purge the cache on browsers that already visited your site and cached a page already. You might get cases where an old cached HTML page is asking for page resources that don't exist anymore or pages break because the user is seeing the old HTML with the new CSS/JavaScript. Also, if you can log in to your site, it can potentially cache one user's logged in page content and share it with others if your Cache-Control settings don't include "private". You could set things up to make sure the browser and then the Cloudflare cache always asks your server first if a page has been updated recently (revalidation) but your server has to be configured right for this (e.g. etag or modified date usage). Cloudflare is actually meant to have a feature that keeps a static version of your site to show in the event of outages: https://www.cloudflare.com/always-online/ https://www.cloudflare.com/always-online/
- scrollaway 9y ago
  Unfortunately afaik Always Online does not work in the event of S3 errors because S3 is available, merely answering with HTTP error codes.
  - seanwilson 9y ago
    Hmm, can you explain further? What kind of error code? So Always Online only kicks in when there's no response and not e.g. 500 error codes?
    - scrollaway 9y ago
      Correct, AFAIK Always Online kicks in when the upstream cannot be reached (eg. connection timeout, unreachable dns etc).
milankragujevic 9y ago
I don't think this is a good idea if you have lots of visits, as Cloudflare might ask you to upgrade to a Business account which is very expensive. It's excellent for small sites and blogs which are static, but for anything visited often, like a Docs site for some popular project, it might get dropped from CF's network. Don't quote me on that, I haven't experienced it, but I heard that they do ask people to upgrade, for example Brian Fritz of omdbapi.com, who had to close down the service to the public because of huge amounts of traffic, and that's just with text, not serving images or video...

kccqzy 9y ago

Programmatically purging caches is slow and costs extra. Just use normal cache-busting techniques like a different URL after a deployment. EDIT: Oops. Misread cloudflare/CloudFront.

hamandcheese 9y ago
Thats fine for assets, but not at all ideal for pages.
sgl75 9y ago
Changing URLs is not really an option for html pages of a static site.
- jamiesonbecker 9y ago
  Only for the root of a site, but there are better cache expiration options as noted elsewhere anyway. For inner pages, browsers do consider /your-page-url?v=1505418887 to be a completely different page from /your-page-url?v=1505418888. However, just expiring cache and/or properly setting HTTP cache control headers at most CDN's is a cleaner and more correct option.
  - sgl75 9y ago
    I'm not following. How does that help my visitor see the latest version of the page when coming from a search engine or external site?
    - jamiesonbecker 9y ago
      Exactly, it doesn't. That's why you should use the actual cache mgmt options from your CDN provider, or just wait for your (properly configured) cache control headers to expire the cache (bear in mind that intermediate caching proxies don't always obey those TTLs or dates either). I was just pointing out that some people adjust their menu links (etc) to include some sort of dynamic variable (ie /?ts=xxx). I don't recommend this sort of scheme, though, except in unusual circumstances: IMO, it's fragile, leaky, and inefficient.
manigandham 9y ago
Cloudflare and all other modern CDNs do this instantly and for free. Only the legacy ones take longer (usually minutes at most) and might charge a small extra.
- jsjohnst 9y ago
  Akamai takes far longer than a few minutes if you use all their POPs. It's in the 6-12hr range.
  - deleted 9y ago
    [deleted]
  - jsizzle 9y ago
    This is not true. Purges on akamai take less than ten minutes for entire cp codes or specific objects. Deploying new cache rules is less than an hour in many cases
    - jsjohnst 9y ago
      How many POPs are you using? I can assure you, I am not wrong here and have internal knowledge of why if you are truly using all POPs it takes so long.
      - notyourday 9y ago
        That's not correct. Site delivery on Akamai globally is purged within several minutes
        jsjohnst 9y ago
        Look, I know what I'm talking about. Just because you can purge faster because of a different deployment doesn't mean that's the case for everyone. "Fast Purge" is not available for every type of deployment.
tokenizerrr 9y ago
No it's not? It's free to just purge everything.

teej 9y ago

Won't help if your cache is empty and Cloudflare can't access S3. I originally discovered this service outage when an image in cloudflare was returning a 500 error about 40 minutes ago.

scrollaway 9y ago
Yes and no. In outage periods, S3 access is often degraded because throttling is greatly increased, leading to 503s. Meaning if you have that cache layer on top of it, you dodge the issue. Where it doesn't help is if Cloudflare itself is having issues (all it does is move the weak link one layer up in this case). But that is easy enough to disable (assuming the cloudflare API or dashboard is up).

sfeng 9y ago

This is also a good time to get your static files deployed onto Azure or Google Cloud as well, and use your CDN or Cloudflare to failover.

samstave 9y ago

This should be a default feature of Cloudflare going forward if they experience massive S3 issues through their customers...? Right?

scrollaway 9y ago
I'm not sure why it would be a default feature. HTML caching should most definitely not be enabled by default. However, I do agree that it should be possible to turn on without pagerules for these sorts of scenarios.
shallot_router 9y ago
Definitely not. If HTML caching were enabled by default, I would've run into all sorts of weird issues with a lot of my Cloudflare-protected sites.

pmlnr 9y ago

Or you can host a mirror at home on a raspberry pi and redirect the dns when you have to.

deleted 9y ago
[deleted]
dna_polymerase 9y ago
Because DNS record propagation is absolutely great for this. Works within seconds!\s
- pmlnr 9y ago
  There is a TTL you can set in your record. Yes, you can set it to a second. However, the case of a static website is not really hitting me as something can't afford a few seconds downtime...
  - dna_polymerase 9y ago
    In my experience I can set the TTL to whatever value I want, last mile providers don't really give a shit about that and just cache it for how long they want.

dalanmiller 9y ago

Does this require paid subscription to Cloudflare?

scrollaway 9y ago
It does not! It does take up a page rule though. You can have up to three page rules per domain for free.

seanwilson 9y ago

> Quick tip for those using static (jekyll, hugo...) sites on s3: If you have cloudflare in front of it, you can turn on aggressive html caching > If you're fancy you can even programmatically purge the cache when you do CI deploys using the cloudflare API. You need to be really careful with this. I'm not sure how you've got things set up but isn't this going to lead to issues unless you have a cache busting or revalidation strategy? You can purge Cloudflare's cache but that's not going to purge the cache on browsers that already visited your site and cached a page already. You might get cases where an old cached HTML page is asking for page resources that don't exist anymore or pages break because the user is seeing the old HTML with the new CSS/JavaScript. Also, if you can log in to your site, it can potentially cache one user's logged in page content and share it with others if your Cache-Control settings don't include "private". You could set things up to make sure the browser and then the Cloudflare cache always asks your server first if a page has been updated recently (revalidation) but your server has to be configured right for this (e.g. etag or modified date usage). Cloudflare is actually meant to have a feature that keeps a static version of your site to show in the event of outages: https://www.cloudflare.com/always-online/ https://www.cloudflare.com/always-online/

scrollaway 9y ago
Unfortunately afaik Always Online does not work in the event of S3 errors because S3 is available, merely answering with HTTP error codes.
- seanwilson 9y ago
  Hmm, can you explain further? What kind of error code? So Always Online only kicks in when there's no response and not e.g. 500 error codes?
  - scrollaway 9y ago
    Correct, AFAIK Always Online kicks in when the upstream cannot be reached (eg. connection timeout, unreachable dns etc).

milankragujevic 9y ago

I don't think this is a good idea if you have lots of visits, as Cloudflare might ask you to upgrade to a Business account which is very expensive. It's excellent for small sites and blogs which are static, but for anything visited often, like a Docs site for some popular project, it might get dropped from CF's network. Don't quote me on that, I haven't experienced it, but I heard that they do ask people to upgrade, for example Brian Fritz of omdbapi.com, who had to close down the service to the public because of huge amounts of traffic, and that's just with text, not serving images or video...