3 ms·

A couple things here I'd say. The first is that you have to be careful with the 'live' page stuff on Cloudflare and blaming it directly on Cloudflare. I can't

by druiid 14y ago

A couple things here I'd say. The first is that you have to be careful with the 'live' page stuff on Cloudflare and blaming it directly on Cloudflare. I can't know the exact circumstances that lead you to see the error, but in our experience the couple times that we've had a problem with this, it's been an actual issue on our side that wasn't presenting itself at great frequency, but enough to trigger a live-site error once in a while. As for imgur, I'd have to look around their site, but I'd say perhaps they didn't do their homework completely? I mentioned it on a previous post but you have to be extremely careful with setting headers for objects or every request will be proxying for an image from Cloudflare, back to the internet, then to your servers, then back through the internet, through Cloudflare and then finally to the user. Instead, you want to make sure that Cloudflare is caching mostly everything EXCEPT for the page body if at all possible. A developer accidentally turned off proper headers a while back and well, we noticed it in page load speed issues pretty quickly. Edit: Also, you probably want to see if you can cache the page body as well ;).

walls 14y ago
It's good to hear some insight into this, as it's really been grating on me lately, so thanks! I'm somewhat curious about what kind of problem you could have that would allow the 'live' site to work, but break through CloudFlare. That seems entirely counter to the functionality... do you happen to remember or are you able to comment on what was happening?
- druiid 14y ago
  Sure. There are at least a couple ways I've seen that it can be triggered (and if one of the Cloudflare guys on HN is around they might have more or correct me if I'm wrong). The first seems to be that if a page is experiencing slow loading times on the upstream locations (and pretty much everyone using Cloudflare is likely just an 'upstream' entry in nginx), you can trigger the error. If the upstream server returns a 500 error I have seen certain circumstances where that will return a live page error... or, if the web-server on the remote end terminates the connection for whatever reason that will also cause it to happen. For us the last time we had a problem it turned out to be pretty complicated. There was a bug (which we reported and was fixed in the last release) for the php NewRelic module. Whenever there was what the module considered a 'slow' mysql query, it would cause a segmentation fault. This particular day there was randomly a few 'slow' queries, thus when the segmentation fault occurred the php thread would die killing off the connection and bringing up the 'page is down' error from Cloudflare. That was a pain to diagnose. Strace for the win.
cbr 14y ago
you have to be extremely careful with setting headers for objects or every request will be proxying for an image from Cloudflare, back to the internet, then to your servers, then back through the internet, through Cloudflare and then finally to the user. I'm pretty sure PageSpeed Service will do this too if you set headers to prohibit caching (which is depressingly common).
- druiid 14y ago
  I imagine that would be correct, yeah. If you don't specify to cache something, it shouldn't be cached. If a proxy out there still caches it, I'd consider it broken.
  - cbr 14y ago
    From memory, if something has no caching headers PageSpeed Service will treat it as Cache-Control: public, max-age=600 Combined with serving html as uncacheable, this is a pragmatic and functional way of dealing with the many sites that haven't put much (any) thought into caching.
    - xpose2000 14y ago
      I just looked at the PageSpeed dash, if no cache headers are specified then it defaults to 1800 seconds.
    - druiid 14y ago
      Ah, interesting. Thanks for noting this. Something I'll keep in mind if we ever look at the PageSpeed service. I'm actually not certain if Cloudflare is doing something similar for pages with no headers set.

by druiid 14y ago

walls 14y ago

It's good to hear some insight into this, as it's really been grating on me lately, so thanks! I'm somewhat curious about what kind of problem you could have that would allow the 'live' site to work, but break through CloudFlare. That seems entirely counter to the functionality... do you happen to remember or are you able to comment on what was happening?

druiid 14y ago
Sure. There are at least a couple ways I've seen that it can be triggered (and if one of the Cloudflare guys on HN is around they might have more or correct me if I'm wrong). The first seems to be that if a page is experiencing slow loading times on the upstream locations (and pretty much everyone using Cloudflare is likely just an 'upstream' entry in nginx), you can trigger the error. If the upstream server returns a 500 error I have seen certain circumstances where that will return a live page error... or, if the web-server on the remote end terminates the connection for whatever reason that will also cause it to happen. For us the last time we had a problem it turned out to be pretty complicated. There was a bug (which we reported and was fixed in the last release) for the php NewRelic module. Whenever there was what the module considered a 'slow' mysql query, it would cause a segmentation fault. This particular day there was randomly a few 'slow' queries, thus when the segmentation fault occurred the php thread would die killing off the connection and bringing up the 'page is down' error from Cloudflare. That was a pain to diagnose. Strace for the win.

cbr 14y ago

you have to be extremely careful with setting headers for objects or every request will be proxying for an image from Cloudflare, back to the internet, then to your servers, then back through the internet, through Cloudflare and then finally to the user. I'm pretty sure PageSpeed Service will do this too if you set headers to prohibit caching (which is depressingly common).

druiid 14y ago
I imagine that would be correct, yeah. If you don't specify to cache something, it shouldn't be cached. If a proxy out there still caches it, I'd consider it broken.
- cbr 14y ago
  From memory, if something has no caching headers PageSpeed Service will treat it as Cache-Control: public, max-age=600 Combined with serving html as uncacheable, this is a pragmatic and functional way of dealing with the many sites that haven't put much (any) thought into caching.
  - xpose2000 14y ago
    I just looked at the PageSpeed dash, if no cache headers are specified then it defaults to 1800 seconds.
  - druiid 14y ago
    Ah, interesting. Thanks for noting this. Something I'll keep in mind if we ever look at the PageSpeed service. I'm actually not certain if Cloudflare is doing something similar for pages with no headers set.