4 ms·

A small website mystery

64 points by kumaranvpl 9y ago

yes_or_gnome 9y ago
> What happened to the Content-Encoding: gzip header though? > I still don’t know where the Content-Encoding: gzip header went! Did Cloudflare remove it? Did my webhost stop serving it for some reason? I have no idea! Anyway, my site seems to work again (I think/hope?) and I thought this was kind of a fun excursion into HTTP headers. I would presume that Cloudflare's HTTP client doesn't send an 'Accept-encoding: gzip' header, and , likely, they don't support caching the 'Content-encoding:' header. I would expect this because Cloudflare's purpose is to optimize your traffic as much as possible, so to do that, they want to negotiate the least optimized content from your upstream server. Requesting unoptimized content from upstream will minimize your server's performance utilization at the coat of increased traffic utilization. However, that small increase in traffic utilization is offset by Cloudflare's content caching. Also, when Cloudflare makes that initial request due to a cache miss, there's a downstream user expecting that content. If Cloudflare wants to minimize the time of that request, as they want to do for all requests, so if they get gzipped content, then they need to decompress, store the content in cache, and, if 'accept-encoding:gzip' is present, then re-gzip the content. They are trying to minimize that turn around by not having to decompress content which the author's rewrite rules, essentially, broke the HTTProtocol contract. I'm sure it's incredibly rare to encounter one, but this very issue is expected behavior for HTTP/1.0 clients as they don't support gzip.
replete 9y ago
Most CDNs strip Accept-Encoding headers
- executesorder66 9y ago
  Why do they do that?
Sarkie 9y ago
how did you clear the Cloudflare cache?
korzun 9y ago
Why is this on the front page with over 30 points? This is not an issue with CloudFlare; their service will pass whatever weak E-Tags you want to the external request. They are proxying the misconfiguration over. The title should be 'My host turned on compression and one of the rules I wrote two years ago suddenly kicked in and broke my website'.
- Matachines 9y ago
  Bc Julia Evans wrote it
  - goldenkey 9y ago
    Hacker News should care more about the content than the author. But that's a lot to ask.
- draw_down 9y ago
  I encourage you to examine the other front page content, to perhaps disabuse you of your notions about the quality of what ends up there. One of the items currently on the front page is literally nothing more than a list of quotes that someone likes. (Who that someone is, I'll leave to you to discover.) It has 115 points.
  - Safety1stClyde 9y ago
    What content gets upvoted or downvoted on these sites is very largely random in terms of content. It seems to be related to how the first few people who see the article react to it. Once it has gained enough votes initially, somehow or other the stupidity of crowds gets it more and more votes.
- deleted 9y ago
  [deleted]
- keyboardhitter 9y ago
  I like reading about how other people troubleshoot issues, even if there isn't a drastic problem ...
Safety1stClyde 9y ago
Looking at this alexa rank: http://www.alexa.com/siteinfo/jvns.ca http://www.alexa.com/siteinfo/jvns.ca the mystery is why this site needs "Cloudflare" or a "Content Distribution Network" at all.
js2 9y ago
I have a static site, so I gzipped every page on my site, and set up this Apache configuration: RewriteEngine on RewriteCond %{HTTP:Accept-Encoding} gzip RewriteCond %{REQUEST_FILENAME}.gz -f RewriteRule ^(.*)$ $1.gz [L] This tells Apache “hey, always send gzipped replies no matter what!!”. I don't think so. The first RewriteCond is checking for the presence of an "Accept-Encoding: gzip" header (and the second that the file exists in gzip'ed form).
i336_ 9y ago
I think https://api.stackexchange.com/docs/compression https://api.stackexchange.com/docs/compression is interesting and relevant here: > While our API is HTTP based, we chose to diverge from standard HTTP in one particular area. During normal operation, we guarantee that all responses are compressed, either with GZIP or DEFLATE. Read this line carefully: > If Content-Encoding is set on the response, use the specified algorithm. If it is missing, assume GZIP. So SO's API (but ONLY the API endpoint) does precisely what this website was doing. Just thought that was interesting to share. Their rationale: > There is a way to remain in compliance with the HTTP spec, which is to reject all requests that do not list "gzip" or "deflate" in their Accept-Encoding header. Unfortunately, this does not work in practice as far too many proxies (affecting ~1% of users in our experience) will strip out this header. > The motivation for this is simple, serving uncompressed content is a loss for all parties. Bandwidth is, in comparison to CPU time, exceptional[ly] expensive and severely limited on many devices. Its really a no-brainer to require compression accordingly. And then there's this hilarious tidbit at the end: > If response is not compressed this suggests a proxy between the user and us is intentionally decompressing content, or errors are occuring very early in processing requests. You can detect uncompressed content by checking for the appropriate magic numbers, assuming your library cannot detect this error for you.
rograndom 9y ago
I ran into a similar problem when using Sucuri's Cloud Proxy. My nginx was configured to serve up Brotli compressed pages, and would work great as long as the first visitor to the site after a cache flush wasn't using chrome. If that happened, then the proxy would cache the Brotli encoded pages and serve them up to browsers that didn't support it and just show a bunch of junk. Ended up ditching the Brotli compression.
undersuit 9y ago
I saw the first dump of the headers and I thought back to the article someone linked in HN comments(https://www.fastly.com/blog/best-practices-for-using-the-vary-header https://www.fastly.com/blog/best-practices-for-using-the-var...), and I was on the right path.

64 points by kumaranvpl 9y ago

> What happened to the Content-Encoding: gzip header though? > I still don’t know where the Content-Encoding: gzip header went! Did Cloudflare remove it? Did my webhost stop serving it for some reason? I have no idea! Anyway, my site seems to work again (I think/hope?) and I thought this was kind of a fun excursion into HTTP headers. I would presume that Cloudflare's HTTP client doesn't send an 'Accept-encoding: gzip' header, and , likely, they don't support caching the 'Content-encoding:' header. I would expect this because Cloudflare's purpose is to optimize your traffic as much as possible, so to do that, they want to negotiate the least optimized content from your upstream server. Requesting unoptimized content from upstream will minimize your server's performance utilization at the coat of increased traffic utilization. However, that small increase in traffic utilization is offset by Cloudflare's content caching. Also, when Cloudflare makes that initial request due to a cache miss, there's a downstream user expecting that content. If Cloudflare wants to minimize the time of that request, as they want to do for all requests, so if they get gzipped content, then they need to decompress, store the content in cache, and, if 'accept-encoding:gzip' is present, then re-gzip the content. They are trying to minimize that turn around by not having to decompress content which the author's rewrite rules, essentially, broke the HTTProtocol contract. I'm sure it's incredibly rare to encounter one, but this very issue is expected behavior for HTTP/1.0 clients as they don't support gzip.

replete 9y ago

Most CDNs strip Accept-Encoding headers

executesorder66 9y ago
Why do they do that?

Sarkie 9y ago

how did you clear the Cloudflare cache?

korzun 9y ago

Why is this on the front page with over 30 points? This is not an issue with CloudFlare; their service will pass whatever weak E-Tags you want to the external request. They are proxying the misconfiguration over. The title should be 'My host turned on compression and one of the rules I wrote two years ago suddenly kicked in and broke my website'.

Matachines 9y ago
Bc Julia Evans wrote it
- goldenkey 9y ago
  Hacker News should care more about the content than the author. But that's a lot to ask.
draw_down 9y ago
I encourage you to examine the other front page content, to perhaps disabuse you of your notions about the quality of what ends up there. One of the items currently on the front page is literally nothing more than a list of quotes that someone likes. (Who that someone is, I'll leave to you to discover.) It has 115 points.
- Safety1stClyde 9y ago
  What content gets upvoted or downvoted on these sites is very largely random in terms of content. It seems to be related to how the first few people who see the article react to it. Once it has gained enough votes initially, somehow or other the stupidity of crowds gets it more and more votes.
deleted 9y ago
[deleted]
keyboardhitter 9y ago
I like reading about how other people troubleshoot issues, even if there isn't a drastic problem ...

Safety1stClyde 9y ago

Looking at this alexa rank: http://www.alexa.com/siteinfo/jvns.ca http://www.alexa.com/siteinfo/jvns.ca the mystery is why this site needs "Cloudflare" or a "Content Distribution Network" at all.

js2 9y ago

I have a static site, so I gzipped every page on my site, and set up this Apache configuration: RewriteEngine on RewriteCond %{HTTP:Accept-Encoding} gzip RewriteCond %{REQUEST_FILENAME}.gz -f RewriteRule ^(.*)$ $1.gz [L] This tells Apache “hey, always send gzipped replies no matter what!!”. I don't think so. The first RewriteCond is checking for the presence of an "Accept-Encoding: gzip" header (and the second that the file exists in gzip'ed form).

i336_ 9y ago

I think https://api.stackexchange.com/docs/compression https://api.stackexchange.com/docs/compression is interesting and relevant here: > While our API is HTTP based, we chose to diverge from standard HTTP in one particular area. During normal operation, we guarantee that all responses are compressed, either with GZIP or DEFLATE. Read this line carefully: > If Content-Encoding is set on the response, use the specified algorithm. If it is missing, assume GZIP. So SO's API (but ONLY the API endpoint) does precisely what this website was doing. Just thought that was interesting to share. Their rationale: > There is a way to remain in compliance with the HTTP spec, which is to reject all requests that do not list "gzip" or "deflate" in their Accept-Encoding header. Unfortunately, this does not work in practice as far too many proxies (affecting ~1% of users in our experience) will strip out this header. > The motivation for this is simple, serving uncompressed content is a loss for all parties. Bandwidth is, in comparison to CPU time, exceptional[ly] expensive and severely limited on many devices. Its really a no-brainer to require compression accordingly. And then there's this hilarious tidbit at the end: > If response is not compressed this suggests a proxy between the user and us is intentionally decompressing content, or errors are occuring very early in processing requests. You can detect uncompressed content by checking for the appropriate magic numbers, assuming your library cannot detect this error for you.

rograndom 9y ago

I ran into a similar problem when using Sucuri's Cloud Proxy. My nginx was configured to serve up Brotli compressed pages, and would work great as long as the first visitor to the site after a cache flush wasn't using chrome. If that happened, then the proxy would cache the Brotli encoded pages and serve them up to browsers that didn't support it and just show a bunch of junk. Ended up ditching the Brotli compression.

undersuit 9y ago

I saw the first dump of the headers and I thought back to the article someone linked in HN comments(https://www.fastly.com/blog/best-practices-for-using-the-vary-header https://www.fastly.com/blog/best-practices-for-using-the-var...), and I was on the right path.