4 ms·

It could be that some particular request header (maybe an unusual user-agent) causes the web server to fail to send a content-type, for some reason. But if the

It could be that some particular request header (maybe an unusual user-agent) causes the web server to fail to send a content-type, for some reason. But if the response was nevertheless marked cacheable, then Cloudflare may have cached it and served it to other users who didn't send the unusual header. This problem would seem to go away if you disabled Cloudflare caching, but would still technically be a problem with the web server, not Cloudflare... When thinking about cost, don't forget to factor in developer time. As a rule of thumb, your time as a developer is worth $1 per minute. (Disclosure: I'm the tech lead of Cloudflare Workers.)

user5994461 8y ago
You need to set "Vary: User-Agent" when the response varies. It's very important.
- manigandham 8y ago
  Cloudflare CDN does not use the `Vary` header and only caches by the URL, unless you have an enterprise plan.
  - user5994461 8y ago
    Good to know. That breaks a lot of stuff. Good thing we always had the enterprise plan at work.
    - kentonv 8y ago
      `Vary` is tricky because it potentially forces the cache to do many more disk reads per lookup, hurting performance. Without `Vary`, there's at most one cache entry per cache key, and logic is simple: load that entry, verify it hasn't expired, and then return. With `Vary`, there could be many entries matching any cache key, and you can't even compute a hash bucket upfront because you have to examine the `Vary` header on the cached response to find out which headers matter. You either have to do a linear scan of all entries for that URL, or you need to maintain some sort of fancy index, and even then it's going to be slower than without `Vary`. FWIW, with Cloudflare Workers you can implement logic that is (almost) strictly superior to Vary, and doesn't have these performance challenges. Instead of varying on a header, you can write code to vary on arbitrary properties of the request. For example, `Vary: User-Agent` would normally partition the cache not only for every browser, but for every minor version of every browser and every OS version, ruining your cache hit rate. With a Worker, you can parse the User-Agent and decide the buckets in your own way in code, then compute a custom cache key (or just add a query parameter to the URL) based on that. (Workers run "in front of" cache, so you get to modify the request before cache lookup occurs.) The only down side compared to the header is that your code has to know what to vary on at request time, whereas the `Vary` header can be determined at response time. But if you're writing code specific to some application or web site, usually this is not that hard to determine based on URL alone. (Disclosure: I'm the tech lead of Cloudflare Workers.)
      - byroot 8y ago
        > With `Vary`, there could be many entries matching any cache key I'm curious. Why can't the headers listed in the `Vary` header just be made part of the cache key?
        kentonv 8y ago
        Because `Vary` is part of the response, not the request. So to find out which headers are listed in `Vary`, you have to look up the response. To look up the response, you need the cache key. So the cache key cannot depend on `Vary`.
        tedunangst 8y ago
        This just means when you look up the response in the cache, it returns not a single response but a set organized by Vary header and you pick the right one. It's not impossible.
        kentonv 8y ago
        Yes that's what I said: > You either have to do a linear scan of all entries for that URL, or you need to maintain some sort of fancy index
        byroot 8y ago
        Oh right. I was only thinking of revalidated requests. Thanks.
      - user5994461 8y ago
        It doesn't affect performance of the browser or the web server, but I understand that it takes some work to support for a CDN. Too bad since you run a CDN ;) I had to debug some pretty nasty issues caused by missing Vary. I would certainly do without it if it were possible, but it's not an option. Vary is strictly required for anything that renders content per browser (User-Agent), compression (Content-Type Content-Encoding) or CORS policies (Origin).
Sir_Cmpwn 8y ago
It's not pure cost. The knowledge and experience gained from chasing the bug all the way down is worth something.
newscracker 8y ago
The reason I mentioned cost here is because NFSN, the web host that Julia said she uses for this blog, is a pay-as-you-go service that works out to be a lot cheaper for low volume use compared to oversold hosts that promise unlimited storage/bandwidth (or very large numbers that in reality would trigger abuse prevention mechanisms). Of course, NFSN is also run without any nonsense marketing, and that’s a plus to support. So I’d assumed that cost may have been a reason she chose NFSN (though her situation might’ve changed since the choice and there wasn’t enough time to move to another host).

user5994461 8y ago

You need to set "Vary: User-Agent" when the response varies. It's very important.

manigandham 8y ago
Cloudflare CDN does not use the `Vary` header and only caches by the URL, unless you have an enterprise plan.
- user5994461 8y ago
  Good to know. That breaks a lot of stuff. Good thing we always had the enterprise plan at work.
  - kentonv 8y ago
    `Vary` is tricky because it potentially forces the cache to do many more disk reads per lookup, hurting performance. Without `Vary`, there's at most one cache entry per cache key, and logic is simple: load that entry, verify it hasn't expired, and then return. With `Vary`, there could be many entries matching any cache key, and you can't even compute a hash bucket upfront because you have to examine the `Vary` header on the cached response to find out which headers matter. You either have to do a linear scan of all entries for that URL, or you need to maintain some sort of fancy index, and even then it's going to be slower than without `Vary`. FWIW, with Cloudflare Workers you can implement logic that is (almost) strictly superior to Vary, and doesn't have these performance challenges. Instead of varying on a header, you can write code to vary on arbitrary properties of the request. For example, `Vary: User-Agent` would normally partition the cache not only for every browser, but for every minor version of every browser and every OS version, ruining your cache hit rate. With a Worker, you can parse the User-Agent and decide the buckets in your own way in code, then compute a custom cache key (or just add a query parameter to the URL) based on that. (Workers run "in front of" cache, so you get to modify the request before cache lookup occurs.) The only down side compared to the header is that your code has to know what to vary on at request time, whereas the `Vary` header can be determined at response time. But if you're writing code specific to some application or web site, usually this is not that hard to determine based on URL alone. (Disclosure: I'm the tech lead of Cloudflare Workers.)
    - byroot 8y ago
      > With `Vary`, there could be many entries matching any cache key I'm curious. Why can't the headers listed in the `Vary` header just be made part of the cache key?
      - kentonv 8y ago
        Because `Vary` is part of the response, not the request. So to find out which headers are listed in `Vary`, you have to look up the response. To look up the response, you need the cache key. So the cache key cannot depend on `Vary`.
        tedunangst 8y ago
        This just means when you look up the response in the cache, it returns not a single response but a set organized by Vary header and you pick the right one. It's not impossible.
        kentonv 8y ago
        Yes that's what I said: > You either have to do a linear scan of all entries for that URL, or you need to maintain some sort of fancy index
        byroot 8y ago
        Oh right. I was only thinking of revalidated requests. Thanks.
    - user5994461 8y ago
      It doesn't affect performance of the browser or the web server, but I understand that it takes some work to support for a CDN. Too bad since you run a CDN ;) I had to debug some pretty nasty issues caused by missing Vary. I would certainly do without it if it were possible, but it's not an option. Vary is strictly required for anything that renders content per browser (User-Agent), compression (Content-Type Content-Encoding) or CORS policies (Origin).

Sir_Cmpwn 8y ago

It's not pure cost. The knowledge and experience gained from chasing the bug all the way down is worth something.

newscracker 8y ago

The reason I mentioned cost here is because NFSN, the web host that Julia said she uses for this blog, is a pay-as-you-go service that works out to be a lot cheaper for low volume use compared to oversold hosts that promise unlimited storage/bandwidth (or very large numbers that in reality would trigger abuse prevention mechanisms). Of course, NFSN is also run without any nonsense marketing, and that’s a plus to support. So I’d assumed that cost may have been a reason she chose NFSN (though her situation might’ve changed since the choice and there wasn’t enough time to move to another host).