7 ms·

That's even with the proper Vary header, right? Seems like a cloudflare bug if that's true. Maybe even a security bug if you find a service that supports text/p

That's even with the proper Vary header, right? Seems like a cloudflare bug if that's true. Maybe even a security bug if you find a service that supports text/plain.

zoogeny 3y ago
I haven't dealt with Cloudflare specifically, but I did deal with a number of big CDNs for large amounts of traffic. They were pretty adamant about NOT supporting arbitrary Vary header values. It broke some logic on a few of our systems and we eventually just decided to work around it instead of pushing our case. Interestingly, one of the big CDN providers did have controls in their UI for explicitly allowing/disallowing Vary header entries but they disabled it for us at some point (e.g. it was still in the UI but greyed out). I assumed once we hit a certain level of traffic it was too computationally expensive? Ever since, I've avoided any kind of fancy header/response variance in APIs just in case I end up in the same situation. It is rarely a necessity. IIRC, the only thing they continued to support variance wise was gzip (e.g. content-encoding). It's also worth noting they were extremely conservative with query parameters too. Also to reiterate, this was very high traffic and high volume with expectations of low latency, so probably not applicable to most people using CDNs for static website assets.
- derefr 3y ago
  > I assumed once we hit a certain level of traffic it was too computationally expensive? Seems strange; AFAIK in e.g. Varnish, Vary just means you get more "stuff" tacked onto the buffer that gets built from the request and then hashed to create the cache key. And actually, come to think of it, if memory for N concurrent in-flight requests is the concern, then you don't even need an actual (dynamically allocated) buffer, either; presuming you're using a streaming hash, you can feed each constituent field directly into the hasher, with only the hasher's (probably stack-allocated) internal static buffer for blockwise hashing required. (Which you're gonna need regardless of whether you're doing any Vary-ing.) So it's really just a question of how many CPU cycles are being spent hashing. And it's likely just going to be a difference between hashing 300 bytes (base request — hostname, path, headers that are always implicitly Varied upon) and 350 bytes (those things, plus whatever you explicitly Varied) per request. Doesn't seem like too much of a win... (especially when hardware-accelerated hashing ops operate on blocks anyway, such that you only get stepwise cost increases for every e.g. 128 bytes.) I wonder why they bothered?
  - zoogeny 3y ago
    To be clear, I'm not trying to make their argument for them since we spent probably 1 day working around it. I'm just passing along an anecdote. One day, Vary header stopped working on one CDN and we had to fix it. When I spoke to our account rep (I literally had a weekly call with them due to our usage) he said they were phasing it out for performance reasons. Not long after we got notice from another CDN asking for similar consideration. I have no inside knowledge as to their infrastructure or systems that made this a requirement. I very much doubt it was the cost to hash, maybe more likely something to do with their network topology and how requests were routed from origin to regional tiers to PoPs? I'm totally speculating here. If this had been a necessity then I would have probably dug into the request more deeply. It was a "pick your battles" kind of thing. Extremely low cost on our side to change, no reason to bother if they claimed it would decrease problems on their side.
  - deleted 3y ago
    [deleted]
  - phanimahesh 3y ago
    The cost of vary headers is usually not in hashing the keys but storing multiple entries per url in an arbitrarily large combination of headers. I can imagine cdns not wanting the hassle, though I don't live the outcome.
    - zoogeny 3y ago
      It's worth noting that the cost of storage wasn't the issue in this case. They already had a system that allowed you to determine which headers in the Vary list would be respected and so you could calculate a worst-case storage load. I mean, it definitely was an issue in general and we were careful about avoiding the same content being stored multiple times but it wasn't the reasoning they communicated behind the change in the anecdote I related. I think the best suggestion was in another thread by @johncolanduoni where he pointed out the difficulty of storing, distributing and retrieving the metadata per-resource that would be necessary for each PoP to correctly determine the Vary requirements at request time.
    - 3y ago
- wbl 3y ago
  The problem with Vary is that it massively expands footprints and reduces cache efficiency when overused. In a CDN this can create noisy neighbor like issues.

That's even with the proper Vary header, right? Seems like a cloudflare bug if that's true. Maybe even a security bug if you find a service that supports text/plain.

zoogeny 3y ago

I haven't dealt with Cloudflare specifically, but I did deal with a number of big CDNs for large amounts of traffic. They were pretty adamant about NOT supporting arbitrary Vary header values. It broke some logic on a few of our systems and we eventually just decided to work around it instead of pushing our case. Interestingly, one of the big CDN providers did have controls in their UI for explicitly allowing/disallowing Vary header entries but they disabled it for us at some point (e.g. it was still in the UI but greyed out). I assumed once we hit a certain level of traffic it was too computationally expensive? Ever since, I've avoided any kind of fancy header/response variance in APIs just in case I end up in the same situation. It is rarely a necessity. IIRC, the only thing they continued to support variance wise was gzip (e.g. content-encoding). It's also worth noting they were extremely conservative with query parameters too. Also to reiterate, this was very high traffic and high volume with expectations of low latency, so probably not applicable to most people using CDNs for static website assets.

derefr 3y ago
> I assumed once we hit a certain level of traffic it was too computationally expensive? Seems strange; AFAIK in e.g. Varnish, Vary just means you get more "stuff" tacked onto the buffer that gets built from the request and then hashed to create the cache key. And actually, come to think of it, if memory for N concurrent in-flight requests is the concern, then you don't even need an actual (dynamically allocated) buffer, either; presuming you're using a streaming hash, you can feed each constituent field directly into the hasher, with only the hasher's (probably stack-allocated) internal static buffer for blockwise hashing required. (Which you're gonna need regardless of whether you're doing any Vary-ing.) So it's really just a question of how many CPU cycles are being spent hashing. And it's likely just going to be a difference between hashing 300 bytes (base request — hostname, path, headers that are always implicitly Varied upon) and 350 bytes (those things, plus whatever you explicitly Varied) per request. Doesn't seem like too much of a win... (especially when hardware-accelerated hashing ops operate on blocks anyway, such that you only get stepwise cost increases for every e.g. 128 bytes.) I wonder why they bothered?
- zoogeny 3y ago
  To be clear, I'm not trying to make their argument for them since we spent probably 1 day working around it. I'm just passing along an anecdote. One day, Vary header stopped working on one CDN and we had to fix it. When I spoke to our account rep (I literally had a weekly call with them due to our usage) he said they were phasing it out for performance reasons. Not long after we got notice from another CDN asking for similar consideration. I have no inside knowledge as to their infrastructure or systems that made this a requirement. I very much doubt it was the cost to hash, maybe more likely something to do with their network topology and how requests were routed from origin to regional tiers to PoPs? I'm totally speculating here. If this had been a necessity then I would have probably dug into the request more deeply. It was a "pick your battles" kind of thing. Extremely low cost on our side to change, no reason to bother if they claimed it would decrease problems on their side.
- deleted 3y ago
  [deleted]
- phanimahesh 3y ago
  The cost of vary headers is usually not in hashing the keys but storing multiple entries per url in an arbitrarily large combination of headers. I can imagine cdns not wanting the hassle, though I don't live the outcome.
  - zoogeny 3y ago
    It's worth noting that the cost of storage wasn't the issue in this case. They already had a system that allowed you to determine which headers in the Vary list would be respected and so you could calculate a worst-case storage load. I mean, it definitely was an issue in general and we were careful about avoiding the same content being stored multiple times but it wasn't the reasoning they communicated behind the change in the anecdote I related. I think the best suggestion was in another thread by @johncolanduoni where he pointed out the difficulty of storing, distributing and retrieving the metadata per-resource that would be necessary for each PoP to correctly determine the Vary requirements at request time.
  - 3y ago
wbl 3y ago
The problem with Vary is that it massively expands footprints and reduces cache efficiency when overused. In a CDN this can create noisy neighbor like issues.