3 ms·

Yes, unfortunately. I've confirmed it with their support. Requests always hit their CDN. For example, set up a small server that sends the following response he

by ezekg 3y ago

Yes, unfortunately. I've confirmed it with their support. Requests always hit their CDN. For example, set up a small server that sends the following response headers: Date: <current server timestamp> Original-Date: <same as Date> Cloudflare will always rewrite the Date header, you just won't notice it until the Date header happens to fall on a time a few ms from the next second (e.g. Wed, 13 Sep 2023 22:02:51.999). This is because by the time Cloudflare processes the request and overwrites your Date header, the Date has changed, causing the Date header to differ from Original-Date. This resulted in a considerable headache on my end [^0], because the Date header was being used for cryptographic signatures. So 1/100 requests would have an invalid signature for no apparent reason. But the reason was Cloudflare not acting as a good proxy, overwriting the Date header that had already been set. [^0]: https://news.ycombinator.com/item?id=28157629 https://news.ycombinator.com/item?id=28157629

solardev 3y ago
I see. That's a good find. Thanks for the details! And sorry about your edge (no pun intended) case.
llama052 3y ago
is using your own headers for date not the solution here? I'd imagine a lot of proxies in general will overwrite that date. At least on the corporate proxy side of things.
- ezekg 3y ago
  It's the ideal solution (and one I already implemented with a Keygen-Date header), yes, but this bug was discovered months after the feature was introduced and relied upon (I run an API product). So it's not as simple as using a different header, since previous clients will still be on the Date header. Backwards compatibility can be a bitch.
aknfxb3m 3y ago
Will your solution work with any CDN at all? It seems the issue is not specific to Cloudflare. `Date` is a registered field [0] used by origin server as defined in RFC 9110 [1]. Any CDN server will add its own date to that header since it's acting as the origin server for that request. For Cloudflare in particular, you could try setting Cache-Control to Private=Date and see if the edge cache skips the override [2]. What you want seems to be a tunnel with WAF. In that case, a Cloudflare Worker might be a workaround to the native caching layer but I'm not sure if that has any effect on the edge device updating the Date header or not. Workers do support WAF. 0 - https://www.iana.org/assignments/http-fields/http-fields.xhtml#field-names https://www.iana.org/assignments/http-fields/http-fields.xht... 1 - https://www.rfc-editor.org/rfc/rfc9110.html https://www.rfc-editor.org/rfc/rfc9110.html 2 - https://developers.cloudflare.com/cache/concepts/cache-control/ https://developers.cloudflare.com/cache/concepts/cache-contr...
- ezekg 3y ago
  I don't know, and that's why I want a WAF without a CDN. I don't need a CDN at all, and I wish I could disable Cloudflare's. But per RFC 9110 [^0], "the Date header field represents the date and time at which the message was originated." The message comes from my server, not Cloudflare. If they're getting the message from me, i.e. it's not cached by their CDN, they should leave the Date header alone because the message didn't originate with them. [^1] Maybe I have a misunderstanding of where Cloudflare sits as the "origin server" when their CDN isn't used for caching (i.e. it's bypassed as much as is allowable). I would think my server is the "origin" and theirs is the "edge." And unfortunately, workers do not allow you to set the Date header -- Cloudflare support and I already tried that a couple years back (i.e. setting Date = Keygen-Date via a worker). I will take a look at 2. I doubt that'd work, but it's worth a try. [^0]: https://www.rfc-editor.org/rfc/rfc9110.html#name-date https://www.rfc-editor.org/rfc/rfc9110.html#name-date [^1]: https://httpwg.org/specs/rfc7230.html#rfc.section.5.7.2 https://httpwg.org/specs/rfc7230.html#rfc.section.5.7.2
  - aknfxb3m 3y ago
    Yeah, looks like CF should not consider itself the origin, at least if the request wasn't cached. But I'm thinking the edge cache is dumb. Some of their docs mention that requests that aren't cached are actually just revalidated and cached at the edge (probably TTL=0) and then served by the dumb edge cache... and they didn't consider they shouldn't be entitled to be the origin in that case.

by ezekg 3y ago

solardev 3y ago

I see. That's a good find. Thanks for the details! And sorry about your edge (no pun intended) case.

llama052 3y ago

is using your own headers for date not the solution here? I'd imagine a lot of proxies in general will overwrite that date. At least on the corporate proxy side of things.

ezekg 3y ago
It's the ideal solution (and one I already implemented with a Keygen-Date header), yes, but this bug was discovered months after the feature was introduced and relied upon (I run an API product). So it's not as simple as using a different header, since previous clients will still be on the Date header. Backwards compatibility can be a bitch.

aknfxb3m 3y ago

Will your solution work with any CDN at all? It seems the issue is not specific to Cloudflare. `Date` is a registered field [0] used by origin server as defined in RFC 9110 [1]. Any CDN server will add its own date to that header since it's acting as the origin server for that request. For Cloudflare in particular, you could try setting Cache-Control to Private=Date and see if the edge cache skips the override [2]. What you want seems to be a tunnel with WAF. In that case, a Cloudflare Worker might be a workaround to the native caching layer but I'm not sure if that has any effect on the edge device updating the Date header or not. Workers do support WAF. 0 - https://www.iana.org/assignments/http-fields/http-fields.xhtml#field-names https://www.iana.org/assignments/http-fields/http-fields.xht... 1 - https://www.rfc-editor.org/rfc/rfc9110.html https://www.rfc-editor.org/rfc/rfc9110.html 2 - https://developers.cloudflare.com/cache/concepts/cache-control/ https://developers.cloudflare.com/cache/concepts/cache-contr...

ezekg 3y ago
I don't know, and that's why I want a WAF without a CDN. I don't need a CDN at all, and I wish I could disable Cloudflare's. But per RFC 9110 [^0], "the Date header field represents the date and time at which the message was originated." The message comes from my server, not Cloudflare. If they're getting the message from me, i.e. it's not cached by their CDN, they should leave the Date header alone because the message didn't originate with them. [^1] Maybe I have a misunderstanding of where Cloudflare sits as the "origin server" when their CDN isn't used for caching (i.e. it's bypassed as much as is allowable). I would think my server is the "origin" and theirs is the "edge." And unfortunately, workers do not allow you to set the Date header -- Cloudflare support and I already tried that a couple years back (i.e. setting Date = Keygen-Date via a worker). I will take a look at 2. I doubt that'd work, but it's worth a try. [^0]: https://www.rfc-editor.org/rfc/rfc9110.html#name-date https://www.rfc-editor.org/rfc/rfc9110.html#name-date [^1]: https://httpwg.org/specs/rfc7230.html#rfc.section.5.7.2 https://httpwg.org/specs/rfc7230.html#rfc.section.5.7.2
- aknfxb3m 3y ago
  Yeah, looks like CF should not consider itself the origin, at least if the request wasn't cached. But I'm thinking the edge cache is dumb. Some of their docs mention that requests that aren't cached are actually just revalidated and cached at the edge (probably TTL=0) and then served by the dumb edge cache... and they didn't consider they shouldn't be entitled to be the origin in that case.