5 ms·

I've just moved over to Caddy and had a really good experience in doing so (thank you to you and all the other Caddy contributors!). One thing that stuck out to

by asb 4y ago

I've just moved over to Caddy and had a really good experience in doing so (thank you to you and all the other Caddy contributors!). One thing that stuck out to me during setup is that it seemed surprising to see gzip encoding not enabled by default. Wouldn't it make sense for a server that enables https by out of the box with zero configuration and soon supports http3 to also gzip by default?

mholt 4y ago
Good question! When I first announced Caddy in 2015 and the server got busy, it started downloading everything as .gz files. (facepalm) (good ol' days) But that's just a fun story, not actually the reason we don't enable it by default... The tricky thing about enabling features by default is that turning them off is awkward in config. And while we like to have "magic" in Caddy, we don't like too much magic. Plus, gzip performance in Go is less good (but more memory-safe) than it is in C and assembly. Klauspost's flate implementation is very fast and that's the one we use. But even with a super-fast implementation, gzip requires memory and CPU that busy servers may be in short supply of. So it's opt-in for now. We can always change it later; going the other way is harder. (Caddy also supports serving pre-compressed files, if enabled! And starting with Caddy 2.6, those can be teleported at the speed of electricity to HTTP clients using sendfile. HTTPS also sees faster file serving in 2.6 due to optimized copying.) See also: https://serverfault.com/questions/296770/why-arent-features-like-gzip-enabled-by-default-on-many-webservers https://serverfault.com/questions/296770/why-arent-features-...
- asb 4y ago
  Thanks for the response! I'd imagined reasoning along those lines. I'd perhaps expect that people anticipating such high load that gzipping output is problematic would be investing enough time in server setup to review and tweak these options. And for everyone else, compression by default is a good starting point. Moving from Nginx (HTTP2 off by default) to Caddy (HTTP2 on by default) you'd already need to review your setup if running a highly trafficked site to ensure any reverse proxied servers can handle the additional request load due to multiplexing. But it's obviously a trade-off and there's lots to weigh up...
- hsbauauvhabzb 4y ago
  Why was gzipping everything a facepalm? Performance?
  - francislavoie 4y ago
    It was making the browser download files with the .gz extension, instead of serving the content in a way that the browser would display. TL;DR it was broken at the time. But it was a very long time ago.
- layer8 4y ago
  If gzip is enabled, is the gzipped version of a static file cached in memory, or is it gzipped anew for each request?
  - mholt 4y ago
    Caddy doesn't cache responses by default, but you can enable caching with the cache plugin: https://github.com/caddyserver/cache-handler/ https://github.com/caddyserver/cache-handler/
    - oynqr 4y ago
      Does this apply to static file serving? Although I suppose relying on the OS cache might be enough.
- tyingq 4y ago
  It is an interesting decision point, and I get your reasoning. But, I have noticed there are a good number of websites that could be better for their users (especially mobile users), because someone just took the defaults. Not high priority, but if Caddy would offer a report of things that aren't configured but perhaps should be...that could help. Things like Cache-control/Expires/Etag headers, Gzip/Deflate, Caching, and so on.
JZerf 4y ago
In addition to the reasons that mholt mentioned in response to your question, enabling HTTP/GZIP compression could possibly be less secure for some web server configurations due to things like the BREACH attack. See https://en.wikipedia.org/wiki/HTTP_compression#Security_implications https://en.wikipedia.org/wiki/HTTP_compression#Security_impl... and https://en.wikipedia.org/wiki/BREACH https://en.wikipedia.org/wiki/BREACH for more info. I might be wrong but I don't think that current web serving protocols mitigate an attack like this. It might be better to default to safe settings that don't use HTTP/GZIP compression even if it might slow things down for the time being.
- thadt 4y ago
  Eh, it's not just current web serving protocols. Any protocol where: - An application uses compression - An attacker is able to supply chosen data to it - The application compresses the attacker's data and static secret data together - The attacker is able to monitor the size of the compressed data - This can be repeated by the attacker a number of times will be vulnerable to having its secret data stolen by techniques like BREACH. If you want your secret data to stay secret, don't compress it with attacker chosen plaintext where the resulting size could be monitored.

by asb 4y ago

mholt 4y ago

Good question! When I first announced Caddy in 2015 and the server got busy, it started downloading everything as .gz files. (facepalm) (good ol' days) But that's just a fun story, not actually the reason we don't enable it by default... The tricky thing about enabling features by default is that turning them off is awkward in config. And while we like to have "magic" in Caddy, we don't like too much magic. Plus, gzip performance in Go is less good (but more memory-safe) than it is in C and assembly. Klauspost's flate implementation is very fast and that's the one we use. But even with a super-fast implementation, gzip requires memory and CPU that busy servers may be in short supply of. So it's opt-in for now. We can always change it later; going the other way is harder. (Caddy also supports serving pre-compressed files, if enabled! And starting with Caddy 2.6, those can be teleported at the speed of electricity to HTTP clients using sendfile. HTTPS also sees faster file serving in 2.6 due to optimized copying.) See also: https://serverfault.com/questions/296770/why-arent-features-like-gzip-enabled-by-default-on-many-webservers https://serverfault.com/questions/296770/why-arent-features-...

asb 4y ago
Thanks for the response! I'd imagined reasoning along those lines. I'd perhaps expect that people anticipating such high load that gzipping output is problematic would be investing enough time in server setup to review and tweak these options. And for everyone else, compression by default is a good starting point. Moving from Nginx (HTTP2 off by default) to Caddy (HTTP2 on by default) you'd already need to review your setup if running a highly trafficked site to ensure any reverse proxied servers can handle the additional request load due to multiplexing. But it's obviously a trade-off and there's lots to weigh up...
hsbauauvhabzb 4y ago
Why was gzipping everything a facepalm? Performance?
- francislavoie 4y ago
  It was making the browser download files with the .gz extension, instead of serving the content in a way that the browser would display. TL;DR it was broken at the time. But it was a very long time ago.
layer8 4y ago
If gzip is enabled, is the gzipped version of a static file cached in memory, or is it gzipped anew for each request?
- mholt 4y ago
  Caddy doesn't cache responses by default, but you can enable caching with the cache plugin: https://github.com/caddyserver/cache-handler/ https://github.com/caddyserver/cache-handler/
  - oynqr 4y ago
    Does this apply to static file serving? Although I suppose relying on the OS cache might be enough.
tyingq 4y ago
It is an interesting decision point, and I get your reasoning. But, I have noticed there are a good number of websites that could be better for their users (especially mobile users), because someone just took the defaults. Not high priority, but if Caddy would offer a report of things that aren't configured but perhaps should be...that could help. Things like Cache-control/Expires/Etag headers, Gzip/Deflate, Caching, and so on.

JZerf 4y ago

In addition to the reasons that mholt mentioned in response to your question, enabling HTTP/GZIP compression could possibly be less secure for some web server configurations due to things like the BREACH attack. See https://en.wikipedia.org/wiki/HTTP_compression#Security_implications https://en.wikipedia.org/wiki/HTTP_compression#Security_impl... and https://en.wikipedia.org/wiki/BREACH https://en.wikipedia.org/wiki/BREACH for more info. I might be wrong but I don't think that current web serving protocols mitigate an attack like this. It might be better to default to safe settings that don't use HTTP/GZIP compression even if it might slow things down for the time being.

thadt 4y ago
Eh, it's not just current web serving protocols. Any protocol where: - An application uses compression - An attacker is able to supply chosen data to it - The application compresses the attacker's data and static secret data together - The attacker is able to monitor the size of the compressed data - This can be repeated by the attacker a number of times will be vulnerable to having its secret data stolen by techniques like BREACH. If you want your secret data to stay secret, don't compress it with attacker chosen plaintext where the resulting size could be monitored.