2 ms·
> I used nginx for a very, very long time but abandoned it. I still think that Nginx is pretty good, however there are a few annoying things about it, certific
by KronisLV 4y ago
> I used nginx for a very, very long time but abandoned it.
I still think that Nginx is pretty good, however there are a few annoying things about it, certificates being just one of them.
Personally, I also found that attempting to use it as a reverse proxy kills the entire instance when there is no DNS record for one of the sites (say, 1 out of 20 that are proxied). I ran into this when running containers that hadn't passed health checks and therefore didn't have any traffic routed to them, which meant that if any of them went down, all of them would be unreachable. Furthermore, the popular suggestion of using a variable for the proxy URL just broke redirects in some apps: https://blog.kronis.dev/everything%20is%20broken/nginx-configuration-is-broken https://blog.kronis.dev/everything%20is%20broken/nginx-confi...
Caddy is pretty good, though I couldn't get the equivalents of all the options that are available in other web servers, for example allowing encoded slashes when hosting a Sonatype Nexus instance like what you can do with Apache. Things might have changed, but last I tried, it didn't quite seem to work: https://help.sonatype.com/repomanager3/planning-your-implementation/run-behind-a-reverse-proxy#RunBehindaReverseProxy-ApachehttpdwithnpmRepositories https://help.sonatype.com/repomanager3/planning-your-impleme...
Furthermore, there were issues with certificates as well: if the configuration for one of the aforementioned 20 sites was bad and the certificate couldn't be renewed, then the entire instance went down. What does this mean? Well, if I run Caddy as a container, I'll get the "fail fast" approach, which will sadly then mean that I'll get a restart, that will still fail with the bad configuration and eventually will hit Let's Encrypt rate limits. Sure, I can have some alerting (extra work) or have restart backoff/delays (though this will be bad for restart times if the instance were to ever crash for other reasons, e.g. load), but neither seems like an actual solution. Very annoying, especially because everything will once again go down, instead of being built for resiliency.
In the end, I kind of just went with Apache for the time being, because despite being a bit awkward at times, it's still an okay web server for the scales that I work at and is okay to deploy inside of containers. I wrote about it more on my blog, "How and why to use Apache httpd in 2022": https://blog.kronis.dev/tutorials/how-and-why-to-use-apache-httpd-in-2022 https://blog.kronis.dev/tutorials/how-and-why-to-use-apache-...
It even has Let's Encrypt (well, ACME) support built in, thanks to mod_md nowadays: https://httpd.apache.org/docs/trunk/mod/mod_md.html https://httpd.apache.org/docs/trunk/mod/mod_md.html
Of course, I still use Nginx + PHP-FPM with supervisord for containers where I need to run PHP, because of some other weirdness when you try to get it working with Apache, which probably has something to do with how I build container images. That said, I couldn't actually find what the problem was, so it was easier for me to just use Apache on the edge and Nginx for the apps (PHP, or maybe even serving static assets), about which I wrote on yet another blog post, "Containers are broken": https://blog.kronis.dev/everything%20is%20broken/containers-are-broken https://blog.kronis.dev/everything%20is%20broken/containers-...
In short, it's nice to have a choice of web servers and being able to pick whatever is the best suited for your needs. It's just that there's a lot of weirdness going on with what should be simple configurations.
- mholt 4y agoFYI, I just revamped how Caddy handles encoded slashes: https://github.com/caddyserver/caddy/pull/4948 https://github.com/caddyserver/caddy/pull/4948 It's much smarter now. Caddy also doesn't stop if a certificate can't be obtained. When did you last try? Maybe a big was fixed but it would have been YEARS ago.
- KronisLV 4y agoThanks for the response, that's really nice to hear! I'll probably check out the more recent versions of Caddy as well, admittedly they're most likely a rather painless experience for most folks when compared with Apache/Nginx, though I've also heard good things about Traefik, which is even integrated in K3s as an ingress. I guess most web servers are workable for a variety of use cases with a bit of effort, though I definitely reminded myself why people weren't exactly the biggest fans of the Apache config syntax just recently.