4 ms·

I actually developed a system similar to this but used docker compose as an alternative to Procfiles and nginx+le to handle dynamic virtual hosting. It's actual

I actually developed a system similar to this but used docker compose as an alternative to Procfiles and nginx+le to handle dynamic virtual hosting. It's actually a golang app that will automatically provision git repos with the necessary hooks and also allow you to exec into a container directly over SSH. I had the thought of using docker stack to achieve zero downtime but haven't had a chance to try that out. Happy to open source it if anyone is interested in using it.

jensneuse 6y ago
please do!
njsubedi 6y ago
Please open source it
dzolvd 6y ago
Would love to see it!
1337shadow 6y ago
The problem with nginx based setups is that one wrong container option (label, env var, etc ...) may cause a syntax error in the nginx configuration file and then nginx won't start, so all services will be down. Loved apache, loved nginx, but I can see traefik is the only http server on my tech blog for the last years... Nginx is made to load a configuration: you don't have the auto-configuration that comes with service discovery. Service discovery is doable with a standard HTTP API through /var/run/docker.sock or /var/run/podman/podman.sock in more advanced systems. As such, service discovering HTTP servers are more reliable because it's built with a service isolation from the ground up: if one service has some poor value then it won't work, but it won't block the other services. Nginx is too far behind now, they might have some service discovery module, but even then the thing that happens when your configuration autogenerates (like with snapshot testing) is that you still have to read the configuration it generates. Traefik offers a great dashboard for this so it's even more pleasant than reading a configuration file that you didn't even write ;) For sure, I bet that in a patch into something like CapRover (or your own solution), changing nginx to traefik would end up removing quite a lot of code ;) I'm not really sure what you mean "acheiving ZDD", ZDD is complicated any time there's a data schema migration, not to mention that containers deployment traditionally is "delete a container: KILL a process" and "create another one like cattle". uWSGI for example, could gracefully renew every worker process on SIGHUP, but re-creating the uWGSI process in another container defeats that. Maybe you have some kind of blue green deployment, maybe even canary, in this case I wonder if basing a container platform on configuration files such as those for nginx would really make it to ZDD. Would love to read more about your setup
- antoniomika 6y ago
  These are all extremely valid points! I guess I should've clarified how things work a bit (which I am currently in the process of documenting how it works and how to deploy it). Ingress management is done with the very useful nginx-proxy[0] service that loads virtual host definitions directly from the docker daemon and sets virtual hosts based on env vars set on the container. Configuration changes are loaded using an nginx reload, so even if there was an error in the configuration (which I personally have never run into though is likely possible), it wouldn't take effect. LE is then handled using the nginx-proxy-letsencrypt-companion[1]. My goal was to abstract away reverse proxy+cert management and I think any solution (traefik, caddy, etc) would work here and I'm more than happy to change it. More or less just went with nginx since it was easy and I didn't have to do any configuration other than adding it to a docker-compose file. I guess my goal wasn't to handle ZDD for stateful applications. As you mentioned, there's a plethora of issues that arise and make that type of application much more difficult to do ZDD. I tend to write a lot of stateless web apps for simple use cases and like to have an easy way to deploy them. In the primitive sense, creating a new container, waiting for it to be ready, and then swapping the upstream used for the reverse proxy with the new pointer would be ideal but isn't supported directly with docker-compose (as mentioned). Happy to talk this through also, especially if you'd be interested in contributing! [0] https://github.com/nginx-proxy/nginx-proxy https://github.com/nginx-proxy/nginx-proxy [1] https://github.com/nginx-proxy/docker-letsencrypt-nginx-proxy-companion https://github.com/nginx-proxy/docker-letsencrypt-nginx-prox...
  - 1337shadow 6y ago
    Actually, the experience with broken config comes from using nginx-proxy (along with the le companion) in production for 18 months or something, that and other inconveniences pushed me to give Traefik a try and I was really delighted, no dealing a configuration template, and I can view the configuration from the web dashboard instead of having to ssh into nginx to get the config. Traefik and Caddy are not comparable in my opinion because Traefik was literally made for self-configuration based on service discovery, see an interresting discussion here: https://www.reddit.com/r/selfhosted/comments/gq90aw/traefik_v2_or_caddy_2_for_docker_setups/ https://www.reddit.com/r/selfhosted/comments/gq90aw/traefik_... I completely agree with you about ZDD, 99.9% of uptime is plenty enough for 99.9% of the projects, and trashing the container to start a fresh process from a fresh system build does come with other advantages. Sure, any kind of blue/green deployment or canary would be really nice to see, but it wouldn't seem to create a lot of value for 99.9% of the projects, and for the rest well there's k8s that deals with clusto Currently I'm just using a bunch of ansible roles with an ansible command line wrapper, so I'll do `bigsudo yourlabs.netdata @somehost` and it'll auto-install yourlabs.traefik if not already there, which will auto-install yourlabs.docker if not already there, and basically just leave me with `https://netdata.somehost.fqdn` https://netdata.somehost.fqdn` Thank you for the invitation to contribute ! As you probably guessed, I'm a bit like you in the sense that I cannot live without making my own system, and I have made different design decisions: - Python for server side, I find it more fun than JS, nothing we can do about that - Python for client side, because we maintain our crazy isophormic component library in python - Not docker, but podman, which can run rootless and daemonless (thought we need the daemon that provides a docker compatible API to have Traefik service discovery) - Not docker build, but something I'm cooking on my own ("shlax") that I find a lot better for my taste, and that uses buildah which can build rootless - Not docker-compose, but shlax, which aims to support a broader range of use cases (such as backup/restore) - The thing I'm building is first a really KISS Sentry alternative, then also a GitLab alternative, and I'm in the process of adding CI into it ... but I stopped doing that until I have finished my little Python lib ("shlax") that replaces docker/compose and ansible to have something to put in the CI test that's not tech that I'm trying to move away from, and so that it can build/test/deploy itself, So, I suppose our goals and design decisions are a bit too different, but I can assure you that I'm always happy to see CapRover featured in social media, and I'm always happy to discuss rare passions like that, if you're looking for a crazy friend recoding his entire little world to just talk about these kind of things feel free to send me an email or give me a call ;)

jensneuse 6y ago

please do!

njsubedi 6y ago

Please open source it

dzolvd 6y ago

Would love to see it!

1337shadow 6y ago

The problem with nginx based setups is that one wrong container option (label, env var, etc ...) may cause a syntax error in the nginx configuration file and then nginx won't start, so all services will be down. Loved apache, loved nginx, but I can see traefik is the only http server on my tech blog for the last years... Nginx is made to load a configuration: you don't have the auto-configuration that comes with service discovery. Service discovery is doable with a standard HTTP API through /var/run/docker.sock or /var/run/podman/podman.sock in more advanced systems. As such, service discovering HTTP servers are more reliable because it's built with a service isolation from the ground up: if one service has some poor value then it won't work, but it won't block the other services. Nginx is too far behind now, they might have some service discovery module, but even then the thing that happens when your configuration autogenerates (like with snapshot testing) is that you still have to read the configuration it generates. Traefik offers a great dashboard for this so it's even more pleasant than reading a configuration file that you didn't even write ;) For sure, I bet that in a patch into something like CapRover (or your own solution), changing nginx to traefik would end up removing quite a lot of code ;) I'm not really sure what you mean "acheiving ZDD", ZDD is complicated any time there's a data schema migration, not to mention that containers deployment traditionally is "delete a container: KILL a process" and "create another one like cattle". uWSGI for example, could gracefully renew every worker process on SIGHUP, but re-creating the uWGSI process in another container defeats that. Maybe you have some kind of blue green deployment, maybe even canary, in this case I wonder if basing a container platform on configuration files such as those for nginx would really make it to ZDD. Would love to read more about your setup

antoniomika 6y ago
These are all extremely valid points! I guess I should've clarified how things work a bit (which I am currently in the process of documenting how it works and how to deploy it). Ingress management is done with the very useful nginx-proxy[0] service that loads virtual host definitions directly from the docker daemon and sets virtual hosts based on env vars set on the container. Configuration changes are loaded using an nginx reload, so even if there was an error in the configuration (which I personally have never run into though is likely possible), it wouldn't take effect. LE is then handled using the nginx-proxy-letsencrypt-companion[1]. My goal was to abstract away reverse proxy+cert management and I think any solution (traefik, caddy, etc) would work here and I'm more than happy to change it. More or less just went with nginx since it was easy and I didn't have to do any configuration other than adding it to a docker-compose file. I guess my goal wasn't to handle ZDD for stateful applications. As you mentioned, there's a plethora of issues that arise and make that type of application much more difficult to do ZDD. I tend to write a lot of stateless web apps for simple use cases and like to have an easy way to deploy them. In the primitive sense, creating a new container, waiting for it to be ready, and then swapping the upstream used for the reverse proxy with the new pointer would be ideal but isn't supported directly with docker-compose (as mentioned). Happy to talk this through also, especially if you'd be interested in contributing! [0] https://github.com/nginx-proxy/nginx-proxy https://github.com/nginx-proxy/nginx-proxy [1] https://github.com/nginx-proxy/docker-letsencrypt-nginx-proxy-companion https://github.com/nginx-proxy/docker-letsencrypt-nginx-prox...
- 1337shadow 6y ago
  Actually, the experience with broken config comes from using nginx-proxy (along with the le companion) in production for 18 months or something, that and other inconveniences pushed me to give Traefik a try and I was really delighted, no dealing a configuration template, and I can view the configuration from the web dashboard instead of having to ssh into nginx to get the config. Traefik and Caddy are not comparable in my opinion because Traefik was literally made for self-configuration based on service discovery, see an interresting discussion here: https://www.reddit.com/r/selfhosted/comments/gq90aw/traefik_v2_or_caddy_2_for_docker_setups/ https://www.reddit.com/r/selfhosted/comments/gq90aw/traefik_... I completely agree with you about ZDD, 99.9% of uptime is plenty enough for 99.9% of the projects, and trashing the container to start a fresh process from a fresh system build does come with other advantages. Sure, any kind of blue/green deployment or canary would be really nice to see, but it wouldn't seem to create a lot of value for 99.9% of the projects, and for the rest well there's k8s that deals with clusto Currently I'm just using a bunch of ansible roles with an ansible command line wrapper, so I'll do `bigsudo yourlabs.netdata @somehost` and it'll auto-install yourlabs.traefik if not already there, which will auto-install yourlabs.docker if not already there, and basically just leave me with `https://netdata.somehost.fqdn` https://netdata.somehost.fqdn` Thank you for the invitation to contribute ! As you probably guessed, I'm a bit like you in the sense that I cannot live without making my own system, and I have made different design decisions: - Python for server side, I find it more fun than JS, nothing we can do about that - Python for client side, because we maintain our crazy isophormic component library in python - Not docker, but podman, which can run rootless and daemonless (thought we need the daemon that provides a docker compatible API to have Traefik service discovery) - Not docker build, but something I'm cooking on my own ("shlax") that I find a lot better for my taste, and that uses buildah which can build rootless - Not docker-compose, but shlax, which aims to support a broader range of use cases (such as backup/restore) - The thing I'm building is first a really KISS Sentry alternative, then also a GitLab alternative, and I'm in the process of adding CI into it ... but I stopped doing that until I have finished my little Python lib ("shlax") that replaces docker/compose and ansible to have something to put in the CI test that's not tech that I'm trying to move away from, and so that it can build/test/deploy itself, So, I suppose our goals and design decisions are a bit too different, but I can assure you that I'm always happy to see CapRover featured in social media, and I'm always happy to discuss rare passions like that, if you're looking for a crazy friend recoding his entire little world to just talk about these kind of things feel free to send me an email or give me a call ;)