5 ms·

When you deploy new versions of your app, is there a short downtime between the moment you stop the old version and the moment you start the new one?

movedx 8y ago
At present, yes. It could be easily overcome with a simple A/B deployment mechanism.
- ngrilly 8y ago
  I used to do that before switching to Docker swarm mode (docker stack deploy automates rolling deploys with zero downtime). In my experience, it's not that simple to do it correctly. You have to start the new containers, wait for them to be ready, then switch traffic to the new containers, drain connections to the old containers, and eventually stop the old containers. You need some kind of reverse proxy to do this, or at least iptable rules. I'm curious: how would you do it? Is your app directly exposed on the Internet, or is it already behind a reverse proxy?
  - movedx 8y ago
    You're not wrong. It's not simple, but it's also not that hard, really. Yeah I use a DigitalOcean load balancer in front of the servers. At this point in time the application serves my family and friends. It's not critical. And even when or if it does become public, even then I would just use maintenance windows. Not sure what the benefit of zero downtime deployments are, to be honest. They make things complicated for a few seconds of downtime to switch traffic over. You can even make a maintenance page that's all fancy and checks for the backend coming back up and then redirects traffic... not that hard at all. I might consider Swarm. Probably not though. I do agree those technologies can complicate matters but they can also make life easier. I guess it's a balancing act, isn't it?
    - ngrilly 8y ago
      Thanks for following up! > Yeah I use a DigitalOcean load balancer in front of the servers. Why the load balancer if the app only serves family and friends, and maintenance windows are okay? It should be able to run on a single server. > They make things complicated for a few seconds of downtime In some cases, it's more than "a few seconds of downtime". In one of the apps I maintain, some requests accept file uploads, and some stream long .csv files that are generated on demand. These requests can take several tens of seconds. If the system is unable to start a new version of the app, and switch traffic to it, while keeping existing connections to the old version, then we're talking about at least 1 minute of downtime. We've thought about simplifying things using a "maintenance page that automatically checks for the backend coming back", but 1 minute is just too long for our paying users. Some day, we deploy new versions multiple times. Our paying users would be unhappy about the multiple interruptions :-) That said, with the advent of single-page applications, maybe it should be okay to transparently retry requests on the client side while the app is restarting on the server side (returning 503). No maintenance page. No complex zero downtime logic. > I might consider Swarm. We were already using Docker and docker-compose. We manage zero downtime deployments using docker-compose and nginx as a reverse proxy, scripted with Ansible but it was a bit too hacky for my taste. It's when I started to consider Docker swarm mode. And it's really great, even on a single-node, especially when you're already using docker-compose. I'm oversimplifying a bit, but it's basically running `docker swarm init` and replacing docker-compose with `docker stack deploy`. I recommend it.
      - movedx 8y ago
        > Why the load balancer if the app only serves family and friends, and maintenance windows are okay? It should be able to run on a single server. I use DO to manage the DNS for the domain. I use their LB to manage the TLS cert for the domain. By doing this I don't have to manage TLS connections in my application nor deal with certification creation and rotations. DO are doing it for me for $10/month. That's a bargain given my time is worth $100/hour. I can also easily swap out the backend server, or add more, as I see fit. This falls in line with your zero-downtime argument. The LB is there and ready to go should I want to attempt such things :) > In some cases, it's more than "a few seconds of downtime". In one of the apps I maintain, some requests accept file uploads, and some stream long .csv files that are generated on demand. These requests can take several tens of seconds. If the system is unable to start a new version of the app, and switch traffic to it, while keeping existing connections to the old version, then we're talking about at least 1 minute of downtime. But this is the key point of it all: your application and its users demand zero-downtime. My mother (in-law) does not :) > I'm oversimplifying a bit, but it's basically running `docker swarm init` and replacing docker-compose with `docker stack deploy`. I recommend it. Looking into it right now. Thanks a lot for the recommendation.

movedx 8y ago

At present, yes. It could be easily overcome with a simple A/B deployment mechanism.

ngrilly 8y ago
I used to do that before switching to Docker swarm mode (docker stack deploy automates rolling deploys with zero downtime). In my experience, it's not that simple to do it correctly. You have to start the new containers, wait for them to be ready, then switch traffic to the new containers, drain connections to the old containers, and eventually stop the old containers. You need some kind of reverse proxy to do this, or at least iptable rules. I'm curious: how would you do it? Is your app directly exposed on the Internet, or is it already behind a reverse proxy?
- movedx 8y ago
  You're not wrong. It's not simple, but it's also not that hard, really. Yeah I use a DigitalOcean load balancer in front of the servers. At this point in time the application serves my family and friends. It's not critical. And even when or if it does become public, even then I would just use maintenance windows. Not sure what the benefit of zero downtime deployments are, to be honest. They make things complicated for a few seconds of downtime to switch traffic over. You can even make a maintenance page that's all fancy and checks for the backend coming back up and then redirects traffic... not that hard at all. I might consider Swarm. Probably not though. I do agree those technologies can complicate matters but they can also make life easier. I guess it's a balancing act, isn't it?
  - ngrilly 8y ago
    Thanks for following up! > Yeah I use a DigitalOcean load balancer in front of the servers. Why the load balancer if the app only serves family and friends, and maintenance windows are okay? It should be able to run on a single server. > They make things complicated for a few seconds of downtime In some cases, it's more than "a few seconds of downtime". In one of the apps I maintain, some requests accept file uploads, and some stream long .csv files that are generated on demand. These requests can take several tens of seconds. If the system is unable to start a new version of the app, and switch traffic to it, while keeping existing connections to the old version, then we're talking about at least 1 minute of downtime. We've thought about simplifying things using a "maintenance page that automatically checks for the backend coming back", but 1 minute is just too long for our paying users. Some day, we deploy new versions multiple times. Our paying users would be unhappy about the multiple interruptions :-) That said, with the advent of single-page applications, maybe it should be okay to transparently retry requests on the client side while the app is restarting on the server side (returning 503). No maintenance page. No complex zero downtime logic. > I might consider Swarm. We were already using Docker and docker-compose. We manage zero downtime deployments using docker-compose and nginx as a reverse proxy, scripted with Ansible but it was a bit too hacky for my taste. It's when I started to consider Docker swarm mode. And it's really great, even on a single-node, especially when you're already using docker-compose. I'm oversimplifying a bit, but it's basically running `docker swarm init` and replacing docker-compose with `docker stack deploy`. I recommend it.
    - movedx 8y ago
      > Why the load balancer if the app only serves family and friends, and maintenance windows are okay? It should be able to run on a single server. I use DO to manage the DNS for the domain. I use their LB to manage the TLS cert for the domain. By doing this I don't have to manage TLS connections in my application nor deal with certification creation and rotations. DO are doing it for me for $10/month. That's a bargain given my time is worth $100/hour. I can also easily swap out the backend server, or add more, as I see fit. This falls in line with your zero-downtime argument. The LB is there and ready to go should I want to attempt such things :) > In some cases, it's more than "a few seconds of downtime". In one of the apps I maintain, some requests accept file uploads, and some stream long .csv files that are generated on demand. These requests can take several tens of seconds. If the system is unable to start a new version of the app, and switch traffic to it, while keeping existing connections to the old version, then we're talking about at least 1 minute of downtime. But this is the key point of it all: your application and its users demand zero-downtime. My mother (in-law) does not :) > I'm oversimplifying a bit, but it's basically running `docker swarm init` and replacing docker-compose with `docker stack deploy`. I recommend it. Looking into it right now. Thanks a lot for the recommendation.