3 ms·
This is actually a problem whether you're using continuous deployment or not. You need to put a release out, which you have done some testing for, but you know
by nbm 10y ago
This is actually a problem whether you're using continuous deployment or not. You need to put a release out, which you have done some testing for, but you know that only real production traffic can shake out certain types of problems.
You don't have to have all three servers getting the same amount of traffic, and you don't have to have a single copy of your service on each server. So, you could reduce the weight of a single server that does canary traffic to reduce the pain, or you could run two copies of your service on a server, and have the canary copy get a trickle of traffic.
Another approach is to use shadow traffic - instead of handling the requests on the canary host, you handle it on the production host _and_ the canary host. You'd need to ensure the canary can't adjust the production database, for example - or maybe you only shadow read requests. If you don't get any errors, or you're able to prove to yourself that they function the same, you can then move to a more traditional canary.
You definitely need to adjust your continuous deployment implementation to your environment, whatever it is.