3 ms·
We do the same - it was surprisingly simple to implement - - CircleCI listens across all GIT branches and injects $CIRCLE_BRANCH into ENV - A build step po
by e1g 9y ago
We do the same - it was surprisingly simple to implement -
- CircleCI listens across all GIT branches and injects $CIRCLE_BRANCH into ENV
- A build step populates a vhost template to point "$CIRCLE_BRANCH.qa.bofh.com" to "/var/www/$CIRCLE_BRANCH/current"
- Another build step creates "manifest.json" with $CIRCLE_BRANCH and $CIRCLE_BUILD_NUM
- The vhost config and the manifest are added to the release tarball/Docker image
- The tarball/image gets shipped to the staging server
- (we ship to S3/Docker, then pull because of security concerns. But can ship inside the build process)
- In staging, a script reads manifest.json and moves content to /var/www/$CIRCLE_BRANCH/$CIRCLE_BRANCH
- The provided nginx vhost is copied into a standard dir with other vhosts
- A symlink of "/var/www/$CIRCLE_BRANCH/current" is forced to point to the fresh release
- nginx -s reload && profit
You can use a wildcard SSL to cover subdomains or add a step to trigger LetsEncrypt certbot as required.
This method works equally well with Docker images - just requires an interim step to launch the container. In staging, we use a naming convention to launch on predictable ports for every app. First 2 numbers are app specific, second 3 numbers are build-specific. For example, "25$CIRCLE_BUILD_NUM" (where $CIRCLE_BUILD_NUM is just the last 3 numbers of the build).
- vmasto 9y agoHow do you handle the database? Do you reuse the one on staging or create a new instance with prepopulated data? If the former how do you deal with migrations and schema edits?
- e1g 9y agoYes, the db has been the most challenging aspect for us. We have 3 situations - 1. "Common baseline". With a relatively stable product, most branches (as in ~51%) do not impact the schema. For testing / QA purposes, these share one central QA db and pollute each out. Turns out, a lot of the times this is quite ok because the PR is about how the data is displayed, or improved logging, or UX change, or a security layer or anything else other than core domain knowledge - they don't care for the data that much. 2. "I'm special". Some branches do modify data (whether the format or the structure). To handle these, the manifest.json file has an option to request a separate database. If present, the rollout script will do "pg_dump + copy" of the shared staging DB, and duplicate it into "qa_$BRANCH", then update the config file (or .env for Docker) with the appropriate connection value. Additionally, it will all *sql files in a dir specified in manifest.json against the clone DB. This is done on every release, which does get annoying by resetting the qa data (we could add another manifest switch here I). On the upside, it forces you to codify all data migration rules from the start. 3. "I am very special". Some changes transform data in a way that requires business processing and cannot be done with easy SQL. Sorry, out of luck - we don't automate special cases yet. The developer has to pull the QA database to localhost, do his magic, and push it back. Not ideal, but hasn't caused any problems yet. If ain't broke...
- vmasto 9y agoThanks for taking the time to answer! FWIW this topic would make for a great technical post/how-to. I also seem to recall that Automattic does this with their front end (calypso) which handles wordpress.com
- sejiss 9y agoWe have a job every morning that clones the current production db instance and then deploys it onto a staging instance. All PRs with schema changes will be applied to this instance when merged. If something goes wrong, we just redeploy the saved image.