3 ms·
I've hot patched prod - placed classes into the system class loader because we couldn't unpack and repack the war. I've edited PHP or Python or Ruby files direc
by solarengineer 6y ago
I've hot patched prod - placed classes into the system class loader because we couldn't unpack and repack the war. I've edited PHP or Python or Ruby files directly. I've stuck a dll from a dev machine onto a prod IIS instance and restarted it for changes to take effect.
I have felt it worth my time to:
- separate the installation from the configuration
- have uniform environments (where a test environment is the same as prod in terms of tech stack and configuration, not necessarily volume)
- have consistent automated deployment approaches
- have consistent automated configuration approaches
With such an investment of time and effort, it has helped me:
- construct a prod-like environment (tech stack + configuration) to test in with the assurance that the app will experience the same constraints in prod.
- provide a tested deployment and configuration approach that environment owners can use to change production.
- push/place my hot fix in the test environment, validate it via automated tests, and then push the same hot fix to prod.
This has helped me ensure that the time between code-commit to go-live on prod is under an hour (including gatekeeping approvals from the environment owners) in even regulated environments. (I'm working on a book on this and will share more in a few weeks)
Depending upon the organisation and the specific project's automated test suite maturity, sometimes testing in prod may be the only option. If you must use containers but wish to retain the flexibility to change the binaries, then consider mounting the binaries into the container from a file system and use the containers for containment.
However, you should strongly consider moving to a package-once push-anywhere process.
If you face a situation where an intermediate environment is broken or blocked and you must push the artefact to prod, then by all means do so manually - after all, you ought to be certifying artefacts and not environments. An automated process that doesn't permit any authorised manual over-ride is only getting in the way. Such a manual override should provide for the right audibility and traceability, though.
Having said all this, the ideal would be uniform environments, consistent deployments and configuration, package-once deploy anywhere, audibility and traceability.