A narrated recording of traffic shifting between two identical environments. The best deploys are the ones nobody notices. Everything in this post exists to make releases small, reversible, and dull enough that we can ship on a Friday afternoon.
Sample footage: Big Buck Bunny, an open movie by the Blender Foundation, released under a Creative Commons licence.
Keep production boring
Our pipeline runs the same steps locally and in CI: install, lint, test, build, and publish an image tagged with the commit hash. If a step cannot run on a laptop, it does not belong in the pipeline.
- Tag images with the commit hash.
- Make every deploy reversible in one command.
- Alert on symptoms, not causes.
- Review infrastructure plans like code.
We watch four signals for every service: request rate, error rate, latency at the 95th percentile, and saturation. Anything else goes on a secondary dashboard that we look at when those four disagree.
None of this is new, and that is the point. The techniques that last are usually the boring ones that are documented well and supported by the platform itself.
As always, the examples are simplified to fit on a page. The production version has more error handling and logging, but the shape of the code is the same.
Questions, corrections, and war stories are welcome in the comments. Several of our best follow-up posts started as a reader pointing out something we had missed.
Simplicity is prerequisite for reliability.
Edsger W. Dijkstra, EWD498, 1975
Counterpoint: for small teams the simpler approach in the first section is usually enough.
It should work without shell access, you just lose the command-line shortcuts.
The renaming spree is a rite of passage.
Shared this with my team. The list near the end is going straight into our pull request template.