The four charts that survived after we deleted twenty others. A test suite is only valuable if the team trusts it. The moment people start re-running failures without reading them, the suite has stopped protecting anyone.
What the dashboards show
Reliable tests control everything outside the code under test: time, randomness, network, and fonts. Once those are fixed, a failure means something changed in the code, which is exactly what we want to know.
We have been running this approach for several months now. The initial setup took about a day, and the ongoing cost is close to zero once the team knows where to look.
None of this is new, and that is the point. The techniques that last are usually the boring ones that are documented well and supported by the platform itself.
As always, the examples are simplified to fit on a page. The production version has more error handling and logging, but the shape of the code is the same.
Questions, corrections, and war stories are welcome in the comments. Several of our best follow-up posts started as a reader pointing out something we had missed.
We will revisit this in six months with fresh numbers. If the results change, we will update the post and note what changed at the end.
What we would change next time
Questions, corrections, and war stories are welcome in the comments. Several of our best follow-up posts started as a reader pointing out something we had missed.
- Freeze time and seed randomness in tests.
- Quarantine flaky tests with an owner and a deadline.
- Keep end-to-end tests for critical journeys.
- Read every failure before re-running.
Any thoughts on how this interacts with server-side rendering?
Our accessibility audit flagged exactly this last month. I wish I had read this first.
Would love a follow-up on how you test this in CI.
It should work without shell access, you just lose the command-line shortcuts.
Thanks for including the numbers. So many posts skip the measurement part.
Any thoughts on how this interacts with server-side rendering?