← CI/CD: build, validate, and deliver
06 / 6 · 40 MIN

Operation and recovery

Diagnose delays and failures without duplicating effects or losing execution control.

Concept and mechanism

A pipeline is also an operational service. It needs observation, owners, time limits, and known interruption behavior. Separate queue time from execution time. In an original calculation, nine queue minutes, four build minutes, seven test minutes, and three deployment minutes total 23 minutes, of which 14 are execution. Optimizing compilation does not directly remove the nine waiting minutes. Analyze the critical path and runner availability before adding parallelism. When a command fails, distinguish transient unavailability, invalid input, and persistent defect. Retry needs a condition and bound, and the repeated body must be safe with respect to effects already produced.

Guided application

In Jenkins, retry runs the block again under defined conditions; it does not automatically undo an external call that already took effect. timeout interrupts through an exception unless explicitly handled and is not a service recovery plan either. In a fictional scenario, a migration executed half its steps before a run was canceled by a newer one. Inspect and reconcile state before repeating. In GitHub, concurrency controls execution by group; cancel-in-progress may cancel the active run without reversing external writes. Current documentation distinguishes the single queue from the max option; waiting order should not be confused with commit order. Retain logs, state, and artifact reference so the next team can recover with context.

IN PRACTICE

Canceling a job after a write does not erase that write.

Common pitfalls

Total time as test duration; retry without idempotency; cancellation as rollback; timeout without recovery.

Related topics: Integration and delivery · Jobs and dependencies · Artifacts and evidence

Take this idea with you

Control execution and recover actual state rather than only the state shown by the pipeline.

Create account

Reference: Bounded retry timeout and stash lifetime · CI/CD practices 2026-09; scoped GitHub Actions GitLab Jenkins and Azure DevOps Services documentation