LEARN. PRACTICE. GROW.Three moments in an operation
Separate request reception, effect commitment and consumer receipt of the response. The lab uses two simultaneous HTTP listeners in one Python process: a teaching gateway and a dependency. The dependency records the effect in SQLite and only then holds its response on an Event. The gateway waits through http.client and returns 504 when that local limit expires. The script observes committed already set while done is not. The result therefore is not an invented account of what might happen: there is actual loopback traffic and a queryable local effect. The boundary remains limited to this process, these threads and this temporary database.
What the error does not decide
The semantics of 504 describe a gateway that did not receive a timely response from a needed dependency. They do not establish cancellation or rollback of the operation. On the first lab path, the client receives outcome-unknown while a five-unit effect already exists. Closing the gateway outbound connection does not delete that row. In real systems, timeout may occur elsewhere and produce another state; do not generalize this observation into claiming every 504 contains an applied effect. Keep uncertainty explicit until the relevant contract and evidence are consulted. A transport status is a useful observation but insufficient on its own to decide business state.
The retry that duplicates
Endpoint /unsafe inserts one effect per call even when client and operation repeat. After the first timeout, the script repeats the same parameters while the original response remains held. The second call returns 200 and the database contains two effects totaling ten units. Final success does not mean only the last call changed state. This example supports discussion of a common runbook error: retry whenever the dashboard shows failure. Before turning that instruction into automation, define which outcomes are retryable, using which identity, for how long and up to what limit. The lab implements no automatic retry loop, backoff or jitter.
Recover the same intent
On /dedupe, the transaction inserts the effect and stores its response in a record identified by client and operation. The first response can also time out at the gateway. A local query retrieves state=applied and effectId; an equivalent retry returns that same identifier without adding an effect. This occurs before releasing the original held response. The script demonstrates recovery of one local intent, not universal single delivery. A new key with the same parameters represents another intent under this contract and creates another effect. Do not derive identity solely from requested quantity: two legitimate operations can have equal parameters. Lab client labels are fictional and unauthenticated.
Decide during a batch window
In a fictional APS case, a reservation precedes the next batch step. The operator sees 504 and proposes another key to unblock the window. The manager first requests original identity and dependency state. If a correlated query confirms the effect, recover its result under the contract; if outcome remains unknown, retain the operation under reconciliation and escalate. A query returning no result in another product may have delays or guarantees different from this local database. Do not assign certainty the contract does not provide. Retain earlier attempts and avoid deleting effects to make the database match the dashboard. Roles and units are fictional examples and represent no real bank procedures.
Workshop and synthesis
Prepare three cards for development, QA and RUN: timeout before sufficient information exists, timeout with a committed effect and a retry that already duplicated the effect. Ask participants to distinguish what they know, what is missing and who decides the next step. Add a constraint that identity cannot be changed merely to obtain a green result. Each group should produce a note containing state, evidence, impact and proposed recovery. This human workshop was prepared but not performed. In the automated exercise, sockets are owned and effects are rows in a temporary database. Summarize the distinction among attempt, operation, effect and response, linking it to earlier outbox, inbox and reconciliation lessons.
python3 content/labs/micro-timeouts/run.py --output /tmp/dr-micro-timeouts-first.json
# Choose a new output path. Requires owned loopback listeners.
# Actual path: client -> teaching gateway -> dependency -> local SQLite.
# timeout-after-unsafe-commit: 504; effects=1, units=5
# unsafe-retry-duplicates-effect: 200; effects=2, units=10
# Response is held AFTER commit by an Event, not by delaying the transaction.
# A timeout ends the gateway wait; it does not undo the committed effect.A fictional gateway returns 504 after its dependency records five units. Retrying without deduplication creates another effect and raises the total to ten.
Common pitfalls
Treating 504 as rollback, using a new key per attempt, treating connection closure as cancellation or local timeout as a guaranteed global deadline.
Related topics: Local transactions · Resilience and load · Operational diagnosis
A missing response can coexist with a committed effect. Recovery depends on identity, state and contract as well as the observed HTTP status.
Reference: RFC9110: 504 Gateway Timeout · Microservice architecture patterns and scoped platform examples; primary guidance consulted 2026-09-30