Identity and parameters
The lab composite key is client plus operation; quantity is stored as part of intent. Repeating the same key with five units retrieves the earlier effect, but sending seven produces 409 and retains existing state. Changing textual JSON field order does not change parsed values and still retrieves the same effectId. These behaviors belong to the presented teaching contract; they are not automatic guarantees of every HTTP API. In a real service, decide which parameters define equivalent intent and which changes require a separate operation. Bind client scope to authenticated identity and authorize queries too. Receiving client in JSON, as this script does, does not provide that security.
Commit the effect and intent record
The endpoint opens one SQLite connection per request and executes a short transaction. The effect and recoverable response are recorded in the same unit; HTTP response suspension occurs only afterward. Within this database, that choice avoids treating both writes as independent steps. It does not extend the transaction to the consumer or external services. If an evolution introduces an external instruction before recording ledger, another failure boundary appears: the supplier can apply the instruction without a local record existing. Identity, queries, durable state and recovery of that flow need review. A local BEGIN IMMEDIATE does not turn HTTP calls into distributed-transaction participants.
Restart and retention
The script closes both listeners, waits for them to stop and opens new ones in the same process while retaining SQLite. A retry finds the record and returns the same response. That observation demonstrates continuity through a clean listener restart without proving crash recovery, process restart or power-loss recovery. The script then deliberately removes one ledger row while retaining its corresponding effect. A late retry is no longer recognized and creates another effect. No automatic TTL is implemented; controlled deletion supports discussion of the timing contract. Retention, request validity and late-arrival handling must be consistent with the period during which consumers may still retry.
Recovery does not eliminate load
Protected retries still open connections, execute handlers and query the database. Preventing duplicate effects does not make unlimited attempts free. Assign an owner for retry policy and check each layer limits to avoid inadvertently multiplying calls. The gateway 0.25-second read timeout is a local exercise control, not a configuration recommendation or guaranteed global deadline. No benchmark, statistical latency analysis or jittered backoff was executed. If an operation finishes after several attempts, retain indicators of failures and delay experienced by the client. Report attempts, operation completion, latency and duplicate effects separately to guide improvement. Successful recovery can preserve integrity while the service still provides a degraded experience.
Changes to the contract
A fictional team wants to accept retries for twenty-four hours and clean records after one hour. The issue is not choosing a larger number by habit; it is resolving the interval during which the client still considers an intent valid after the server has forgotten it. Longer retention, bounded validity or explicit rejection of late arrivals may be needed, with compatible change and accepted impact. If duplication already occurred, changing policy prevents new occurrences but does not reconcile earlier effects. Apply the same discipline to parameter changes, client scope and migration to messaging. A broker has its own acknowledgements and redelivery rules; the HTTP lab does not validate that product.
Handover workshop and summary
Assign supplier, development and RUN roles. Present a lost response, a quantity conflict and a late request after record removal. The group should write how it queries state, when it retries, when it stops and who authorizes a different operation. Add a dashboard showing many 504 responses but one effect per operation: the team should retain the experience and load problem despite functional recovery. The human workshop remains unperformed. The automated lab passed thirteen groups twice, closed four listeners across its lifecycle and removed the temporary directory. Summarize the contract through scope, intent, local commitment, retention, observability and recovery while preserving security and distribution limitations.
python3 content/labs/micro-timeouts/run.py --output /tmp/dr-micro-timeouts-second.json
# same-key-different-intent-rejected: 409, original units remain 5
# equivalent-json-order-reuses-result: same effectId
# clean-listener-restart-retains-record: same response, same SQLite database
# removed-record-allows-late-duplicate: effects=2, units=10
# Record deletion is controlled; no automatic TTL is implemented.
# Listener restart is not OS-process restart, crash or power-loss testing.The same key with a different quantity is rejected; retrying after record removal creates another effect even though the intent is old.
Common pitfalls
Choosing retention only by storage, treating client labels as authentication, local transactions as distributed guarantees or idempotency as absence of load.
Related topics: Local transactions · Resilience and load · Operational diagnosis
An idempotency contract needs scoped identity, stable intent and sufficient memory for the admitted retry period.
Reference: Making retries safe with idempotent APIs · Microservice architecture patterns and scoped platform examples; primary guidance consulted 2026-09-30