LEARN. PRACTICE. GROW.Define the expected set
A batch can only be reconciled when the expected content for that cycle is known. In the exercise, the manifest requires events a, b, and c, totaling 600 units. The incorrect file contains a:100, b:200, and b:300. Three-row count and 600-unit sum pass, but only two distinct identities exist. The laboratory reads the CSV and calculates these controls; it does not accept the set. Before writing, also confirm structure and types required by the contract. Do not rename the second b to c without producer evidence. An invented correction can preserve totals while still representing the wrong event.
Separate identity, content, and attempt
The local contract identifies a batch by source, business date, and batch_id. It also retains a digest of associated content. First application inserts three events and one receipt in the same transaction; repeating the same identity and intent retains three events and 600 units. Changing filename or operator does not necessarily create another business operation. Conversely, the same batch-7 label on another date can identify a different cycle under this contract. Always record the scope used. This exercise design does not establish a universal key rule for external services, whose semantics and retention window must be confirmed.
Compare exit before and after commit
The laboratory creates two disposable databases and deliberately exits a child process with code 17 in each. In the first, exit occurs after writes but before COMMIT; reopening shows zero events and zero receipts. In the second, exit occurs after COMMIT and before any caller acknowledgement; reopening shows three events, 600 units, and one receipt. The same exit code accompanies different persistent states. Recovery of the first applies the batch; recovery of the second recognizes replay without another effect. Local process exits were tested, not power loss, disk failure, networking, or distributed recovery.
Handle intent conflict and partial failure
A replay with the same key changes the first row from 100 to 999 units. Code compares content against the receipt and rejects identity-conflict, preserving the earlier result. In another experiment, a new batch inserts d and then conflicts with existing identity a. Handling explicitly rolls back the transaction, including the new receipt and d. Only the three earlier events remain. Support should assume neither that the prefix persisted nor that every SQL error automatically reverses everything. Confirm the transactional boundary and implemented handling. Replay must start from that state with reconciled intent, without changing identifiers merely to pass a constraint.
Communicate the limit of known outcome
A receipt in this database demonstrates the local effect defined by the exercise. It does not confirm that another system received, accepted, or published output. During handover, separately record cycle, observed commit, confirmed consumers, and still-unknown states. If an external service’s key-retention window expired, do not promise that resending the same key still prevents duplication. Request reconciliation through the applicable procedure. For correction after commit, evaluate an appropriate compensating operation; ROLLBACK on a new connection does not automatically undo the previous transaction. Keep the decision and cutoff impact visible to the service owner.
Produce a verifiable recovery plan
Finish the workshop with a plan another authorized operator can evaluate. Identify source, date, batch, content, persistent outcome, replay restrictions, and final-validation evidence. Classify each statement as observed, inferred, or still unknown. The laboratory retains versions and script hash; it uses temporary files and does not touch the BigSavant application database. The recorded execution used CPython 3.13.1 and SQLite 3.53.4 on Darwin. There was no scheduler, external transfer, or actual banking data. The authored plan is an original exercise requiring adaptation to actual contracts and mechanisms before use in a production operation.
python3 content/labs/aps-evidence/run.py
# Uses temporary SQLite databases only; no network or application database
# The recorded run used CPython 3.13.1 with SQLite 3.53.4
# Two disposable child processes deliberately exit with status 17
# One exits before COMMIT and the other after COMMIT
# Reopened state and replay outcome are checked explicitly.A fictional batch presents three rows and 600 units but only two distinct identities. Another exits after commit without acknowledging the caller.
Common pitfalls
Accepting totals without identities, changing a key to bypass conflict, or treating a nonzero exit code as proof that no commit occurred.
Related topics: SQL · SFTP · L3 Support
Reconcile intent against persistent state. Valid replay retains identity and parameters; external effects require their own evidence.
Reference: Data Processing Pipelines · BigSavant APS professional curriculum 2026-09; vendor-neutral operational guidance reviewed 2026-09-30