Receipts, idempotency, and transaction boundaries | SFTP: transfers and batch operations

Accept bytes and identify intent

The original consumer receives an identifier, file, expected hash, and failure mode. It reads the file once into memory and calculates the hash over those same bytes. It starts the transaction only when comparison succeeds. This single read avoids validating one byte sequence while calculating the synthetic effect from another, but it neither locks out external changes during reading nor provides streaming for arbitrarily large files. The expected reference comes from the experiment itself, without a signature or external manifest validation. The identifier represents delivery intent. Two periods can legitimately produce identical bytes and require two processing operations; a retry of the same delivery should retain its identity. The lab demonstrates both situations without imposing a universal identity model. A real contract must also define sender, period, and uniqueness scope.

Fail before and after COMMIT

The fixture opens BEGIN IMMEDIATE and writes two rows: a receipt with identity and digest, and a synthetic effect containing the byte count. Both belong to the same SQLite transaction. In before-commit mode, the subprocess exits through os._exit after the INSERT statements and before COMMIT; a fresh connection observes both tables empty. In after-commit mode, the subprocess exits after COMMIT and before printing acknowledgement; the fresh read finds the receipt and effect. Missing acknowledgement alone cannot distinguish these states. The operator therefore consults persisted evidence using the original identifier. The experiment uses process death rather than a power cut. Its conclusion is limited to observed recovery with this SQLite version and local storage; it does not establish durability for a financial platform or distributed cluster.

Recover retries and reject conflicts

When the same identity and digest are retried, the consumer finds the receipt, ends the read transaction, and returns duplicate with the earlier reference. Both tables retain exactly the same rows. If the identity already exists with a different digest, the reply is identity-conflict even when the new file matches its own expected hash. Integrity and identity are different conditions. A business correction needs an explicit procedure linking the new intent to the previous one. INSERT OR REPLACE is not a neutral solution: for certain uniqueness conflicts, SQLite removes the existing row before inserting its replacement. That can erase the association needed to reconcile the first effect. The sixth subprocess uses a new identifier with identical bytes and creates another effect, demonstrating why changing a name or identity is not a safe retry.

Bound the operational contract

The demonstrated effect is only a SQLite row in the same transaction as the receipt. There is no financial posting, external API, or payment system. If a consumer calls another service between INSERT and COMMIT, a failure can leave an external effect without a local receipt. An outbox design can coordinate persisted intent and later dispatch, but still requires suitable recovery and deduplication at the destination; that design was not executed here. Also define how long receipts survive and how retries arriving after retention are handled. The guide below lasts forty minutes and ends with a written explanation of the limits. Its published code was executed automatically, but the guide has not yet been performed with participants. Use result comparison to distinguish evidence from a hypothesis requiring another authorized experiment.

40-MINUTE GUIDE
0–8: Save the previous lesson's complete code as run.py. Use an authorized local working area, an unprivileged account, and seven executables from the same build. The executed reference was OpenSSH 10.5p1 without PAM, OpenSSL 3.6.1, Python 3.13.1, and SQLite 3.51.2. Replace /path/openssh with local paths.
python3 run.py \
 --ssh /path/openssh/ssh \
 --sshd /path/openssh/sshd \
 --sshd-session /path/openssh/sshd-session \
 --sshd-auth /path/openssh/sshd-auth \
 --keygen /path/openssh/ssh-keygen \
 --sftp /path/openssh/sftp \
 --sftp-server /path/openssh/sftp-server \
 --output./recovery-evidence.json
8–18: Read checks and observations. Confirm five SFTP sessions, six workers, and thirty checks. Explain why partialBytes may vary and why wrongPrefixResume exits with zero despite a mismatched hash.
18–30: Compare worker states for integrity-rejection, failure-before-commit, failure-after-commit, retry-same-identity, identity-content-conflict, and new-identity-same-bytes. Connect each exit to persisted evidence. Do not reuse the output file if you want to retain earlier runs.
30–40: Write a lost-acknowledgement runbook: identity, status query, retry condition, conflict, escalation, and retention. Separately list external networking, concurrency, power failure, external effects, and expired retention, which this experiment does not execute. Confirm temporary-file and daemon cleanup in the report. Do not use real data or credentials.

IN PRACTICE

An acknowledgement is lost after the fictional consumer commits. Reusing the original identifier returns its existing receipt; inventing a new identifier creates a second effect.

Common pitfalls

Globally deduplicating by hash, using INSERT OR REPLACE to conceal conflicts, or assuming that a SQLite transaction includes an external call.

Related topics: Transactions and receipts · Retry and business intent

Take this idea with you

Recover the same intent through stable identity and persisted evidence. Each external effect needs its own recovery contract.

Create account

Reference: SQLite transaction boundaries · BigSavant SFTP 2026-09; selected OpenSSH client, server and extension behavior