← REST APIs: integrate applications and diagnose failures
10 / 12 · 60 MIN

Pagination, snapshots, and export recovery

Distinguish continuation from consistency and delivery through omission, revocation, and premature-checkpoint cases.

Define the set the business expects

An export may require current data as it reads or data as it existed at a cutoff instant. These requirements produce different criteria. Record population, filters, fields, version, destination, and completion condition before choosing a cursor. Finishing traversal does not establish that every item reached the destination. In a fictional case, the source enumerates one thousand IDs but the destination confirms 980. Reporting must show the twenty outstanding items and their causes even when every read request returned 200. Separate read progress from delivery progress.

Compare positions with identities

After reading [1,2], deleting 1 changes the collection to [2,3,4,5,6]. Requesting offset=2 now skips 2 and 3 and returns [4,5]. The laboratory demonstrates omission through actual HTTP requests to its fictional collection. A boundary id>2 with immutable-ID ordering avoids this particular shift. It does not guarantee a snapshot: if record 3 changes, the next page can show its new value. Timestamp-only ordering also needs a tie-breaker when several records share the same timestamp. State the total ordering explicitly in the contract.

Distinguish cutoff, snapshot, and duration

Fixing the largest ID from the first read bounds later entries above it but does not freeze other fields. To reproduce values at an instant, consider versioning or a materialized export with snapshot identity. The choice has retention, duration, capacity, and access costs. Separate HTTP reads do not automatically share one database transaction. The fixture retains a Python copy of records for comparison; this demonstrates an in-memory frozen view without implementing PostgreSQL isolation or persistence across restarts. Production design must establish those guarantees with appropriate evidence.

Preserve scope and authorization

An opaque cursor allows internal details to change without requiring consumers to interpret offsets or timestamps. Under the exercise contract, each token is bound to tenant, filter, and mode, expires against a synthetic clock, and cannot be edited by the client. Changing the filter starts another traversal. Current authorization is checked before reading stored pages. AIP-158 is provider design guidance, not a universal rule for every REST API. The fixture’s 400, 403, and 410 responses belong to its explicit contract and should not be generalized to other providers.

Save progress after confirmed delivery

If the next cursor is saved before applying the current page, failure can make resumption skip undelivered items. Reversing the order can permit replay after failure before progress is saved. Design must relate confirmation, checkpoint, and stable destination identity. Depending on the systems involved, use idempotent effects, reconciliation, or a transaction that actually covers the required scope. Do not declare exactly-once execution merely because the cursor is unique. An expired token requires following the restart contract and reconciling earlier work rather than silently accepting a partial result.

Forty-minute workshop and handover

In the first ten minutes, predict pages after deletion of 1. In the next ten, compare amount 30 with 999 under keyset and snapshot. Spend another ten reviewing revoked access, a changed filter, and an expired cursor. In the final ten, draw a failure between delivery and checkpoint and define evidence needed to resume. Submit the ID matrix, completion criteria, and escalation plan. The laboratory does not execute external destination writes, disk failure, or a distributed transaction; these portions remain design exercises requiring separate practical acceptance.

# Workshop: 40 minutes / Oficina: 40 minutos
# 00-10: offset deletion / eliminação e offset
# 10-20: live values versus frozen copy / valores atuais e cópia congelada
# 20-30: scope, revocation, expiry / âmbito, revogação, expiração
# 30-40: delivery checkpoint and handover / checkpoint de entrega e handover
# Run the previous lesson code; inspect evidence.json.
# Executa o código da aula anterior e analisa evidence.json.
# External writes and distributed transactions are design exercises only.
IN PRACTICE

The first page [1,2] does not make offset=2 a stable identity. After deleting 1, the next request can skip 3 without an HTTP error.

Common pitfalls

Stopping just because a page is short; treating a cursor as authorization; confusing keyset with snapshot; saving progress before confirmed delivery.

Related topics: Requests and outcomes · Caching and pagination · Concurrent changes and verifiable recovery

Take this idea with you

A complete export needs a defined population, coherent traversal, valid access, confirmed delivery, and demonstrable resumption.

Create account

Reference: AIP-158 Pagination · HTTP semantics RFC9110; OpenAPI3.2.1; selected primary standards and provider contracts consulted2026-09-30