LEARN. PRACTICE. GROW.Concept and mechanism
An event has an occurrence time and can be processed later. Event-time windows preserve the meaning of a report grouped by operation hour, but require handling delays. The watermark represents expected progress and helps classify late data; it does not eliminate the possibility of receiving additional elements from an earlier window. Define lateness, triggers, accumulation mode, and the destination update contract. If one cumulative emission is 100 and the next is 130, summing both incorrectly produces 230. The consumer must know whether it receives replacements, versions, or deltas. Business deadlines and runner semantics should be exercised together.
Guided application
In Dataflow, exactly-once results do not mean every transform runs only once. An external call can repeat after failure even when committed output is deduplicated. Use stable operation identity and destination idempotency, or an architecture reconciling effects. Repeated logs are execution evidence rather than sufficient proof of duplicate committed results. Preserve input until the necessary recovery guarantee exists; deleting a file inside a transform before commit can prevent replay. For performance, inspect key distribution. If one customer dominates aggregation while other workers are idle, adding workers may not help. Fanout and recombination are valid only when they preserve the operation and ordering requirements.
A duplicate external request can coexist with one committed output row.
Common pitfalls
Watermark as absolute completeness; output as external effect; more workers as a hot-key cure; accumulation as delta.
Related topics: Design, governance, and identity · Quality, migration, and cutover · Ingestion, publication, and CI/CD
Specify guarantees at each boundary and how consumers receive corrections.
Reference: Dataflow exactly-once processing · Current linked standard guide (document title v4.2); edition date unconfirmed (2026-09-30 inspection)