LEARN. PRACTICE. GROW.Bind the decision to the image being deployed
A release should not be approved merely because the dashboard shows the last run as green. Identify the candidate image, digest, code source, build process and current criteria. In a fictional APS case, the candidate tag moved after tests. The positive report describes A, but deployment intends B. The shared name does not establish identical content and dependencies. This case does not describe internal BNP Paribas procedures, and its local evidence rules are teaching assumptions. Provenance helps connect an artifact to how it was built. Validation should verify the statement and compare source and builder with the trust policy rather than accepting every valid signature. In Cloud Build, review the generation path: images-based publication and the verification option form part of the documented design. An explicit docker push step does not itself create the expected provenance. Distinguish metadata that was never generated from existing metadata that the observer cannot read; those failures need different remediation. Prepare a decision package with consistent identifiers: candidate digest, criteria revision, results, completion state and owners. If a team reruns tests, retain the link to the image actually used. Do not merely update labels in an old report. For the go-live meeting, frame the concrete question: what evidence demonstrates that this content meets these criteria? That exposes a gap without claiming the application is necessarily vulnerable or that the previous image was incorrect.
Require scan completion and coverage
A started scan is not yet a result. If state is pending, an empty findings list does not establish absence of problems. Define pipeline states that permit progression, block it or require investigation. In this lesson, the criterion requires a completed scan for the candidate digest and no blocking findings. A timeout remains missing completion evidence even if the preceding build step finished successfully. Keep the distinction visible to whoever must decide within the release window. Also confirm supported scope. Storing an image in a repository does not prove that a scanner analyzes every component. Artifact Analysis documentation bounds operating systems and package types and does not support Windows Server containers. The team needs to recognize what remained outside analysis and assign suitable coverage. An aggregate indicator should distinguish no findings, missing analysis, unsupported type and failed execution. Combining those situations into one green category hides decisions that matter for both risk and delivery. After deployment, conditions can change. Continuous validation with check-based platform policies, marked preview when sources were inspected, can log violations for images used by running Pods. A finding demonstrates detection, not that the workload was terminated or fixed. Define an owner, response deadline and closure evidence. The decision process continues after admission: observe the service, confirm the action's effect and record residual risk while a permanent correction remains unavailable. Neither a clean historical report nor a new alert completes that work by itself.
Distinguish simulation, enforcement and effective change
A policy can exist without blocking operations. In Binary Authorization, DRYRUN_AUDIT_LOG_ONLY lets teams observe violations without preventing the corresponding deployments. Successful deployment with a violation event can be exactly the configured behavior. Before treating it as a product failure, check the applicable rule and mode. Then compare the result with the requirement: if production requires blocking, an observation phase does not establish that the requirement is already fulfilled or that every deployment path uses enforcement. Organization Policy also distinguishes a live policy from dryRunSpec configuration. For supported constraints, dry run lets teams observe effects before applying a restriction. Do not generalize that support to every legacy constraint type. Change review should identify resource, inheritance, scope, mode and the creation or update operations affected. A file with the right intent does not prove that effective configuration was applied in the intended project, under the identity and method used in production. For a pipeline policy change, compare approved and observed state and investigate differences. A manual change might be an authorized intervention not yet reflected in automation, or an improper modification. The difference needs classification and an owner, not blind automatic restoration. Record the decision and reconcile configuration sources so the next run does not recreate divergence. At handover, demonstrate an allowed request and one that should be rejected in the correct scope, preserving the distinction between rehearsal and actual enforcement.
Coordinate patching with inventory and service
The denominator of a patching metric matters as much as its percentage. If asset inventory includes 40 VMs but only 35 are visible in VM Manager, do not declare the other five compliant. Its operating-system view depends on the OS Config agent and inventory management. Missing data can have several causes and should appear as an owned coverage gap. A healthy dashboard for the observed subset does not automatically represent the entire perimeter, even when all systems began from the same template. For the Saturday window, plan service dependencies: node sequence, compatibility, functional validation, observation and contingency. The operating-system patch mechanism does not itself know whether financial reconciliation remains correct. A service can start yet fail on a rare batch path. Define representative checks with the application team and identify who can decide to continue, stop or defer the next group. Use business and technical criteria concrete enough for the on-call team to apply without guessing the intended acceptance standard. If a patch job is cancelled, the agent attempts to finish its current task before stopping. Do not confuse the central request with instant stopping across machines or with rollback. Collect per-VM state and already applied changes before resuming traffic. Shift handover should identify systems in different states and pending actions. This prevents the next team repeating an intervention without knowing what remains installed or is still completing, and preserves a reliable basis for later recovery decisions.
Follow the event from source to destination
A logging strategy should explain the entire path: event occurrence, category generation, filters, routing, destination write, retention and authorized querying. A failure at any point can produce an empty dashboard, but corrections differ. In the fictional case, the analyst sees the event at source and the sink reports a write denial in the central project. Investigation should check writer identity and destination permissions rather than expand the analyst's access. Record the concrete error and resource being addressed. Within the same resource, independent sinks evaluate each entry separately. An exclusion on sink A does not automatically remove route B. When reducing exposure or cost, identify all relevant destinations and verify effects on each. This observation does not replace examining specific aggregated-sink and hierarchy interception rules. Draw the actual flow before applying conclusions from a simpler design, and assign ownership for both sides of a cross-project delivery dependency. For Data Access generation, also check exempted identities. The category can be enabled while an exemption applies to the application's service account. That explains a collection gap distinct from lacking query permission. Record configuration valid at the event time and avoid inferring that no read occurred merely because no record is found. To accept SIEM integration, use an identified synthetic event, observe its arrival and confirm expected destination handling without unnecessary sensitive data. This demonstrates one intended path rather than universal delivery under every failure condition.
Provide useful log access without losing scope
Centralizing logs makes investigation easier but also concentrates information from several applications. A filtered view can grant subset access; it represents effective isolation only when other grants do not permit more. Unconditional project-level roles/logging.viewAccessor has broader scope than a grant to one specific view. Adding a restricted view does not remove an existing broad role. Review needs to consider the user's effective query paths, including permissions inherited or granted independently of the new view. Define what each team needs to query and why. An APS analyst might need application errors and a correlation identifier without full payloads from other teams. Use synthetic examples to check allowed and denied queries. Keep log access, sink administration and retention-change capability separate in the responsibility model. Operational investigation should remain possible without giving every operator full control over evidence that a later audit might require. Record how an exceptional investigation obtains any additional approved scope. Locking a log bucket and its retention is irreversible. Before that decision, confirm the approved period, scope and operational impact. It is not an experiment undone the next day by changing a description or deleting a bucket with entries still within retention. Include owners for querying, retention and delivery-failure response in the plan. A written policy needs a sustainable operating process to remain useful after the project ends and its implementation team hands responsibility to production.
Turn detections into traceable operational work
An installed tool does not establish detection coverage. A Cloud IDS endpoint needs Packet Mirroring policies delivering relevant traffic. Endpoint existence does not show that intended VMs are covered. Similarly, secondary sampling rate=1.0 in VPC Flow Logs retains records produced by the earlier stage; it does not remove primary sampling or create capture of every packet. State what the observation supports and what remains outside scope before promising forensic detail that the system does not collect. During triage, check whether presentation settings hide results the team expects to review. In SCC, a matching static mute rule takes precedence over a dynamic mute rule. Dynamic expiry might not produce the expected effect while the static rule remains applicable. Exception review should examine the complete rule set, owners and operational rationale rather than only the date displayed on one isolated rule. Assign a follow-up action where the intended expiry and effective behavior differ. When automating ticket creation, distinguish a message from a business event. A producer can publish the same occurrence twice with different IDs even when the consumer uses exactly-once delivery. Define a stable event-appropriate key and idempotent behavior to avoid duplicate work without losing different occurrences. Do not use “first alert of the day” as a universal key. Production handover should include repetition, acknowledgment failures and unavailable destinations, with evidence that operational response remains coherent in those situations.
Exercise: assess evidence without inventing approval
The local exercise receives a candidate and fictional reports. It requires exact digest and policy-revision matches plus recent results for every declared check. It verifies no signatures, downloads no images and runs no scanners. Its result describes only consistency of supplied assertions. Digests are synthetic and timestamps are relative numbers to make boundary behavior easy to observe. This is a teaching model, not a substitute for a real release gate or trusted evidence collection. Before running python3 run.py, predict three situations. With two current positive checks for the candidate, local criteria are satisfied. If the scan belongs to another digest, it does not cover the candidate. If the scan remains pending while integration passed, the conclusion stays incomplete. The program also preserves known failures while another check is unresolved. A current negative result does not silently disappear because another positive report exists; the conflict needs investigation and explicit reconciliation rather than selection of the preferred result. Try the age boundary: age 10 passes with max_age=10; age 11 remains unresolved. A future or missing timestamp also fails to establish freshness. Reversing report order should not change the decision. Then write the committee note with artifact, criteria, gap and next action. Even criteriaSatisfied=true does not authorize production. Summarize the lesson by distinguishing identity, completion, coverage, enforcement and response: each dimension needs its own evidence linked to the service and change being evaluated.
"""Offline release-evidence exercise; not a scanner or signature verifier.
Only exact digest and policy revision match. Freshness is inclusive.
All required checks need positive, current evidence. Applicable current failures
remain visible even when other checks are unknown. Conflicting current reports
withhold satisfaction; a later pass does not silently erase another current fail.
"""
from copy import deepcopy
from hashlib import sha256
from itertools import permutations, product
from math import isfinite
from pathlib import Path
import json
import re
def text(value):
if not isinstance(value, str) or not value.strip:
raise ValueError('expected nonempty string')
return value
def digest(value):
if not isinstance(value, str) or not re.fullmatch(r'sha256:[0-9a-f]{64}', value):
raise ValueError('expected canonical fictional SHA256 digest')
return value
def number(value):
if type(value) not in (int, float) or not isfinite(value):
raise ValueError('expected finite number')
return value
def assess(candidate, reports, now=100, max_age=10):
number(now)
number(max_age)
if max_age < 0 or not isinstance(candidate, dict) or not isinstance(reports, list):
raise ValueError('invalid candidate, reports or age')
expected = digest(candidate.get('digest'))
revision = text(candidate.get('policyRevision'))
required = candidate.get('requiredChecks')
if not isinstance(required, list) or not required:
raise ValueError('nonempty required checks needed')
for kind in required:
text(kind)
if len(set(required))!= len(required):
raise ValueError('duplicate required check')
seen, ignored, grouped = set, [], {kind: [] for kind in required}
for report in reports:
if not isinstance(report, dict):
raise ValueError('invalid report')
identifier = text(report.get('id'))
if identifier in seen:
raise ValueError('duplicate report ID')
seen.add(identifier)
kind = text(report.get('check'))
result = report.get('passed')
if result is not None and type(result) is not bool:
raise ValueError('passed must be bool or None')
at = report.get('checkedAt')
if at is not None:
number(at)
mismatch = []
if digest(report.get('digest'))!= expected:
mismatch.append('digest')
if text(report.get('policyRevision'))!= revision:
mismatch.append('policyRevision')
if kind not in grouped:
mismatch.append('check-not-required')
if mismatch:
ignored.append(dict(id=identifier, reasons=mismatch))
else:
grouped[kind].append(report)
checks = []
for kind in sorted(required):
passed, failed, unresolved = [], [], []
for report in grouped[kind]:
at = report.get('checkedAt')
if at is None or at > now or now - at > max_age or report.get('passed') is None:
unresolved.append(report['id'])
elif report['passed']:
passed.append(report['id'])
else:
failed.append(report['id'])
status = 'failed' if failed else ('unresolved' if unresolved or not passed else 'satisfied')
checks.append(dict(check=kind, status=status, passes=sorted(passed),
failures=sorted(failed), unresolved=sorted(unresolved)))
return dict(criteriaSatisfied=all(c['status'] == 'satisfied' for c in checks),
checks=checks, ignored=sorted(ignored, key=lambda r: r['id']),
signaturesVerified=False, artifactScanned=False, productionAuthorized=False)
def main:
a, b = 'sha256:' + 'a' * 64, 'sha256:' + 'b' * 64
candidate = dict(digest=a, policyRevision='r8', requiredChecks=['scan', 'integration'])
scan = dict(id='scan-1', digest=a, policyRevision='r8', check='scan', passed=True, checkedAt=100)
integration = dict(scan, id='integration-1', check='integration')
fixtures = []
def case(name, reports, expected, **kwargs):
before = deepcopy((candidate, reports))
result = assess(candidate, reports, **kwargs)
assert result['criteriaSatisfied'] is expected, name
assert (candidate, reports) == before
fixtures.append(dict(id=name, **result))
return result
case('matching-evidence', [scan, integration], True)
case('different-digest', [dict(scan, digest=b), integration], False)
case('different-policy', [dict(scan, policyRevision='r7'), integration], False)
case('pending-scan', [dict(scan, passed=None), integration], False)
case('missing-check', [integration], False)
case('stale-report', [dict(scan, checkedAt=89), integration], False)
case('freshness-boundary', [dict(scan, checkedAt=90), integration], True)
case('future-report', [dict(scan, checkedAt=101), integration], False)
case('known-failure-and-unknown', [dict(scan, passed=False), dict(integration, passed=None)], False)
case('conflicting-current-reports', [scan, dict(scan, id='scan-2', passed=False), integration], False)
case('irrelevant-report', [scan, integration, dict(scan, id='scan-old-artifact', digest=b, passed=False)], True)
case('missing-timestamp', [dict(scan, checkedAt=None), integration], False)
combinations = 0
for scan_result, integration_result, timestamp, matching_digest, matching_policy in product(
(False, True, None), (False, True, None), (89, 90, 100, 101), (False, True), (False, True)):
s = dict(scan, passed=scan_result, checkedAt=timestamp, digest=a if matching_digest else b,
policyRevision='r8' if matching_policy else 'r7')
i = dict(integration, passed=integration_result)
result = assess(candidate, [s, i])
expected = scan_result is True and integration_result is True and timestamp in (90, 100) and matching_digest and matching_policy
assert result['criteriaSatisfied'] == expected
if integration_result is False:
assert result['checks'][0]['status'] == 'failed'
combinations += 1
reports = [scan, integration, dict(scan, id='scan-2', passed=False)]
expected = assess(candidate, reports)
permutation_count = 0
for order in permutations(reports):
assert assess(candidate, list(order)) == expected
permutation_count += 1
invalid = [
(dict(candidate, digest='latest'), [scan], {}),
(dict(candidate, policyRevision=''), [scan], {}),
(dict(candidate, requiredChecks=[]), [scan], {}),
(dict(candidate, requiredChecks=['scan', 'scan']), [scan], {}),
(dict(candidate, requiredChecks='scan'), [scan], {}),
(candidate, [scan, scan], {}),
(candidate, [dict(scan, passed=1)], {}),
(candidate, [dict(scan, passed='true')], {}),
(candidate, [dict(scan, checkedAt=True)], {}),
(candidate, [dict(scan, checkedAt=float('nan'))], {}),
(candidate, [dict(scan, digest='sha256:bad')], {}),
(candidate, [dict(scan, id='')], {}),
(candidate, [dict(scan, check=None)], {}),
(candidate, [None], {}),
(candidate, [scan], dict(max_age=-1)),
(candidate, [scan], dict(now=float('inf'))),
]
for c, r, kwargs in invalid:
try:
assess(c, r, **kwargs)
except ValueError:
pass
else:
raise AssertionError('invalid input accepted')
print(json.dumps(dict(labId='pcse-release-evidence', fixtures=fixtures,
stateCombinations=combinations, inputPermutations=permutation_count,
invalidInputs=len(invalid), inputPreserved=True, orderIndependent=True,
network=False, cloudExecuted=False, persistentWrites=False,
scriptSha256=sha256(Path(__file__).read_bytes).hexdigest), indent=2))
if __name__ == '__main__':
main
The release candidate changed digest, scanning is pending and the central sink cannot write; a green dashboard does not resolve those three gaps.
Common pitfalls
Confusing tag with digest, pending scanning with no findings, simulation with blocking and a received event with completed response.
Related topics: CI/CD and software supply chain · Change management and production handover · Observability and incident response
Every approval should identify the artifact, criterion, scope and state actually observed.
Reference: Generate and validate build provenance · Current linked guide; edition date unconfirmed (2026-09-30 inspection)