LEARN. PRACTICE. GROW.1. Identify who can execute changes
A fictional funds operations team is preparing an update to a pricing service. A supplier changes code, the platform team manages the trigger, and APS follows the production transition. This case does not describe internal BNP Paribas procedures. The objective is to connect a particular change to its execution identity and the evidence needed to accept the result. Start by drawing three relationships: who modifies the repository, who authorizes execution, and which account runs the build. Someone without console access can still influence cloud resources through executed code. For trigger-initiated builds, the account configured on the trigger takes precedence over the account in the build file. A user-managed build account also requires a supported logging destination: Cloud Logging or user-owned Cloud Storage. A bucket created by Google but owned by the user differs from a Google-owned bucket. Include those distinctions in the technical review. Propose an acceptance rehearsal: record the code revision, effective identity, log destination and expected permitted operation. Pull-request approval does not replace that observation. For GitHub triggers, Comment control restricts initiation; the reviewer still needs to examine content and execution privileges. In the change meeting, explain the consequence: approval of a different revision leaves the decision disconnected from the delivered artifact. Assign an owner to resolve any mismatch before requesting acceptance.
2. Define image analysis coverage and freshness
The pricing service uses a recovery image retained since the previous quarter. The team promotes a tag and presents an old report as evidence of a new scan. In Artifact Analysis, initial scanning follows the digest; changing a tag does not itself trigger another scan. Metadata updates depend on recent use. Results for images not pulled for over 30 days can be stale, and pulling an image does not demonstrate an immediate refresh. For a manifest list, documented selection of one image does not prove coverage of every platform. Build your own decision table with five columns: target platform, resolved digest, observed result time, acceptance criterion and owner. Suppose the recovery pool will use arm64 while the submitted report covers amd64. Adding the word approved to the tag does not resolve the gap. Evidence is missing for the artifact that the pool will execute. An absence of findings can be accurate within the available report's scope while remaining insufficient for the proposed decision. For practice, write a short message to the release owner. State the covered platform, the platform whose coverage remains unproven, and the next observation required. Keep exception authorization separate from the technical conclusion. If the deadline prevents obtaining current results, record the gap and request a decision from the appropriate owner. Schedule pressure cannot supply missing security evidence.
3. Close exceptions and confirm maintenance coverage
During urgent recovery, APS uses a Binary Authorization exception in a Pod manifest. The service responds again and the incident is closed. Review should still look for the breakglass label in the manifest used for future creations. Ticket state does not remove persistent configuration. The currently documented label is image-policy.k8s.io/break-glass; an older annotation remains supported but should not be presented as recommended configuration. Removing the exception requires coordinating workload lifecycle, restoring intended controls and observing appropriate admission. It does not mean that existing Pods are automatically terminated. In a separate maintenance activity, a patch report contains SUCCEEDED_REBOOT_REQUIRED and SKIPPED. The first leaves a reboot outstanding. The second can describe MIG members whose participation was not enabled for the job. Keeping these categories separate prevents a coverage claim based only on the absence of FAILED. For the fictional case, draw a maintenance closure sequence: confirm included members, identify outstanding actions, agree the reboot impact and observe the service afterward. If the strategy replaces instances from a corrected template, request evidence of the instances actually replaced. A new template does not demonstrate that every existing member uses it. Present two outcomes in the handover: the ability to serve requests and the state of the controls. Both are needed for informed acceptance, with unresolved actions assigned to named owners.
4. Reconstruct the log route and the missing interval
At 10:00 the team fixes a sink that failed to send logs to the central archive for an hour. New events arrive, but this does not demonstrate recovery of the earlier interval. A corrected sink does not automatically route history; check what remains in source storage and plan separate recovery where possible. Log Router temporary storage should not be treated as protection against configuration errors. In another incident, an intercepting aggregated sink on an ancestor changes routing to child sinks, retaining the documented origin _Required exception. Draw two timelines for the case: event arrival at the source service and observation at the central destination. Mark configuration time, the affected interval and the first evidence after correction. Keep three possibilities in the report: history recovered, history available but not yet recovered, and history whose availability is unproven. A later sample cannot distinguish these possibilities for the earlier interval. When investigating a project whose expected destination has no logs, begin with hierarchy and route before recreating filters. Then identify the destination writer and available errors. The security team might need evidence for a period the operational dashboard does not show. Agree the required interval, alternative source and reconciliation owner with that team. Do not infer completeness from a count that covers only the current destination. Record which observations support each conclusion and which gaps remain.
5. Distinguish missing data, silence and resolution
A query searches for jsonPayload.result!="ok". Some events lack the result field and do not appear in the output. In the Logging query language, that comparison on a missing payload field does not match the entry. Treat field presence and observed value as separate questions. In the exercise, divide a sample into three groups: a known favorable outcome, a known unfavorable outcome and a missing outcome. The third requires investigation of instrumentation or format; it should not automatically count as success. Apply the same care to Security Command Center. Muting a finding changes ordinary visibility but does not demonstrate resource remediation; muted findings remain queryable. In the toxic-combination workflow assumed already configured here, changing case priority does not change finding severity. These properties are not interchangeable. The example does not promise that integration in every edition or environment. Prepare a daily meeting note: what changed on the resource, what changed only in presentation, and what remains to be observed. If a mute rule reduces noise during maintenance, assign an owner to review its scope. Fewer alerts may be desirable without demonstrating less exposure. Propose two measures for the scenario: technical conditions corrected with supporting evidence, and silenced occurrences with a recorded justification. Explain why combining them into one resolution metric would hide outstanding work from the service owner.
6. Preserve progress between effect and acknowledgment
A consumer creates incident INC-73 for a security event and fails to acknowledge the message. On the next delivery, the incident should not be recreated merely because transport repeated delivery. The consumer needs to retain progress until acknowledgment succeeds and use that progress when acknowledgment fails. The inverse failure also exists: acknowledge first, then lose the process before saving the effect. A delivery guarantee does not automatically include the incident system's transaction. For this case, propose a logical identity consisting of environment, source and event identifier. Explicitly assume an immutable-content contract for each identity. Two deliveries with the same key and equivalent content can point to the same persisted incident. The same key with different content requires investigation of the contract; do not silently discard it as a retry. This proposal is a design exercise, not a Google-provided distributed transaction implementation. Dead-letter forwarding also has limits. Maximum delivery attempts are approximate and forwarding is best effort. The Pub/Sub service account requires permission to publish to the destination and acknowledge on the source subscription. A test under a human account does not verify those permissions. Define who investigates forwarded messages, which evidence allows another attempt and how completed effects will be protected against repetition. The design should address both lost work and duplicate work, while exposing uncertainty to the team responsible for recovery.
7. Guided exercise: audit an event ledger
Save the displayed code as run.py and run python3 run.py locally with Python 3.13. It uses only the standard library and synthetic data. Before reading the output, predict the normal case: delivery at minute 10, an effect recorded at minute 11 and successful acknowledgment at minute 12. The program compares declared records; it does not contact a broker, create incidents or verify actual persistence. The hash compares payload bytes without authenticating their origin. Inspect retry-reuses-effect. The first acknowledgment fails, but another equivalent delivery reuses INC-73. The old delivery remains listed without its own successful acknowledgment; this describes history and does not prove current backlog. Compare duplicate-effects, where the same identity has two effect identifiers. In repeated-evidence-same-effect, two records describe one effect and the program does not invent a second incident. Next inspect content-conflict and explain why an equal key alone cannot establish equivalence. Change the times to place acknowledgment before commit. The output identifies the absence of a demonstrated earlier commit. Equal times also leave ordering unproven because this model resolves time in minutes. Finally change a commit's environment or source: a record from another context cannot satisfy the observed delivery. Submit three missing pieces of evidence, the owner responsible for obtaining each and the decision each could change. Do not present the output as automatic authorization for reprocessing.
8. Decide and communicate the operational handover
Return to the fictional pricing service. The release passed its pipeline, but the recovery image has old evidence, a VM group awaits reboot and an archive hour still requires reconciliation. Do not use a single green status to conceal these differences. Build a table containing object, scope, revision, evidence, outstanding action, owner and deadline. The committee can then decide about availability, exposure and remaining effort without confusing execution results with operational acceptance. Practice an English shift handover in four sentences: the affected service, the demonstrated result, the uncertainty and the owner of the next action. For INC-73, an appropriate answer states that the incident is recorded, the initial acknowledgment failed and the new delivery must be reconciled with existing progress. An inadequate answer declares resolution merely because the message disappeared from a dashboard. This lesson connects security automation to objective 4.1 and collection, detection and response to objective 4.2 of the consulted guide. It does not cover the entire domain by itself. Use the questions to justify decisions and explain alternatives rather than memorize status names. The local exercise does not approve production: it does not verify record integrity, distributed clocks, permissions or real service state. Acceptance still requires evidence from the relevant environment and a decision by the owners defined for the project.
"""Offline audit of a fictional event ledger; no broker or transaction implementation."""
import copy
import hashlib
import itertools
import json
import re
from pathlib import Path
def require(value,message):
if not value: raise ValueError(message)
def nonempty(value):
return isinstance(value,str) and bool(value.strip)
def minute(value):
return type(value) is int and value >= 0
def digest(value):
return hashlib.sha256(value.encode('utf-8')).hexdigest
def key(row):
require(isinstance(row,dict),'record object')
require(all(nonempty(row.get(k))for k in ('environment','source','eventId')),'logical context')
return tuple(row[k]for k in ('environment','source','eventId'))
def inspect(data):
require(isinstance(data,dict) and minute(data.get('asOf')),'input and time')
require(type(data.get('inventoryComplete')) is bool,'inventory flag')
for name in ('deliveries','commits','acknowledgments'):
require(isinstance(data.get(name),list),'record lists')
ids=set
for row in data[name]:
require(isinstance(row,dict) and nonempty(row.get('id')),'record id')
require(row['id'] not in ids,'unique record id');ids.add(row['id'])
require(minute(row.get('at')) and row['at']<=data['asOf'],'record time')
if name!='acknowledgments':key(row)
if name=='deliveries':require(nonempty(row.get('payload')),'synthetic payload')
if name=='commits':
require(nonempty(row.get('effectId')),'effect id')
require(isinstance(row.get('payloadSha256'),str) and re.fullmatch('[0-9a-f]{64}',row['payloadSha256']),'digest')
if name=='acknowledgments':
require(nonempty(row.get('deliveryId')),'delivery reference')
require(row.get('outcome')in('success','failed'),'ack outcome')
deliveries={r['id']:r for r in data['deliveries']}
for ack in data['acknowledgments']:
require(ack['deliveryId']in deliveries,'known delivery')
require(ack['at']>=deliveries[ack['deliveryId']]['at'],'ack not before delivery')
groups={key(r)for r in data['deliveries']+data['commits']};result=[]
for k in sorted(groups):
ds=[r for r in data['deliveries']if key(r)==k]
cs=[r for r in data['commits']if key(r)==k]
hashes=sorted({digest(r['payload'])for r in ds}|{r['payloadSha256']for r in cs})
effects=sorted({r['effectId']for r in cs})
gaps=[];unconfirmed=[];failed=[]
for delivery in sorted(ds,key=lambda r:r['id']):
acks=[a for a in data['acknowledgments']if a['deliveryId']==delivery['id']]
successes=[a for a in acks if a['outcome']=='success']
if not successes:unconfirmed.append(delivery['id'])
failed.extend(a['id']for a in acks if a['outcome']=='failed')
for ack in successes:
# Strictly earlier: equal minute values cannot prove event ordering.
matching=[c for c in cs if c['payloadSha256']==digest(delivery['payload']) and c['at']<ack['at']]
if not matching:gaps.append(ack['id'])
result.append(dict(environment=k[0],source=k[1],eventId=k[2],deliveryIds=sorted(r['id']for r in ds),
payloadHashes=hashes,effectIds=effects,contentConflict=len(hashes)>1,
duplicateEffects=len(effects)>1,ackWithoutProvenPriorCommit=sorted(gaps),
unconfirmedDeliveryIds=unconfirmed,failedAcknowledgmentIds=sorted(failed),
commitWithoutObservedDelivery=bool(cs)and not ds))
return dict(events=result,inventoryCoverageUnproven=not data['inventoryComplete'],
actualPersistenceVerified=False,brokerStateVerified=False,reprocessingAuthorized=False)
def sample:
context=dict(environment='training',source='detector-a',eventId='evt-41')
payload='SYNTHETIC batch security observation'
return dict(asOf=100,inventoryComplete=True,
deliveries=[dict(context,id='d1',at=10,payload=payload)],
commits=[dict(context,id='c1',at=11,payloadSha256=digest(payload),effectId='INC-73')],
acknowledgments=[dict(id='a1',deliveryId='d1',at=12,outcome='success')])
def checks:
fixtures=[]
def check(name,data,**expected):
before=copy.deepcopy(data);actual=inspect(data);assert data==before
row=actual['events'][0]if actual['events']else{}
for k,v in expected.items:assert row[k]==v,(name,k,row[k],v)
fixtures.append(dict(id=name,**actual));return actual
check('normal',sample,contentConflict=False,duplicateEffects=False,ackWithoutProvenPriorCommit=[])
x=sample;x['acknowledgments'][0]['outcome']='failed'x['deliveries'].append(dict(x['deliveries'][0],id='d2',at=20));x['acknowledgments'].append(dict(id='a2',deliveryId='d2',at=21,outcome='success'))
check('retry-reuses-effect',x,effectIds=['INC-73'],duplicateEffects=False,unconfirmedDeliveryIds=['d1'],ackWithoutProvenPriorCommit=[])
x=sample;x['commits'][0]['at']=13;check('ack-before-commit',x,ackWithoutProvenPriorCommit=['a1'])
x=sample;x['commits']=[];check('ack-without-commit',x,ackWithoutProvenPriorCommit=['a1'])
x=sample;x['commits'][0]['at']=12;check('same-minute-order-unproven',x,ackWithoutProvenPriorCommit=['a1'])
x=sample;x['acknowledgments'][0]['outcome']='failed'check('ack-failed',x,unconfirmedDeliveryIds=['d1'],failedAcknowledgmentIds=['a1'])
x=sample;x['commits'].append(dict(x['commits'][0],id='c2',effectId='INC-74',at=15));check('duplicate-effects',x,duplicateEffects=True)
x=sample;x['deliveries'].append(dict(x['deliveries'][0],id='d2',payload='SYNTHETIC changed meaning',at=20));check('content-conflict',x,contentConflict=True)
x=sample;x['commits'][0]['payloadSha256']=digest('SYNTHETIC different');check('wrong-payload-commit',x,contentConflict=True,ackWithoutProvenPriorCommit=['a1'])
x=sample;x['commits'][0]['environment']='other-training'a=check('environment-boundary',x);assert len(a['events'])==2;assert next(r for r in a['events']if r['environment']=='training')['ackWithoutProvenPriorCommit']==['a1']
x=sample;x['commits'][0]['source']='detector-b'a=check('source-boundary',x);assert len(a['events'])==2;assert next(r for r in a['events']if r['source']=='detector-a')['ackWithoutProvenPriorCommit']==['a1']
x=sample;x['inventoryComplete']=False;assert check('incomplete-ledger',x)['inventoryCoverageUnproven']
x=sample;x['deliveries']=[];x['acknowledgments']=[];check('commit-outside-observation',x,commitWithoutObservedDelivery=True)
x=sample;x['commits'].append(dict(x['commits'][0],id='c2'));check('repeated-evidence-same-effect',x,duplicateEffects=False)
combinations=0
for commit_time,ack_time,outcome in itertools.product((10,11,12),(10,11,12),('success','failed')):
x=sample;x['commits'][0]['at']=commit_time;x['acknowledgments'][0].update(at=ack_time,outcome=outcome)
r=inspect(x)['events'][0];assert bool(r['ackWithoutProvenPriorCommit'])==(outcome=='success'and commit_time>=ack_time);combinations+=1
x=sample;x['deliveries'] += [dict(x['deliveries'][0],id='d2',at=20),dict(x['deliveries'][0],id='d3',at=30)];reference=inspect(x);permutations=0
for order in itertools.permutations(x['deliveries']):
y=copy.deepcopy(x);y['deliveries']=list(order);assert inspect(y)==reference;permutations+=1
invalid=[]
for field,value in [('asOf',True),('asOf',-1),('inventoryComplete','yes'),('deliveries',None)]:
x=sample;x[field]=value;invalid.append(x)
for field,value in [('id',''),('at',101),('at',1.5),('environment',''),('source',''),('payload','')]:
x=sample;x['deliveries'][0][field]=value;invalid.append(x)
for field,value in [('effectId',''),('payloadSha256','bad'),('at',True)]:
x=sample;x['commits'][0][field]=value;invalid.append(x)
for field,value in [('deliveryId','missing'),('at',9),('outcome','unknown')]:
x=sample;x['acknowledgments'][0][field]=value;invalid.append(x)
for name in ['deliveries','commits','acknowledgments']:
x=sample;x[name].append(dict(x[name][0]));invalid.append(x)
x=sample;del x['commits'][0]['eventId'];invalid.append(x)
for x in invalid:
try:inspect(x)
except ValueError:pass
else:raise AssertionError('invalid input accepted')
return dict(fixtures=fixtures,timingCombinations=combinations,inputPermutations=permutations,invalidInputs=len(invalid),
inputPreserved=True,orderIndependent=True,network=False,cloudExecuted=False,persistentWrites=False,
scriptSha256=hashlib.sha256(Path(__file__).read_bytes).hexdigest)
if __name__=='__main__':print(json.dumps(checks,ensure_ascii=False,indent=2))
After creating INC-73, the consumer loses acknowledgment. Reconcile the next delivery with the persisted effect, preserving content discrepancies.
Common pitfalls
Confusing a tag with a new scan, SKIPPED with patched, mute with remediation, or message acknowledgment with a business transaction.
Related topics: Pipeline identities and trust · Log coverage and retention · Idempotency and incident recovery
Every operational conclusion needs scope and a relevant observation. Preserving progress and gaps prevents lost work and duplicate effects.
Reference: Run builds with a user-managed service account · Current linked guide; edition date unconfirmed (2026-09-30 inspection)