LEARN. PRACTICE. GROW.1. Define what the change must demonstrate
A fictional fund valuation application will move to another network segment. Its batch must query prices over TCP/8443, administrators need a separate path, and an unauthorized source must remain blocked. This scenario does not describe internal BNP Paribas procedures. Start before choosing the tool: write the source, destination, protocol, port, change revision and expected result for each path. “The network works” is too broad to support a decision. Distinguish three questions. Does configuration predict a compatible path? Does a recent observation show delivery under the tested conditions? Does the application complete its required operation within the deadline? Keep separate answers because one can be favorable while another fails. For example, the API might have no listening process even when network rules permit the connection. An absent observation must remain visible as a gap with an owner and a next action. At the committee, the PM presents service impact, available evidence and time needed to resolve gaps. APS confirms essential flows; networking identifies controls; development verifies the functional operation. Define a negative check as well: an unauthorized source attempts the same destination and should be blocked. Acceptance requires checking intended behavior, including restrictions, rather than maximizing allowed connections. Before proceeding, explain the specific failure each check could detect and which decision would change if that failure appeared.
2. Use diagnostics within their scope
Connectivity Tests helps investigate the path predicted by configuration. In supported scenarios it can add data plane probes. The report should identify which mechanism produced each result. A favorable prediction is not a completed application transaction. When the tool warns that a feature is unsupported, retain that warning in the analysis and assign an alternative check. An empty evidence field is not an approved control. Consider an API behind an external Application Load Balancer. The path to its IP can look compatible while Cloud Armor blocks the HTTP request. Analysis of that path does not evaluate the Cloud Armor policy; investigation needs evidence from that layer. If potentially relevant firewall rules use FQDN objects, an unsupported-feature warning also prevents claiming complete coverage of those rules. These limits guide evidence collection without demonstrating that the controls are disabled. In a second example, every probe sent toward a load balancer succeeds. That does not automatically identify all visited backends. List the destinations that matter to the service and compare that list with observed coverage. If the link between probes and destinations is missing, state the limitation. When testing an IP outside Google, do not extend the measurement into the partner’s application processing. The report should state where observation ends and who confirms the next part of the flow. Ask what additional observation would justify expanding the claim.
3. Investigate disagreements and the return path
Disagreement between configuration and observation is useful information. First confirm that results describe the same flow and revision. A recent change can make reports collected at different times incomparable. Record the time, change identifier, endpoints and test direction in the ticket. Retain permission errors or unsupported arguments that prevented analysis from finishing. If prediction indicates delivery but the application fails, inspect the listening process and operating-system firewall. Do not begin by opening every port: that change can increase exposure without fixing a stopped process. If probes succeed in one direction while replies fail, investigate the reverse direction too. Unidirectional analysis does not automatically cover the return path. Identify who can observe each side and correlate the client attempt with processing and response at the destination. In the fictional case, APS captures three failed new attempts after the change. Networking has an earlier favorable test and development confirms that an old session still works. The PM does not select the most convenient result. Retain all three facts, check whether they belong to the same revision and request a new attempt with coordinated observation. If time before rollback runs out without demonstrating the required flow, use the agreed decision criterion. The window deadline limits investigation; it does not turn a gap into favorable evidence. Record the remaining hypothesis so the next attempt starts with a specific investigation.
4. Separate existing connections from new connections
The VPC firewall tracks allowed connections. Reply traffic matching that state should not be interpreted as an independent new connection. This matters when designing a change and interpreting tests: a persistent session can keep working without demonstrating that the application can open another session under current conditions. Specify whether a check starts a new connection or reuses an existing pool. When reviewing two VPC rules applicable to a new connection, consider priority before apparent target specificity. A priority-420 allow is not overridden merely because a priority-900 deny names a narrower target. In a matching allow/deny priority tie, deny takes precedence. Do not transfer these local comparisons into one global list mixing hierarchical policies without respecting evaluation order. In the review exercise, write two columns: existing-session behavior and new-attempt behavior. Decide beforehand which one is required after service restart. If the criterion requires reconnection, a response through the old pool does not satisfy it. Retain an expected negative result for an unauthorized source as well. When presenting the check, identify the context that produced the observation and avoid saying only “port open.” That phrase hides who attempted the connection, in which direction, with what previous state and for what purpose. Good evidence allows the attempt to be repeated without depending on the operator’s memory.
5. Review associations, sources and observability
A created hierarchical firewall policy still needs the association determining where it applies. In review, show the associated organization or folder and the hierarchy of covered resources. Distinguish an ancestor allow or deny decision from delegation through goto_next. A lower control does not override a terminal ancestor decision merely by having a smaller priority number. This distinction avoids proposing a local correction that cannot produce the intended effect. For VPC rules, inspect the actual source set too. Combining a range and a source network tag can look like a simultaneous requirement, but the valid combination forms a union. A useful exercise is to list three fictional sources: one belonging to the range, another identified by the tag, and another meeting neither condition. Explain which match before accepting the design. If the service adds IPv6, review that family explicitly; IPv4 coverage does not demonstrate the new path. Observability needs its own review. Enabling logging for a continuously active connection does not force a new record to appear at that instant. Furthermore, the implied rules described in the documentation do not offer the same logging support as an explicit rule configured for that purpose. For a report with no records, check the rule, protocol, enablement time and whether a new attempt occurred. Missing logs, absent traffic and no blocking are different claims requiring appropriate evidence.
6. Diagnose Private Service Connect with the producer
For a PSC published service, Accepted confirms configuration acceptance but does not guarantee useful delivery. Keep functional testing in production handover. Needs attention indicates a producer-side issue and can coexist with partial traffic; an exhausted NAT subnet is a documented hypothesis. Closed after attachment deletion is terminal and requires a plan to recreate the involved resources. Do not confuse lifecycle recovery with a simple accept-list change. During an incident, identify the endpoint and counter direction. dropped_sent_packets_count relates to the endpoint connection maximum. dropped_received_packets_count indicates replies for which PSC cannot find a matching connection. Late replies and timeouts help investigate the second case, but one counter does not automatically assign a unique application cause. Compare the same time window on consumer and producer sides and retain examples of useful and failed requests. For the fictional pricing service, the PM brings APS and the producer together with a concrete question: do expired requests correspond to late replies and receive drops? Define who collects each piece of evidence and when mitigation will be decided. Also check configuration dependencies where relevant: global access requires published-service compatibility, and a DNS zone conflict can prevent expected automatic creation. Before deleting an existing zone, identify its dependent consumers. Resolving a creation conflict does not justify disrupting another service without impact analysis.
7. Exercise: build an evidence matrix
This lesson’s program is a local worksheet, not a Google Cloud simulator. It accepts declared requirements with source, destination, protocol and destination port. Observations include revision, minute, kind configuration or probe, and outcome reachable, blocked or unknown. For probes, the author also declares whether the connection is new or reused. The program does not verify those values externally: the validity of declarations remains the evidence collector’s responsibility. Run python3 run.py and inspect the included cases. At time 100 with maxAge=10, an observation from minute 90 is inside the window and one from minute 89 is excluded. An r6 observation does not demonstrate revision r7. Exclusions are listed with reasons instead of silently disappearing. Configuration results appear separately from new-connection probes. A reused session remains identified but does not establish reconnection. Before changing code, predict three results. First, configuration says reachable and a new probe says blocked when reachable was expected: failure takes precedence. Second, one probe succeeds and another is unknown: coverage remains unproven. Third, a requirement expects blocked and the probe confirms blocked: the observed outcome matches the intended restriction. Then add a second mandatory flow without evidence. Even if the first is demonstrated, the declared set is incomplete. Explain the difference between a favorable result on one row and coverage of the entire matrix. Preserve both conclusions in the handover.
8. Present a decision that can be reviewed
An observed-as-expected result describes only supplied observations for one requirement. Even when every row has that result, application testing can remain outstanding and the inventory can be incomplete. The program retains inventoryCoverageUnproven separately. It sends no packets, evaluates no real rules and does not authorize production. Source and destination identifiers are exercise labels; they do not validate resources, identities or every property of a real connection. To complete the case, write a short English note containing four elements: requirement, observation, limitation and action. For example, identify the pricing flow and revision, describe failed new connections, state that the old session does not demonstrate reconnection, and assign joint investigation before the rollback deadline. Add who decides production acceptance and the concrete condition still to satisfy. A recipient who missed the call should be able to distinguish an observed fact from a hypothesis. Review rejected alternatives too. Opening access indiscriminately does not establish cause; hiding a conflicting result weakens the decision; repeating a test outside the required scope does not fill the gap. If recovery is selected, record the resulting revision and repeat applicable checks. Evidence from the failed implementation does not replace confirmation of recovery. The final objective is a traceable decision with explicit limits, responsibilities and a concrete way to confirm the required service. Use the same structure when handing the investigation to another time zone.
"""Original offline worksheet. Inputs are declarations, not verified cloud evidence."""
import copy
import hashlib
import itertools
import json
from pathlib import Path
def require(condition, message):
if not condition:
raise ValueError(message)
def integer(value, minimum=0):
return type(value) is int and value >= minimum
def nonempty(value):
return isinstance(value, str) and bool(value.strip)
def flow(row):
require(isinstance(row, dict), 'record must be an object')
require(all(nonempty(row.get(k)) for k in ('source', 'destination')), 'flow endpoints')
require(row.get('protocol') in ('TCP', 'UDP'), 'protocol')
require(integer(row.get('destinationPort'), 1) and row['destinationPort'] <= 65535, 'port')
return tuple(row[k] for k in ('source', 'destination', 'protocol', 'destinationPort'))
def assess(data):
require(isinstance(data, dict), 'input object')
require(nonempty(data.get('revision')), 'revision')
require(integer(data.get('asOf')) and integer(data.get('maxAge')), 'integer minutes')
require(type(data.get('inventoryComplete')) is bool, 'inventory flag')
requirements, evidence = data.get('requirements'), data.get('evidence')
require(isinstance(requirements, list) and len(requirements) > 0, 'nonempty requirements')
require(isinstance(evidence, list), 'evidence list')
ids, flows = set, set
for row in requirements:
key = flow(row)
require(nonempty(row.get('id')) and row['id'] not in ids, 'unique requirement id')
require(key not in flows, 'unique requirement flow')
require(row.get('expected') in ('reachable', 'blocked'), 'expected result')
ids.add(row['id']); flows.add(key)
ids = set
for row in evidence:
flow(row)
require(nonempty(row.get('id')) and row['id'] not in ids, 'unique evidence id')
ids.add(row['id'])
require(nonempty(row.get('revision')), 'evidence revision')
require(integer(row.get('checkedAt')) and row['checkedAt'] <= data['asOf'], 'observation time')
require(row.get('kind') in ('configuration', 'probe'), 'evidence kind')
require(row.get('outcome') in ('reachable', 'blocked', 'unknown'), 'outcome')
if row['kind'] == 'probe':
require(row.get('connection') in ('new', 'reused'), 'connection kind')
result, excluded = [], []
usable = []
for row in evidence:
reasons = []
if flow(row) not in flows: reasons.append('other-flow')
if row['revision']!= data['revision']: reasons.append('other-revision')
if row['checkedAt'] < data['asOf'] - data['maxAge']: reasons.append('stale')
if reasons: excluded.append({'id': row['id'], 'reasons': reasons})
else: usable.append(row)
for requirement in sorted(requirements, key=lambda r: r['id']):
matching = [r for r in usable if flow(r) == flow(requirement)]
config = sorted({r['outcome'] for r in matching if r['kind'] == 'configuration'})
fresh = [r for r in matching if r['kind'] == 'probe' and r['connection'] == 'new']
observed = sorted({r['outcome'] for r in fresh})
unexpected = any(x not in (requirement['expected'], 'unknown') for x in observed)
if unexpected: status = 'observed-unexpected'
elif observed == [requirement['expected']]: status = 'observed-as-expected'
else: status = 'not-demonstrated'
result.append({'id': requirement['id'], 'expected': requirement['expected'],
'configurationStates': config, 'newProbeStates': observed,
'newProbeIds': sorted(r['id'] for r in fresh),
'reusedProbeIds': sorted(r['id'] for r in matching if r['kind'] == 'probe' and r['connection'] == 'reused'),
'status': status, 'conflictingNewProbes': 'reachable' in observed and 'blocked' in observed})
return {'requirements': result, 'excluded': sorted(excluded, key=lambda r: r['id']),
'allDeclaredFlowsObservedAsExpected': all(r['status'] == 'observed-as-expected' for r in result),
'inventoryCoverageUnproven': not data['inventoryComplete'],
'actualNetworkTested': False, 'applicationHealthVerified': False,
'productionAcceptanceAuthorized': False}
def sample:
f = dict(source='batch-a', destination='pricing', protocol='TCP', destinationPort=8443)
return dict(revision='r7', asOf=100, maxAge=10, inventoryComplete=True,
requirements=[dict(f, id='prices', expected='reachable')],
evidence=[dict(f, id='p1', revision='r7', checkedAt=95, kind='probe', connection='new', outcome='reachable')])
def checks:
fixtures = []
def check(name, data, expected):
before = copy.deepcopy(data); actual = assess(data)
assert data == before
assert actual['requirements'][0]['status'] == expected, name
fixtures.append(dict(id=name, **actual))
return actual
base = sample
check('current-success', base, 'observed-as-expected')
x=sample; x['evidence'][0]['kind']='configuration'
check('configuration-only', x, 'not-demonstrated')
x=sample; x['evidence'][0]['connection']='reused'
check('old-pool', x, 'not-demonstrated')
x=sample; x['evidence'][0]['outcome']='blocked'
x['evidence'].append(dict(x['evidence'][0], id='c1', kind='configuration', outcome='reachable'))
check('prediction-cannot-hide-failure', x, 'observed-unexpected')
x=sample; x['evidence'].append(dict(x['evidence'][0], id='p2', outcome='blocked'))
assert check('mixed-probes', x, 'observed-unexpected')['requirements'][0]['conflictingNewProbes']
x=sample; x['evidence'].append(dict(x['evidence'][0], id='p2', outcome='unknown'))
check('unknown-plus-success', x, 'not-demonstrated')
x=sample; x['evidence'][0]['checkedAt']=90
check('window-boundary', x, 'observed-as-expected')
x=sample; x['evidence'][0]['checkedAt']=89
assert check('stale-result', x, 'not-demonstrated')['excluded'][0]['reasons']==['stale']
x=sample; x['evidence'][0]['revision']='r6'
assert check('previous-revision', x, 'not-demonstrated')['excluded'][0]['reasons']==['other-revision']
x=sample; x['evidence'][0]['destinationPort']=443
assert check('different-port', x, 'not-demonstrated')['excluded'][0]['reasons']==['other-flow']
x=sample; x['inventoryComplete']=False
assert check('incomplete-inventory', x, 'observed-as-expected')['inventoryCoverageUnproven']
x=sample; x['requirements'][0]['expected']='blocked' x['evidence'][0]['outcome']='blocked'
check('negative-check', x, 'observed-as-expected')
x=sample; x['requirements'].append(dict(x['requirements'][0], id='admin', destinationPort=22, expected='blocked'))
assert not check('missing-second-flow', x, 'not-demonstrated')['allDeclaredFlowsObservedAsExpected']
combinations=0
for expected,outcome,kind,connection,revision in itertools.product(('reachable','blocked'),('reachable','blocked','unknown'),('configuration','probe'),('new','reused'),('r7','r6')):
x=sample; x['requirements'][0]['expected']=expected
x['evidence'][0].update(outcome=outcome,kind=kind,connection=connection,revision=revision)
target='not-demonstrated'
if kind=='probe' and connection=='new' and revision=='r7' and outcome!='unknown':
target='observed-as-expected' if outcome==expected else 'observed-unexpected'
assert assess(x)['requirements'][0]['status']==target
combinations+=1
x=sample; x['evidence'] += [dict(x['evidence'][0],id='p2',outcome='blocked'),dict(x['evidence'][0],id='p3',revision='r6')]
reference=assess(x); permutations=0
for p in itertools.permutations(x['evidence']):
y=copy.deepcopy(x); y['evidence']=list(p); assert assess(y)==reference; permutations+=1
invalid=[]
for key,value in [('revision',''),('asOf',True),('maxAge',-1),('inventoryComplete','yes'),('requirements',[]),('evidence',None)]:
x=sample; x[key]=value; invalid.append(x)
for key,value in [('destinationPort',True),('destinationPort',65536),('source',''),('expected','unknown')]:
x=sample;x['requirements'][0][key]=value;invalid.append(x)
for key,value in [('checkedAt',101),('checkedAt',1.5),('kind','live'),('connection','unknown'),('outcome','pass'),('protocol','ICMP'),('revision','')]:
x=sample;x['evidence'][0][key]=value;invalid.append(x)
x=sample;x['evidence'].append(dict(x['evidence'][0]));invalid.append(x)
x=sample;x['requirements'].append(dict(x['requirements'][0],id='duplicate-flow'));invalid.append(x)
x=sample;del x['evidence'][0]['checkedAt'];invalid.append(x)
for x in invalid:
try: assess(x)
except ValueError: pass
else: raise AssertionError('invalid input accepted')
return {'fixtures':fixtures,'stateCombinations':combinations,'inputPermutations':permutations,
'invalidInputs':len(invalid),'inputPreserved':True,'orderIndependent':True,
'network':False,'cloudExecuted':False,'persistentWrites':False,
'scriptSha256':hashlib.sha256(Path(__file__).read_bytes).hexdigest}
if __name__ == '__main__':
print(json.dumps(checks, ensure_ascii=False, indent=2))
During a fictional migration, the old pool responds but three new connections fail. The team retains results, checks revision and investigates destination and return path before go/no-go.
Common pitfalls
Treating Accepted as application health, confusing probes with full coverage, ignoring reused connections or interpreting missing logs as no blocking.
Related topics: Change management and rollback · Monitoring and observability · Resilience and production handover
Define the claim each check supports, retain failures and gaps, and verify required flows on the revision being accepted.
Reference: Connectivity Tests overview · Current linked guide; edition date unconfirmed (2026-09-30 inspection)