LEARN. PRACTICE. GROW.Measure the complete path of a change
In a cloud project, a team can work quickly yet deliver slowly. To understand the difference, follow a request from acceptance to delivery and record processing, waiting, rework and owners. In this lesson’s example, four hours of work sit inside a forty-hour path. Reducing processing to two hours leaves thirty-eight hours overall because the thirty-six waiting hours remain. The improvement is real, but reporting should show its five-percent effect on the complete delivery interval. Map this with development, QA, security, platform and APS because one team may not see queues before and after its own involvement. Choose an observable wait, identify who can change it and define how to measure the outcome. Avoid starting with a tool purchase when the blocker is a rule, a dependency or a missing owner.
Unblock work before enlarging the queue
A test column holding four blocked items gains no capacity when six more arrive. Keep unfinished work visible and direct collaboration toward restoring the missing environment, information or decision. A work-in-process limit helps make the constraint explicit; it should not be bypassed by closing incomplete tickets. For repetitive low-risk changes, reviewing the process can reduce unnecessary waits. Prepare an authorized pilot with peer review, test evidence, traceability and escalation criteria for higher-risk changes. Define the pilot’s duration, scope, owner and suspension conditions. Then compare timing and stability with an appropriate baseline. The intended outcome is completing useful changes with verifiable control. An increase in started items, or removing an approval without authorization, does not demonstrate that improvement to the business or RUN. Keep an explicit route for urgent work so exceptions remain visible rather than silently changing the normal process.
Choose the right unit for each metric
Thirty deployments and three failures requiring intervention correspond to a ten-percent rate. If each failure generates four alerts, there are still three affected deployments rather than twelve. Retain the link between signals and the event counted by the metric. Likewise, recovery from failed deployments should be distinguished from recovery after incidents unrelated to a change. Both matter but answer different questions. Compare outcomes over time within each service’s context, avoiding an isolated frequency ranking that encourages artificial changes. Documentation also needs a useful measure. In a runbook rehearsal, observe whether a RUN operator can find the sequence, meet prerequisites and complete the task without depending on its author. Page count describes volume; performance on that task provides evidence about the autonomy the handover is intended to deliver. Record points of confusion and use them to improve the next version.
Define scope before summing costs
A usage report and an invoice report can assign the same row to different months. First decide which question the report answers. To reconcile an invoice, preserve invoice.month; to investigate consumption, preserve usage time or period. Do not duplicate the charge to represent both perspectives. Also retain currency in the grouping key. One hundred EUR and one hundred twenty USD do not become two hundred twenty in a currency chosen by row order. Any conversion needs an explicit rule and the reference used. In the local exercise, invoice_month and usage_month are simplified input labels, and the ID is created for the exercise. They do not promise a unique identifier or that schema in the real export. Before automating a production report, confirm the available data’s grain and identification rules with FINOPS and source owners.
Exercise: preserve a row before expanding credits
The original dataset contains L1 with cost 120 and credits of -12 and -8, plus L2 costing 80 without credits. Correct accounting is 120 - 12 - 8 + 80 = 180. A CROSS JOIN expanding credits produces two copies of L1’s cost and no row for L2, yielding 220. Before running the lab, write the intermediate rows and explain every component. Run python3 content/labs/pca-billing-grain/run.py to compare per-row aggregation with two incorrect transformations. Implementation uses in-memory SQLite and integer millionths within explicit bounds; it neither executes GoogleSQL nor contacts Cloud Billing. Code validates the synthetic schema, IDs, month labels and amount precision. Results demonstrate arithmetic and cardinality within that model. They do not establish export completeness, invoice settlement or application of tax rules. Preserve this distinction when reporting what the exercise tested.
Investigate a correct total reached incorrectly
In the committee case, change L2’s cost to 120. Manual accounting becomes 220 and the incorrect query also returns 220. L1’s repeated cost and L2’s missing cost now have equal values and cancel out. This sample does not validate the transformation. Restore L2=80, add rows without credits and try two legitimate rows with the same cost. A SUM(DISTINCT cost) repair also fails: it removes equal values without respecting row identity. With two costs of 100 and total credits of -20, it returns 80 instead of 180. Useful reconciliation checks per-row contributions, currencies, periods and totals, including datasets capable of exposing different errors. To support the committee, a temporarily reconciled report with clear scope and cutoff can be used while the automated transformation is corrected and rehearsed again. Record the workaround’s owner and when the automated report will be reassessed.
Connect spending to outcomes and assumptions
When the business asks for cost per accepted report, use distinct accepted reports as the denominator. On a day costing 600, with twelve hundred HTTP requests and three hundred accepted reports, the KPI is two units per report. Dividing by requests would produce a different measure, and retries could artificially improve that indicator. Define numerator, denominator, period and quality criteria before comparing trends. In transition budgeting, separate recurring costs, overlap and one-time payments. The six-month example contains 24000 for the new platform, 6000 for the old platform during the first two months and 10000 for migration: total 40000. These are fictional assumptions, not vendor prices or proof of return. If old-platform closure is delayed, update the overlap period and forecast. Keep assumptions awaiting validation visible so the sponsor can decide on cost, schedule and benefit using a consistent scope.
Present decisions that can be checked
Selecting a managed service should also account for future exit. A familiar protocol does not establish extension compatibility or recovery on another target. Prepare a rehearsal using functionality actually relied on and estimate adaptation, transfer, operations and skills. Apply the same care to sustainability claims. For one month, 120 kgCO2e location-based and 80 kgCO2e market-based represent two accounting bases; their difference does not prove a project-caused reduction. Compare equivalent periods and scopes while identifying methodology and revisions. A useful committee update is: “The sample total matches, but two transformation errors cancel out. We will use the reconciled report while correcting and retesting the automated calculation.” Summarize each decision with requirement, observation, limitation, action, owner and date. FINOPS, APS and the sponsor can then check progress without relying on an isolated metric or an apparently favorable total.
"""Original finite billing-grain exercise. SQLite in memory, never BigQuery.
Input IDs are synthetic exercise IDs, not an asserted Cloud Billing export key.
Amount strings have <=8 integer digits and <=6 fractional digits; no rounding.
At most 1000 rows and 10 credits per row keep integer SQL sums within int64.
This model excludes tax logic, FX, export completeness and invoice settlement.
"""
import copy
import itertools
import json
import re
import sqlite3
FIELDS = {'id', 'currency', 'invoice_month', 'usage_month', 'cost', 'credits'}
def micros(value):
if not isinstance(value, str) or not re.fullmatch(r'-?\d{1,8}(?:\.\d{1,6})?', value, flags=re.ASCII):
raise ValueError('amount must be a bounded decimal string')
sign = -1 if value.startswith('-') else 1
whole, _, fraction = value.lstrip('-').partition('.')
return sign * (int(whole) * 1000000 + int(fraction.ljust(6, '0') or '0'))
def money(value):
sign = '-' if value < 0 else ''
whole, fraction = divmod(abs(value), 1000000)
return f'{sign}{whole}.{fraction:06d}'
def validate(rows):
if not isinstance(rows, list) or len(rows) > 1000:
raise ValueError('expected at most 1000 rows')
seen = set
for row in rows:
if not isinstance(row, dict) or set(row)!= FIELDS:
raise ValueError('unexpected row fields')
if not isinstance(row['id'], str) or not row['id'] or row['id'] in seen:
raise ValueError('exercise IDs must be nonempty and unique')
seen.add(row['id'])
if not isinstance(row['currency'], str) or not re.fullmatch(r'[A-Z]{3}', row['currency']):
raise ValueError('currency requires three uppercase letters; no registry validation')
for field in ['invoice_month', 'usage_month']:
value = row[field]
if not isinstance(value, str) or not re.fullmatch(r'20\d{2}(0[1-9]|1[0-2])', value, flags=re.ASCII):
raise ValueError('exercise month must be YYYYMM within 2000–2099')
micros(row['cost'])
if not isinstance(row['credits'], list) or len(row['credits']) > 10:
raise ValueError('expected at most 10 credits per row')
for credit in row['credits']:
micros(credit)
def totals(rows, mode='per-row'):
validate(rows)
if mode not in {'per-row', 'incorrect-flat', 'incorrect-distinct'}:
raise ValueError('unknown demonstration mode')
db = sqlite3.connect(':memory:')
try:
db.executescript('''
CREATE TABLE lines(id TEXT PRIMARY KEY, currency TEXT, invoice_month TEXT,
usage_month TEXT, cost INTEGER);
CREATE TABLE credits(line_id TEXT, amount INTEGER);
''')
for r in rows:
db.execute('INSERT INTO lines VALUES (?,?,?,?,?)',
(r['id'], r['currency'], r['invoice_month'], r['usage_month'], micros(r['cost'])))
db.executemany('INSERT INTO credits VALUES (?,?)', [(r['id'], micros(c)) for c in r['credits']])
if mode == 'per-row':
sql = '''SELECT l.currency, l.invoice_month,
SUM(l.cost + COALESCE((SELECT SUM(c.amount)
FROM credits c WHERE c.line_id=l.id),0))
FROM lines l GROUP BY l.currency,l.invoice_month
ORDER BY l.currency,l.invoice_month'''
else:
cost = 'SUM(l.cost)' if mode == 'incorrect-flat' else 'SUM(DISTINCT l.cost)'
sql = f'''SELECT l.currency,l.invoice_month,{cost}+SUM(c.amount)
FROM lines l JOIN credits c ON c.line_id=l.id
GROUP BY l.currency,l.invoice_month ORDER BY l.currency,l.invoice_month'''
return [dict(currency=c, invoice_month=m, net=money(n)) for c, m, n in db.execute(sql)]
finally:
db.close
def row(identity, cost='120', credits=None, currency='EUR', invoice_month='202610', usage_month='202609'):
return dict(id=identity, currency=currency, invoice_month=invoice_month,
usage_month=usage_month, cost=cost, credits=[] if credits is None else credits)
def demo:
checks = []
def check(name, condition):
if not condition:
raise AssertionError(name)
checks.append(name)
def reject(name, rows):
try:
totals(rows)
except ValueError:
checks.append(name)
else:
raise AssertionError(name)
def value(rows, mode='per-row'):
return totals(rows, mode)[0]['net']
original = [row('A', credits=['-12', '-8']), row('B', '80')]
check('per-row total preserves 180', value(original) == '180.000000')
check('flattening yields wrong 220', value(original, 'incorrect-flat') == '220.000000')
cancel = [row('A', credits=['-12', '-8']), row('B', '120')]
check('offsetting errors hide behind 220', value(cancel) == value(cancel, 'incorrect-flat') == '220.000000')
equal = [row('A', '100', ['-6', '-4']), row('B', '100', ['-10'])]
check('equal legitimate costs both retained', value(equal) == '180.000000')
check('distinct-value repair loses a cost', value(equal, 'incorrect-distinct') == '80.000000')
check('row without credits retained', value([row('A', '80')]) == '80.000000')
check('inner join loses credit-free population', totals([row('A', '80')], 'incorrect-flat') == [])
separate = totals([row('A', '100'), row('B', '120', currency='USD')])
check('currencies stay separate', separate == [dict(currency='EUR', invoice_month='202610', net='100.000000'), dict(currency='USD', invoice_month='202610', net='120.000000')])
check('invoice axis retained despite usage month', totals([row('A')])[0]['invoice_month'] == '202610')
check('positive credit adjustment is not discarded', value([row('A', '90', ['10'])]) == '100.000000')
check('negative cost adjustment retained', value([row('A', '-15')]) == '-15.000000')
check('empty scope produces no totals', totals([]) == [])
check('micro-unit precision retained', value([row('A', '1.000001', ['-0.000001'])]) == '1.000000')
check('signed zero normalized', value([row('A', '-0')]) == '0.000000')
check('all input orders reconcile identically', all(totals(list(p)) == totals(original) for p in itertools.permutations(original)))
reject('non-list scope rejected', {})
bad = row('A');bad['extra'] = 1;reject('unknown fields rejected', [bad])
for name, amount in [('boolean amount rejected', True), ('float amount rejected', 1.2), ('scientific notation rejected', '1e2'), ('nonfinite amount rejected', 'NaN'), ('excess precision rejected', '1.0000001'), ('overbound amount rejected', '100000000')]:
reject(name, [row('A', amount)])
reject('duplicate synthetic ID rejected', [row('A'), row('A')])
reject('invalid currency syntax rejected', [row('A', currency='eur')])
reject('invalid invoice month rejected', [row('A', invoice_month='202613')])
reject('invalid usage month rejected', [row('A', usage_month='202600')])
reject('empty ID rejected', [row('')])
bad = row('A');bad['credits'] = None;reject('null credit collection rejected', [bad])
bad = row('A');bad['credits'] = '-10'reject('string credit collection rejected', [bad])
reject('overbound row count rejected', [row(str(i)) for i in range(1001)])
reject('overbound credit count rejected', [row('A', credits=['-1'] * 11)])
snapshot = copy.deepcopy(original);totals(original);check('input is not mutated', original == snapshot)
return dict(passed=len(checks), checks=checks, correct=totals(original), incorrect=totals(original, 'incorrect-flat'),
cancellation=totals(cancel), currencies=separate, network=False, vendorExecution=False, persistentWrites=False,
runtime='SQLite in-memory relational exercise; no GoogleSQL engine or live invoice validated')
if __name__ == '__main__':
print(json.dumps(demo, indent=2))
A query duplicates a cost of 120 and loses another cost of 120: the total matches the reference, but transformation fails when the second value changes to 80.
Common pitfalls
Measuring only active time, hiding blocked work, counting alerts as deployments, adding currencies, using DISTINCT to repair grain and treating emission bases as before/after states.
Related topics: Migration, costs and acceptance criteria · Data reconciliation and time boundaries · Change governance and RUN autonomy
Define unit and scope, preserve original contributions and test examples that could contradict the conclusion before taking it to the committee.
Reference: Visibility of work in the value stream · Current linked standard guide; edition date unconfirmed (2026-09-30 inspection)