LEARN. PRACTICE. GROW.From definition to executed pipeline
In the fictional Cedar case, the team defines attributes/sanitize to remove a marker from a span. Validation passes, but the marker remains in the console. The pipeline processors list contains only batch: the component was defined but not wired. Separate these conditions when reading configuration. Also confirm distribution and version because Collector binaries may include different components. Our exercise pins otelcol-contrib 0.162.0. An instance name uses a type and optionally a designation, such as attributes/prepare. The reference must match its definition. A missing reference should be corrected before starting the flow. Static validation is an initial barrier but confirms neither reception nor exported content. Record the expected path: sender, protocol, endpoint, receiver, processors in declared order, and exporter. This sequence helps locate the first boundary where observation differs from expectation.
Order and action matter
The synthetic span starts with demo.stage=raw and demo.keep=original. The prepare processor upserts stage to prepared, attempts to insert a different keep value, and updates an absent key. The result retains keep=original and does not create the absent key. In the next processor, sanitize copies stage to demo.copy and removes demo.secret. If prepare runs first, the copy contains prepared; if sanitize runs first, it contains raw even though stage changes later. The copy is not automatically updated. This distinction avoids diagnosing an output explained by sequence as an export failure. Removing one span key does not demonstrate removal from resource attributes, log bodies, or other pipelines. The test uses an entirely fictional marker. Do not put customer data into the debug exporter to repeat the exercise. Define intended coverage and prepare suitable cases for each location and path under applicable access conditions.
Batch, debug, and evidence limits
The batch processor groups items for sending. send_batch_size is a trigger rather than a hard maximum; send_batch_max_size defines the upper bound when nonzero and must be at least the trigger. The lab rejects the combination three and two. In a design using memory_limiter and sampling, documentation recommends batching after those treatments; this workshop exercises neither memory pressure nor sampling. The detailed debug exporter allows local output inspection, but its format is not a stable API. Text checks are therefore tied to the pinned version. An OTLP HTTP 200 response and an accepted-span counter demonstrate a reception boundary; they demonstrate neither remote storage, vendor querying, nor payment outcome. If the sender uses HTTP while the receiver accepts only gRPC on that port, begin with transport compatibility. Increasing sampling does not fix that mismatch.
Local laboratory with a control
Obtain the official binary appropriate for your system and verify version 0.162.0. Set DR_OTELCOL to its path, copy the complete code below into run.py, and run it with Python 3.13. The code downloads no software: it creates temporary configurations and starts three processes one at a time, bound only to 127.0.0.1. It sends a fictional span and queries internal metrics. The first case leaves the processor unwired and confirms the marker arrives; the second checks transformation; the third reverses order. Two invalid configurations are rejected. Processes are terminated and temporary files removed afterward. Twenty-three checks were executed in the authoring environment. The downloaded artifact was compared with the official release digest; this is not independent verification of its Sigstore signature. The lab does not validate Dynatrace, persistence, saturation, failover, notifications, or a production pipeline.
"""Original loopback-only synthetic OTLP lab for otelcol-contrib 0.162.0.
PT: definir DR_OTELCOL para o binário oficial e executar este código completo.
EN: set DR_OTELCOL to the official binary and run this complete script.
No production credentials, agents, external export, or customer data are used.
"""
import copy
import hashlib
import json
import os
from pathlib import Path
import socket
import subprocess
import tempfile
import time
import urllib.request
BINARY = os.environ.get('DR_OTELCOL')
if not BINARY:
raise SystemExit('Set DR_OTELCOL to otelcol-contrib 0.162.0')
BINARY = str(Path(BINARY).resolve)
version = subprocess.check_output([BINARY, '--version'], text=True).strip
if version!= 'otelcol-contrib version 0.162.0':
raise SystemExit('This lab requires otelcol-contrib 0.162.0')
checks = []
def check(name, condition):
if not condition:
raise AssertionError(name)
checks.append(name)
def port:
with socket.socket as s:
s.bind(('127.0.0.1', 0))
return s.getsockname[1]
def config(receiver_port, metrics_port, processors):
return {
'receivers': {'otlp/lab': {'protocols': {'http': {'endpoint': f'127.0.0.1:{receiver_port}'}}}},
'processors': {
'attributes/prepare': {'actions': [
{'key':'demo.stage','action':'upsert','value':'prepared'},
{'key':'demo.keep','action':'insert','value':'replacement'},
{'key':'demo.absent','action':'update','value':'must-not-appear'}]},
'attributes/sanitize': {'actions': [
{'key':'demo.copy','action':'upsert','from_attribute':'demo.stage'},
{'key':'demo.secret','action':'delete'}]},
'batch': {'timeout':'100ms','send_batch_size':1,'send_batch_max_size':2}},
'exporters': {'debug/lab': {'verbosity':'detailed'}},
'service': {
'telemetry': {'metrics': {'readers': [{'pull': {'exporter': {'prometheus': {
'host':'127.0.0.1','port':metrics_port}}}}]}},
'pipelines': {'traces': {'receivers':['otlp/lab'],'processors':processors,'exporters':['debug/lab']}}}}
def payload:
attrs = [('demo.secret','SYNTHETIC_ONLY'),('demo.stage','raw'),('demo.keep','original')]
return {'resourceSpans':[{'resource':{'attributes':[{'key':'service.name','value':{'stringValue':'dr-fictional-lab'}}]},
'scopeSpans':[{'scope':{'name':'dr-authored-lab'},'spans':[{
'traceId':'11111111111111111111111111111111','spanId':'2222222222222222',
'name':'fictional-position-query','kind':2,
'startTimeUnixNano':'1791030000000000000','endTimeUnixNano':'1791030000001000000',
'attributes':[{'key':k,'value':{'stringValue':v}} for k,v in attrs]}]}]}]}
def validate(tmp, cfg, name, expected):
file = tmp / (name+'.json')
file.write_text(json.dumps(cfg))
result = subprocess.run([BINARY,'validate','--config',str(file)],capture_output=True,text=True,timeout=15)
check(name+' validation', (result.returncode == 0) == expected)
return file
def run_case(tmp, name, processors):
rp, mp = port, port
while rp == mp:
mp = port
cfg = config(rp,mp,processors)
file = validate(tmp,cfg,name,True)
log = tmp / (name+'.log')
with log.open('w') as output:
process = subprocess.Popen([BINARY,'--config',str(file)],stdout=output,stderr=output)
try:
for _ in range(100):
if process.poll is not None:
raise RuntimeError('Collector exited: '+log.read_text)
if 'Everything is ready' in log.read_text:
break
time.sleep(.05)
else:
raise RuntimeError('Collector startup timeout')
request = urllib.request.Request(f'http://127.0.0.1:{rp}/v1/traces',
data=json.dumps(payload).encode,headers={'Content-Type':'application/json'})
with urllib.request.urlopen(request,timeout=5) as response:
check(name+' OTLP accepted', response.status == 200)
body=json.loads(response.read)
check(name+' no partial rejection', not body.get('partialSuccess',{}).get('rejectedSpans',0))
for _ in range(100):
text = log.read_text
if 'fictional-position-query' in text:
break
time.sleep(.05)
else:
raise RuntimeError('Debug export timeout')
with urllib.request.urlopen(f'http://127.0.0.1:{mp}/metrics',timeout=5) as response:
metrics=response.read.decode
check(name+' internal receiver signal', 'otelcol_receiver_accepted_spans' in metrics)
return text
finally:
if process.poll is None:
process.terminate
try:
process.wait(timeout=10)
except subprocess.TimeoutExpired:
process.kill;process.wait(timeout=5)
with tempfile.TemporaryDirectory(prefix='dr-otel-pipeline-') as directory:
tmp=Path(directory)
raw=run_case(tmp,'defined-but-unwired',['batch'])
check('unwired processor does not delete', 'demo.secret: Str(SYNTHETIC_ONLY)' in raw)
clean=run_case(tmp,'ordered',['attributes/prepare','attributes/sanitize','batch'])
check('wired deletion', 'demo.secret' not in clean and 'SYNTHETIC_ONLY' not in clean)
check('ordered copy', 'demo.copy: Str(prepared)' in clean)
check('insert preserves existing', 'demo.keep: Str(original)' in clean)
check('update does not insert', 'demo.absent' not in clean)
reverse=run_case(tmp,'reversed',['attributes/sanitize','attributes/prepare','batch'])
check('processor order changes copied value', 'demo.copy: Str(raw)' in reverse)
check('reversed still deletes selected attribute', 'demo.secret' not in reverse)
check('resource attribute retained', 'service.name: Str(dr-fictional-lab)' in clean)
invalid=config(port,port,['attributes/missing'])
validate(tmp,invalid,'missing-processor-reference',False)
invalid=config(port,port,['batch'])
invalid['processors']['batch'].update(send_batch_size=3,send_batch_max_size=2)
validate(tmp,invalid,'batch-trigger-exceeds-max',False)
check('payload was not mutated', payload['resourceSpans'][0]['scopeSpans'][0]['spans'][0]['attributes'][0]['value']['stringValue']=='SYNTHETIC_ONLY')
print(json.dumps(dict(version=version,checks_passed=len(checks),checks=checks,
binary_sha256=hashlib.sha256(Path(BINARY).read_bytes).hexdigest,
scope='Three real local Collector runs and two rejected configurations using only synthetic OTLP traces. No vendor backend, production monitoring, persistence, saturation or failover test.'),indent=2))
prepare → sanitize produces demo.copy=prepared; sanitize → prepare produces demo.copy=raw. The difference follows ordering, not a faulty exporter.
Common pitfalls
Defined component treated as active; insert treated as upsert; ignored order; trigger treated as maximum; debug treated as proof of remote storage.
Related topics: OpenTelemetry Collector · Attribute handling · Boundary-based diagnosis
Validate configuration and demonstrate observed behavior with a pinned version, a control, and explicit limits.
Reference: Collector configuration · Observability 2026-09; selected OpenTelemetry, Prometheus and Dynatrace Classic concepts