LEARN. PRACTICE. GROW.1. Design intent by direction
In a fictional fund-processing migration, the data center has two Direct Connect paths to the cloud. Before choosing BGP attributes, write down intent: which prefixes should prefer A, which may share load, and how should return traffic behave? Local router policy controls its own outbound selection; communities advertised to AWS can influence the opposite direction. For private and transit VIFs, compare destination specificity first, then attributes of eligible paths for that prefix. High preference on an aggregate does not eliminate a more specific route through the standby path. Record concrete destination examples and expected paths to avoid approving a policy merely because a connection is named primary.
2. Read advertisements without inventing reachability
Consider A advertising 10.72.0.0/16 and B advertising 10.72.8.0/24. For 10.72.8.12, the more specific route guides selection; lengthening B AS_PATH does not make the aggregate more specific. When reviewing allowed prefixes, identify the association type. With VGW, the list filters the VPC CIDR; with TGW, the DX gateway can advertise configured prefixes without a matching VPC. Receiving an advertisement therefore proves neither a destination nor working return traffic. Draw the sequence of VIF, gateway, attachment, table, and destination. When associating several TGWs, check overlap between lists and distinct ASNs for gateways in different Regions. A default route covers every IPv4 prefix in the other associations.
3. Prepare a change across teams
Cross-account implementation distributes responsibility: the TGW owner proposes association and the DX gateway owner accepts, potentially overriding prefixes. The PM should obtain the actually accepted configuration and an owner for each side. Architecture approval does not install router configuration. If the DX gateway already uses a private VIF or VGW association, do not assume it can accept a TGW without migration. Plan dependencies, the window, a temporary path, and rollback. For public VIFs, regional or continental scope communities control propagation but do not constitute application authorization. Keep advertisement, transport, filtering, and transaction criteria separate. Store identifiers and results in the ticket without copying keys or credentials.
4. Measure the mechanism exercised
The Resiliency Toolkit interrupts selected BGP peerings. Use it with an explicit hypothesis: when A loses BGP, the reconciliation flow should recover over B within the agreed limit. The test does not demonstrate physical diversity of ducts or carriers. Collect ConnectionState, BGP state by VIF and IP family, prefixes, timings, and application confirmation. A healthy IPv4 session does not validate IPv6. In BFD, transmission interval respects the larger of sender minimum and peer receive minimum for that direction; do not transfer the BGP timer-negotiation rule. Even with fast detection, reconvergence, retries, and reconciliation add time. Measure the batch effect before approving the SLA.
5. Distinguish continuity from completed rotation
MACsec protects the eligible segment between devices without constituting end-to-end application encryption. The CKN/CAK pair is installed and rotated through coordination; the SAK is derived and renewed by the protocol. Associating a new key in AWS without installing it locally can preserve service using the previous key. Acceptance needs evidence of the new CKN in use at both ends, preserving a recovery option until confirmation. For requirements prohibiting unencrypted fallback, check must_encrypt and effective state. Documentation describes a change to should_encrypt when the last CKN is disassociated following secret-lifecycle problems. Monitor changes and notifications; initial configuration is not a permanent guarantee.
6. Close with operational evidence
Build a matrix containing flow, test address, IP family, primary path, alternate path, protection mechanism, and owner. Execute a positive test and a bounded failure for each row, collecting the functional result. ConnectionErrorCount with Sum helps count interval errors but does not automatically attribute their cause to AWS; correlate both ends. If one large flow dominates bytes, do not declare ECMP broken merely because distribution is uneven. At handover, separate observed facts, hypotheses, and pending rehearsals. An outstanding item can have accepted mitigation, a deadline, and an owner; it should not disappear inside an aggregate green indicator. Support needs to know when to escalate, how to reconcile, and which evidence to preserve before changing configuration again.
# Local evidence exercise, not a BGP simulator or an AWS check.
required = {"ipv4", "ipv6"}
observed = {"ipv4": {"bgp": True, "transaction": True},
"ipv6": {"bgp": False, "transaction": False}}
missing = sorted(f for f in required
if not observed.get(f, {}).get("bgp")
or not observed.get(f, {}).get("transaction"))
print(missing) # ['ipv6']
assert missing == ["ipv6"]
10.72.8.12 follows the /24 through B even when A /16 has high preference. After withdrawing B in rehearsal, confirm the alternate route and file reconciliation.
Common pitfalls
Confusing advertisements with service, BFD with total recovery, SAK rotation with CAK rotation or continuity with new-key adoption.
Related topics: BGP and hybrid routing · Resilience and operational handover
Accept connectivity when path, protection and business outcome have consistent evidence.
Reference: Direct Connect routing and BGP communities · ANS-C01