Self-study deck · aligned to AWS Certified Advanced Networking – Specialty (ANS-C01)

AWS Networking
A to Z

Follow a packet from one ENI to many VPCs, accounts, Regions and on-premises sites: routing, BGP, DNS, load balancing, inspection, troubleshooting and cost — at production depth and exam depth.

Use ← → / Space to navigate · I contents · O overview · T theme · F fullscreen

AWS Networking A to Z

Contents

Click any line to jump there · press I to return to this page

Chapter 0

Orientation

The mental model of the AWS network, the building blocks, and how the ANS-C01 exam is built.

Mental model

The AWS network is software, not cables

Every packet leaves an ENI and is handled by the Nitro card and the VPC mapping service before it touches a wire.

What is different from a data center

  • No broadcast or multicast by default (multicast only via Transit Gateway).
  • No ARP spoofing or L2 tricks: the VPC answers ARP for you.
  • No L2 between subnets; every subnet is routed by the implicit VPC router.
  • Security groups are enforced on the host, before traffic reaches the wire.

What stays the same

  • IP addressing, CIDR, longest prefix match.
  • TCP/UDP, ports, MTU and fragmentation.
  • BGP for hybrid routing; IPsec for VPN.
  • DNS is still the first thing that breaks.

Why it matters

  • Tricks such as gratuitous ARP failover or VRRP floating IPs do not work; use Route Server, ENI move or an NLB instead.
  • The "router" is the route table, not a box you can SSH into.
  • Troubleshoot with logs and analyzers, not with tcpdump on a switch.
Mental model

Building blocks: Region → AZ → VPC → subnet → ENI

A VPC spans every AZ in a Region; a subnet lives in exactly one AZ; an ENI lives in exactly one subnet.

Region ap-northeast-1VPC 10.0.0.0/16AZ aAZ cPublic 10.0.0.0/24Private 10.0.10.0/24Public 10.0.1.0/24Private 10.0.11.0/24NAT gatewayENI + EIPALB nodeENI 10.0.0.12EC2 / EKS podENI 10.0.10.25Interface endpointENI 10.0.10.80NAT gatewayENI + EIPALB nodeENI 10.0.1.17RDS standbyENI 10.0.11.40Lambda in VPCHyperplane ENI

Everything that has an IP in a VPC has an ENI: EC2, Lambda in a VPC, RDS, ALB nodes, NAT gateways, interface endpoints, Resolver endpoints, Transit Gateway attachments.

Mental model

Three questions for every flow

Ask them in this order. Most "network issues" are answered by the first two.

1 · Does the name resolve?

  • Right IP from the right resolver?
  • Private hosted zone associated with this VPC?
  • Resolver rule forwarding to on-premises?
  • Tool: dig, Resolver query logs.
→

2 · Is there a route?

  • Source subnet route table → target.
  • Every hop: TGW route table, VIF, BGP.
  • And the return path.
  • Tools: Reachability Analyzer, Route Analyzer.
→

3 · Is it allowed?

  • SG on both ENIs, NACL on both subnets.
  • Network Firewall, endpoint policy, resource policy.
  • OS firewall and the app listening.
  • Tool: Flow Logs (ACCEPT / REJECT).

Then check the size: MTU mismatches pass the TCP handshake and fail on the first large packet.

Service map

Which AWS networking service answers which need

The rest of the deck goes service by service; this is the map.

NeedServicesChapter
Network for workloadsVPC, subnets, route tables, IGW, NAT, SG, NACL, IPAM1
Reach AWS services privatelyGateway / interface endpoints, PrivateLink, VPC Lattice2
Connect many VPCs and accountsPeering, Transit Gateway, Cloud WAN, RAM, VPC sharing3
Connect on-premisesSite-to-Site VPN, Direct Connect, DX gateway, Client VPN4
Name thingsRoute 53 public / private zones, VPC Resolver, DNS Firewall5
Spread and accelerate trafficALB, NLB, GWLB, CloudFront, Global Accelerator, API Gateway6
Filter and encryptWAF, Shield, Network Firewall, Firewall Manager, Verified Access, ACM7
See and fixFlow Logs, Traffic Mirroring, Reachability Analyzer, Network Manager8
The exam

AWS Certified Advanced Networking – Specialty (ANS-C01)

Recommended: 5+ years of networking, 2+ years of cloud and hybrid networking.

Content domains

DomainWeightMain chapters
1 Network Design30%2, 3, 4, 5, 6, 8
2 Network Implementation26%1, 3, 4, 5, 6, 7, 8, 9
3 Network Management and Operation20%1, 2, 3, 4, 5, 6, 8, 10
4 Network Security, Compliance, and Governance24%1, 5, 7, 8, 9

Format

  • 65 questions: 50 scored + 15 unscored (not marked).
  • Types: multiple response and matching.
  • Matching needs every pair right for credit.
  • Scaled score 100–1,000; pass at 750.
  • Compensatory: you pass on the total, not per domain.
  • No penalty for guessing: answer everything.

Task numbers (for example "Task 1.5") on each slide point to the official exam guide task statements.

What changed in 2025–2026

Recent launches and retirements this deck covers

The exam guide predates many of these; questions still use the older names and patterns.

Added

  • VPC Route Server (Apr 2025), NAT gateway regional mode (Nov 2025).
  • SG referencing over Transit Gateway (Sep 2024) and Cloud WAN (Jun 2025).
  • Cross-Region PrivateLink for AWS services (Nov 2025).
  • Network Firewall native TGW attachment (Jun 2025).
  • Site-to-Site VPN 5 Gbps tunnels (Nov 2025).
  • Route 53 Global Resolver (GA Mar 2026).
  • AWS Interconnect – multicloud (GA with Google Cloud Apr 2026).

Renamed or changed

  • Route 53 Resolver → Route 53 VPC Resolver (Mar 2026).
  • VPC Encryption Controls became paid on 1 Mar 2026.
  • Network Firewall proxy: preview re-launched Aug 2026 as a no-source-preservation mode (us-east-2 only).

Retired

  • AWS App Mesh support ended 30 Sep 2026 (still on the in-scope list).
  • Migrate to ECS Service Connect (ECS) or VPC Lattice.
How to use this deck

A study loop that sticks

Read a part, do its lab, then answer the self-check out loud without notes.

Per part

  • Read the concept slides; redraw the diagrams from memory.
  • Study the "Exam traps" slide: these are the distractors you will see.
  • Answer the scenarios before reading the answer column.

Per chapter

  • Run the matching lab in a sandbox account.
  • Do every "Break it" step, then fix it with the tools from Chapter 8.
  • Answer the self-check questions; revisit anything you hesitate on.

Before the exam

  • Re-read every "Exam traps" and "Exam-style scenarios" slide.
  • Use the keyword cheat sheet in the back matter.
  • Take an official practice question set on AWS Skill Builder.

Navigation: ← → or Space to move, I for Contents, O for overview, T for theme, F for fullscreen. Every Contents line is a link.

Chapter 1

VPC foundations

Tasks 1.6, 2.2, 3.3, 4.1 — addressing, routing, filtering, interfaces and DNS inside one VPC.

Chapter 1 · Part 1.1

IP addressing and subnets

CIDR rules, reserved addresses, IPv6, IPAM, and sizing subnets so you never run out.

Task 1.6 · VPC CIDR rules

What you can and cannot put in a VPC

The primary IPv4 CIDR is fixed for the life of the VPC. Plan it before the first subnet.

Size and count

  • IPv4 VPC and subnet CIDRs: /16 (65,536 addresses) to /28 (16).
  • You cannot resize a CIDR; add a secondary CIDR instead.
  • IPv4 CIDR blocks per VPC: 5 by default, adjustable up to 50 (primary counts).
  • The primary CIDR can never be removed; secondaries can.
  • Each new CIDR adds a local route to every route table.

Forbidden or risky ranges

  • Not allowed: 0.0.0.0/8, 127.0.0.0/8, 169.254.0.0/16, 224.0.0.0/4.
  • Avoid 172.17.0.0/16: Docker bridge default, used by Cloud9 and SageMaker AI.
  • Publicly routable (non-RFC 1918) CIDRs are allowed but only routed privately.
  • 100.64.0.0/10 (CGNAT) is the usual secondary for EKS pods.

Secondary CIDR restrictions

  • Primary in 10/8 → no 172.16/12 or 192.168/16 secondaries.
  • Primary in 172.16/12 → no 172.31.0.0/16 secondary.
  • No 198.19.0.0/16 with RFC 1918 primaries.
  • A new CIDR cannot be the same or larger than an existing route destination.
  • Must not overlap peers or other VPCs on the same DX gateway.
Task 3.3 · Reserved addresses

Every subnet loses five addresses

Usable IPv4 addresses = 2^(32 − prefix) − 5. A /28 has 11, a /24 has 251.

Subnet 10.0.0.0/24

AddressReserved for
10.0.0.0Network address
10.0.0.1VPC router (implicit router, default gateway)
10.0.0.2Amazon DNS (base of the primary VPC CIDR + 2); reserved in every subnet
10.0.0.3Reserved for future use
10.0.0.255Broadcast (not supported, so reserved)

Notes

  • IPv6 subnets also reserve the first four and the last address.
  • With BYOIP IPv4 you can use the network and broadcast addresses of the range.
  • The DNS server lives at the primary CIDR base + 2, even for subnets in a secondary CIDR.
  • CLI normalises CIDRs: 100.68.0.18/18 becomes 100.68.0.0/18.
  • Network Address Usage (NAU): 64,000 units per VPC by default — IPs, ENIs and prefix-list CIDRs all count.
Task 3.3 · Subnet plan

A three-tier, three-AZ address plan that leaves room to grow

Carve by tier, keep a spare /17, and give the busiest tier the biggest subnets.

VPC 10.20.0.0/16 (65,536)AZ aAZ cAZ dSparePublic /2410.20.0.0/24 · ALB, NATPublic /2410.20.1.0/24Public /2410.20.2.0/24App /1910.20.32.0/19 · 8,187 usableApp /1910.20.64.0/19App /1910.20.96.0/19Data /2410.20.8.0/24 · RDS, cacheData /2410.20.9.0/24Data /2410.20.10.0/2410.20.128.0/17reserved for a 4th AZ or EKSEndpoints and TGW attachments can use tiny /28 subnets per AZ (10.20.15.0/28 …).

Public subnets stay small (only load balancers and NAT live there). App subnets get the space because Auto Scaling and EKS pods consume IPs fast.

Task 1.6 · IPv6 in a VPC

IPv6: global addresses, no NAT, different gateways

Every IPv6 address in a VPC is globally unique (GUA). "Private" is achieved with routing, not with address ranges.

Addressing

  • VPC: Amazon-provided /56 (you cannot choose the range), BYOIP, or IPAM pools.
  • Up to five IPv6 CIDRs per VPC, /44 to /60 in steps of /4.
  • Subnets: /44 to /64 in steps of /4; /64 is the norm.
  • Dual-stack subnet: IPv4 + IPv6. IPv6-only subnet: no IPv4 at all (Nitro instances).

Gateways

  • Internet gateway: inbound and outbound IPv6.
  • Egress-only internet gateway: outbound-initiated IPv6 only — the IPv6 "NAT" equivalent, without translation.
  • One egress-only IGW per VPC.
  • Route ::/0 → eigw-… in private subnets.

Talking to IPv4 from IPv6-only

  • Enable DNS64 on the subnet: Resolver synthesises 64:ff9b::/96 AAAA records.
  • Route 64:ff9b::/96 → NAT gateway, which performs NAT64.
  • Works to IPv4 in the VPC, on-premises, or the internet.
  • Resource-based hostnames are required in IPv6-only subnets.
Task 1.6 · DNS64 + NAT64

How an IPv6-only pod reaches an IPv4-only API

DNS64 rewrites the answer; NAT64 rewrites the packet. Both are needed.

VPC (IPv6-only subnet, DNS64 on)Pod / instance2600:1f14:…::10VPC ResolverDNS64NAT gatewayNAT64 → public IPv4api.example.comIPv4 only: 198.51.100.71 AAAA api.example.com?2 64:ff9b::c633:64073 dst 64:ff9b::c633:6407 → nat-…4 src EIP → 198.51.100.7Route table: 64:ff9b::/96 → nat-0abcRoute table: ::/0 → eigw-0def (native IPv6)
Task 2.4 · Amazon VPC IPAM

IPAM: allocate CIDRs from pools instead of spreadsheets

IPAM plans, allocates and audits IP space across Regions and accounts. VPCs request a CIDR from a pool.

Top pool 10.0.0.0/8organizationap-northeast-1 pool10.0.0.0/12us-east-1 pool10.16.0.0/12prod10.0.0.0/14dev10.4.0.0/14prod10.16.0.0/14dev10.20.0.0/14Allocation rules on a pool: min/max/default netmask, required tags.

What it does

  • Pools are hierarchical: top → Regional (locale) → environment.
  • VPC create with --ipv4-ipam-pool-id and a netmask: no overlap by design.
  • Monitors every CIDR: overlapping, non-compliant, unmanaged.
  • Shares pools to accounts or OUs with AWS RAM.
  • Integrates with AWS Organizations (delegated admin account).
  • Also manages public IPv4 (Amazon-provided contiguous blocks, BYOIP) and IPv6.
  • Two tiers: Free and Advanced; organization-wide pools and monitoring need Advanced.
Task 3.3 · IP exhaustion

Running out of IPs: causes and fixes

Most often hit by EKS with the VPC CNI, where every pod takes a VPC IP.

Symptoms

  • Pods stuck ContainerCreating: "failed to assign an IP address".
  • Auto Scaling launches fail: InsufficientFreeAddressesInSubnet.
  • Lambda in VPC, ALB scaling or interface endpoint creation fails.
  • CloudWatch: AvailableIpAddressCount falling (subnet metric via IPAM / describe-subnets).

Fixes

  • Add a secondary CIDR (e.g. 100.64.0.0/16) and new subnets.
  • EKS custom networking: pods in the secondary CIDR, nodes in the primary.
  • Prefix delegation: assign /28 prefixes to ENIs to pack more pods per node.
  • IPv6 EKS cluster: pods get IPv6, IPv4 problem disappears.
  • Shrink warm pools (WARM_IP_TARGET, MINIMUM_IP_TARGET).

Routing the secondary CIDR

  • The CGNAT range need not be routable on-premises.
  • Pods reach on-premises via SNAT to the node IP (VPC CNI default) or a private NAT gateway.
  • This "non-routable CIDR" pattern is covered in Chapter 3 (overlapping IPs).
Task 2.1 · Build it

CLI: VPC from an IPAM pool, a dual-stack subnet, and a secondary CIDR

Every command maps to a concept on the previous slides.

# VPC from IPAM (no hand-picked CIDR) + Amazon IPv6 /56
aws ec2 create-vpc --ipv4-ipam-pool-id ipam-pool-0a1b \
  --ipv4-netmask-length 16 --amazon-provided-ipv6-cidr-block \
  --tag-specifications 'ResourceType=vpc,Tags=[{Key=Name,Value=lab}]'

# Dual-stack subnet in one AZ
aws ec2 create-subnet --vpc-id vpc-0abc --availability-zone-id apne1-az1 \
  --cidr-block 10.20.32.0/19 --ipv6-cidr-block 2406:da14:aa:ab00::/64
aws ec2 modify-subnet-attribute --subnet-id subnet-0123 \
  --assign-ipv6-address-on-creation

# Out of space? add a secondary CIDR
aws ec2 associate-vpc-cidr-block --vpc-id vpc-0abc \
  --cidr-block 100.64.0.0/16

# How many IPs are left per subnet
aws ec2 describe-subnets --filters Name=vpc-id,Values=vpc-0abc \
  --query 'Subnets[].[SubnetId,CidrBlock,AvailableIpAddressCount]' \
  --output table

Notes

  • Use AZ IDs (apne1-az1), not names: ap-northeast-1a maps to different physical AZs in different accounts.
  • AZ IDs matter for cross-account designs (shared subnets, PrivateLink, AZ-local routing).
  • --assign-ipv6-address-on-creation gives new ENIs an IPv6 address automatically.
  • An IPv6-only subnet: create-subnet --ipv6-native with no --cidr-block.
Exam traps

Addressing traps

Wrong answers that look right.

Resizing

  • You cannot enlarge a VPC or subnet CIDR. Add a secondary CIDR and new subnets.

Mixed RFC 1918

  • A 10/8 VPC cannot take a 192.168/16 secondary. Use another 10.x block or 100.64/10.

IPv6 "private"

  • There is no private IPv6 range in a VPC. Outbound-only = egress-only IGW, not NAT gateway.

NAT64 alone

  • NAT64 without DNS64 does nothing: the client never gets a 64:ff9b:: address.

AZ names

  • us-east-1a in two accounts can be different AZs. Match on AZ ID.

The 5 reserved

  • A /28 subnet holds 11 hosts, not 14 or 16.
Exam-style scenarios

Addressing scenarios

Scenario → answer → why.

ScenarioAnswerWhy
EKS pods exhaust a 10.1.0.0/16 VPC; on-premises has no free 10.x spaceSecondary CIDR 100.64.0.0/16 + custom networking (or IPv6 cluster)CGNAT range need not be routable on-premises
50 teams create VPCs; CIDRs keep overlappingIPAM with Regional and environment pools shared via RAMAllocation from pools prevents overlap; IPAM reports non-compliance
IPv6-only workload must call an IPv4-only SaaSDNS64 on the subnet + 64:ff9b::/96 route to a NAT gatewayDNS64 synthesises, NAT64 translates
Private IPv6 instances must reach the internet but not be reachableEgress-only internet gateway, ::/0 routeStateful outbound-only for IPv6
Need a subnet for 30 hosts with the smallest CIDR/26 (59 usable)/27 gives 27 usable after the 5 reserved
Chapter 1 · Part 1.2

Route tables and gateways

Longest prefix match, internet and NAT gateways, ingress routing, prefix lists and Route Server.

Task 3.1 · Route tables

How the VPC router decides

Every subnet is associated with exactly one route table. The most specific matching route wins.

Rules

  • Main route table: used by subnets with no explicit association. Keep it private.
  • Custom route tables: associate explicitly; a table can serve many subnets.
  • Longest prefix match first; for equal prefixes, static beats propagated.
  • local route for each VPC CIDR is always present.
  • You can add routes more specific than local (e.g. a subnet CIDR → firewall ENI) for middlebox inspection.
  • Routes per table: 500 non-propagated by default (max 1,000); propagated routes: 100, not adjustable.

Route targets

TargetUsed for
igw-Internet (public IPv4 and IPv6)
eigw-Outbound-only IPv6
nat-IPv4 outbound via NAT; NAT64
tgw- / vgw- / pcx-Transit Gateway, VPN/DX, peering
vpce-Gateway endpoint (prefix list) or GWLB endpoint
eni- / instanceAppliance, NAT instance, firewall
cagw- / lgw-Wavelength carrier, Outposts local gateway
Task 3.1 · Public, private, isolated

What makes a subnet public is one route

A subnet is public if its route table has 0.0.0.0/0 (or ::/0) → internet gateway. Nothing else.

VPC 10.0.0.0/16 · one AZ shownPublic subnetPrivate subnetIsolated subnetNAT gateway / ALBApp instanceDatabase10.0.0.0/16 → local0.0.0.0/0 → igw-0a110.0.0.0/16 → local0.0.0.0/0 → nat-0b210.0.0.0/16 → localpl-s3 → vpce-gw (optional)via NAT

An instance in a public subnet still needs a public IPv4 or Elastic IP to use the IGW. Public IPv4 addresses are billed per hour (since Feb 2024), in use or not.

Task 1.6 · Internet gateway

Internet gateway: horizontally scaled 1:1 NAT

The IGW is not a box with a throughput limit; it maps public IPv4 ↔ private IPv4 and passes IPv6 untouched.

How it works

  • One IGW per VPC; attach, then route to it.
  • Outbound: rewrites source private IP → associated public IP / EIP.
  • Inbound: rewrites destination public IP → private IP.
  • Instances never see their public IPv4 on the NIC (ip addr shows only the private IP).
  • IPv6: no translation.

Limits that do apply

  • Instance bandwidth via IGW: up to 5 Gbps for instances with fewer than 32 vCPUs; 50% of instance bandwidth above that.
  • Max MTU 1500 through the IGW.
  • Security groups and NACLs still filter.
  • VPC Block Public Access can block all IGW traffic Region-wide.

Public IPv4 choices

  • Auto-assigned public IP: changes on stop/start.
  • Elastic IP: static, re-mappable, billed hourly.
  • BYOIP or Amazon-provided contiguous IPv4 blocks via IPAM.
  • Prefer IPv6 or a load balancer to cut public IPv4 count.
Task 3.3 · NAT gateway

Zonal NAT gateway: the numbers that matter

Managed, redundant inside one AZ. Put one per AZ and route each AZ to its own.

Capacity

  • 5 Gbps baseline, scales automatically to 100 Gbps.
  • 1 Mpps, scales to 10 Mpps; beyond that packets drop.
  • 55,000 simultaneous connections per IP to each unique destination (IP + port + protocol).
  • Up to 8 IPv4 addresses per gateway (1 primary + 7 secondary) to raise that.
  • EIPs per public NAT gateway: 2 by default, adjustable to 8.
  • Need more: split workloads across subnets with separate NAT gateways.

Behaviour

  • TCP, UDP and ICMP. Also NAT64.
  • No security group on a NAT gateway; filter with NACLs (it uses ports 1024–65535).
  • MTU up to 8500; PMTUD supported; MSS clamping on all packets.
  • Metric to watch: ErrorPortAllocation (port exhaustion), PacketsDropCount.
  • Connections through it are always tracked.

Routing rules

  • Cannot route into a NAT gateway over VPC peering (client → peering → NAT → internet: no).
  • NAT → peering → destination works ("return to sender").
  • From VPN/DX: not via a VGW; works via Transit Gateway.
  • Private NAT gateway: no EIP, translates to its private IP — used for overlapping CIDRs.
Task 3.3 · NAT per AZ

Zonal NAT design: one gateway per AZ, AZ-local routes

A single shared NAT gateway is a single-AZ failure point and adds inter-AZ data charges.

Internet gatewayVPCAZ aAZ cNAT gw apublic subnet aNAT gw cpublic subnet cApp art-a: 0/0 → nat-aApp crt-c: 0/0 → nat-cAZ-localAZ-local✗ avoid: App c → nat-a (cross-AZ)

Regional NAT gateway (next slide) removes the per-AZ wiring: one ID, follows workloads across AZs.

Task 3.3 · Regional NAT gateway

Regional availability mode (Nov 2025)

One NAT gateway ID for the whole VPC that expands to every AZ where you have ENIs.

ZonalRegional
PlacementOne AZ, in a public subnetNo subnet; spans AZs automatically
RoutesOne route table per AZ → its own NATSame route (0/0 → nat-…) in every AZ
IPsUp to 8 per gatewayUp to 32 per AZ
Private NATYesNo — use zonal for private NAT
Own route tableUses its subnet's tableAWS creates one with a route to the IGW
New AZYou create a new gatewayExpands itself (can take up to 60 min; cross-AZ meanwhile)
aws ec2 create-nat-gateway --vpc-id vpc-0abc \
  --availability-mode regional
# manual mode: you place IPs per AZ
aws ec2 associate-nat-gateway-address \
  --nat-gateway-id nat-0123 \
  --availability-zone us-east-1b \
  --allocation-ids eipalloc-0456

Modes

  • Automatic: AWS manages IPs and AZ expansion (recommended).
  • Manual: you manage IPs and expansion per AZ.
  • Converting zonal → regional resets connections: use a maintenance window.
  • Not in GovCloud, China or constrained AZs.
Task 3.3 · NAT options

NAT gateway vs NAT instance vs no NAT

Pick by availability, throughput, control and cost.

NAT gatewayNAT instanceAvoid NAT
AvailabilityManaged, redundant in AZ (regional mode spans AZs)You script failover (or Route Server)—
Throughput5 → 100 Gbps autoInstance bandwidth—
Security groupNoYes—
Port forwarding / bastionNoYes—
SetupCreate + routeDisable source/dest check, iptables MASQUERADE—
CostHourly + per-GB processingInstance cost onlyGateway endpoints for S3/DynamoDB are free; interface endpoints for other AWS APIs
Pick whenDefault for productionTiny labs, custom filteringMost traffic goes to AWS services
Task 4.1 · Ingress routing

Gateway route tables: steer inbound traffic through an appliance

Associate a route table with the IGW (or VGW). Its routes override where inbound packets go inside the VPC.

InternetIGWedge rt: 10.0.2.0/24 → eni-fwVPC 10.0.0.0/16Appliance subnet 10.0.1.0/24Workload subnet 10.0.2.0/24Firewall ENIsource/dest check offWeb serverrt: 0/0 → eni-fw1231 to 10.0.2.10 → eni-fw · 2 inspected, forwarded · 3 reply via 0/0 → eni-fw → IGW

Targets in a gateway route table: an ENI or a GWLB endpoint, for destinations inside the VPC CIDR. The appliance subnet routes 0/0 → IGW, and the return path must cross the appliance too (symmetry). Used for third-party firewalls and GWLB designs (Chapter 7).

Task 3.1 · Prefix lists

Managed prefix lists: name a set of CIDRs once, reference it everywhere

Used in route tables, security groups, Transit Gateway route tables and Network Firewall rules.

Customer-managed

  • Your CIDR sets, e.g. "on-prem-dc", "office-egress".
  • Versioned; edit once, every reference updates.
  • Share with AWS RAM across accounts.
  • Max entries (not used entries) counts toward quotas: a 20-entry list in an SG rule = 20 rules.
  • Up to 1,000 entries per list (default quota).

AWS-managed

  • com.amazonaws.<region>.s3 / .dynamodb: gateway endpoint ranges.
  • com.amazonaws.global.cloudfront.origin-facing: lock an ALB to CloudFront.
  • com.amazonaws.<region>.ec2-instance-connect for EC2 Instance Connect.
  • Read-only; AWS updates them.
  • CloudFront list is large: watch the SG rule count.
aws ec2 create-managed-prefix-list \
  --prefix-list-name on-prem \
  --max-entries 10 \
  --address-family IPv4 --entries \
  Cidr=192.168.0.0/16,Description=dc1 \
  Cidr=172.20.0.0/14,Description=dc2

aws ec2 create-route --route-table-id rtb-0a \
  --destination-prefix-list-id pl-0123 \
  --transit-gateway-id tgw-0abc

aws ec2 authorize-security-group-ingress \
  --group-id sg-0a --ip-permissions \
  'IpProtocol=tcp,FromPort=443,
   ToPort=443,PrefixListIds=
   [{PrefixListId=pl-0123}]'
Task 1.5 · VPC Route Server

VPC Route Server (GA Apr 2025): BGP into VPC route tables

Appliances advertise routes over BGP; Route Server installs the best next hop in your route tables and fails over automatically.

VPCFirewall A (active)MED 0 · 192.0.2.0/24Firewall B (standby)MED 100 · 192.0.2.0/24Route server endpointsBGP + BFD peersSubnet route table192.0.2.0/24 → eni-AA fails → BFD detects→ route now → eni-B

Pieces and limits

  • Route server → associate to a VPC → endpoints in subnets → peers (your appliances) over BGP.
  • Propagation: choose which route tables (subnet, unassociated VPC tables, IGW edge table) receive routes.
  • Liveness: BFD (fast) or BGP keepalive.
  • Flow: RIB → best path (FIB) → route table update.
  • Not for VGW route tables; for Transit Gateway use TGW Connect.
  • Limits: 100 routes per route server (and per peer), 2 endpoints per subnet per route server.
  • Replaces Lambda scripts that rewrote routes for "floating IP" failover.
Exam traps

Routing and gateway traps

Wrong answers that look right.

NAT over peering

  • Peered VPC cannot use your NAT gateway or IGW (no edge-to-edge routing). Give it its own NAT or use TGW.

NAT from VGW

  • On-premises traffic via a VGW cannot be routed to a NAT gateway; via Transit Gateway it can.

SG on NAT gateway

  • There is none. Filter with NACLs or the instance SGs.

Main route table

  • New subnets land on the main table. If it has 0/0 → igw, they are public by accident.

Regional NAT

  • Cannot do private NAT. Overlapping-CIDR designs still use zonal private NAT gateways.

Propagated routes

  • 100 propagated routes per VPC route table, not adjustable. Summarize or advertise a default route.
Exam-style scenarios

Routing and gateway scenarios

Scenario → answer → why.

ScenarioAnswerWhy
NAT gateway shows ErrorPortAllocation when calling one SaaS endpointAdd secondary IPs to the NAT gateway (up to 8 IPs: 1 primary + 7 secondary), or regional modeEach IP adds 55,000 connections per destination
All inbound internet traffic must pass a third-party IDS before reaching web serversIGW gateway route table → appliance ENI / GWLB endpointIngress routing steers traffic entering the VPC
Active/standby firewall instances need sub-second route failover without LambdaVPC Route Server with BGP + BFDInstalls next hop dynamically from BGP
Private subnets in 3 AZs; must survive an AZ loss with least route managementRegional NAT gatewayOne ID, expands to every AZ with workloads
Allow SSH only from 40 office CIDRs, used by 30 security groupsCustomer-managed prefix list referenced by the SGsEdit once; but 40 max entries count as 40 rules per SG
Chapter 1 · Part 1.3

Security groups and network ACLs

Stateful vs stateless filtering, connection tracking, SG referencing and VPC Block Public Access.

Task 4.1 · SG vs NACL

Security groups and network ACLs side by side

Know the mechanics, not just "stateful vs stateless".

Security groupNetwork ACL
Attached toENI (instance, RDS, ALB, Lambda, endpoint…)Subnet (every ENI in it)
StateStateful: replies allowed by connection trackingStateless: allow the return direction explicitly
RulesAllow only; all rules evaluated togetherAllow and deny; numbered, lowest first, first match wins
DefaultNew SG: no inbound, all outboundDefault NACL allows all; a new custom NACL denies all
SourcesCIDR, prefix list, other SGsCIDR only
Quotas (default)60 inbound + 60 outbound per SG (IPv4 and IPv6 counted separately); 5 SGs per ENI (max 16); rules × SGs ≤ 1,00020 rules per direction (max 40)
Use forPrimary micro-segmentationCoarse subnet guardrail, explicit deny of a range, cutting live connections
Task 4.1 · Evaluation order

The path a packet takes through both filters

Inbound: subnet NACL in → SG in. Reply: SG (tracked, allowed) → NACL out (must allow ephemeral ports).

Client203.0.113.9:51544Subnet 10.0.1.0/24NACL inboundallow tcp 443SG inboundallow 443 from 0/0Web server10.0.1.20:443SG (tracked)reply auto-allowedNACL outboundallow 1024–65535Missing outbound ephemeral rule = SYN arrives, SYN-ACK dropped: client times out.
Task 4.1 · Ephemeral ports

NACL return traffic: whose ephemeral range?

The reply goes to the client's source port. NACLs must allow the range the client actually uses.

Typical source port ranges

ClientEphemeral range
Many Linux kernels (incl. Amazon Linux)32768–61000
Windows Server 2008+49152–65535
NAT gateway1024–65535
Elastic Load Balancing1024–65535
Safe NACL rule1024–65535

Design rules

  • Inbound to a public subnet from the internet: allow service port in, 1024–65535 out.
  • Outbound from private instances via NAT: allow 443 out, 1024–65535 in.
  • Remember NACLs apply between subnets inside the VPC too.
  • Leave a gap in rule numbers (100, 200, …) for later inserts.
  • Deny rules must have a lower number than the allow they override.
  • NACLs do not filter traffic to the Amazon DNS server, IMDS, Time Sync or DHCP.
Task 4.2 · Connection tracking

Why removing an SG rule may not cut a live session

Tracked flows keep working until they go idle. Untracked flows stop immediately.

Tracked vs untracked

  • Untracked: a TCP/UDP rule allows 0.0.0.0/0 (or ::/0) and the other direction allows all to 0/0 on all ports.
  • Everything else is tracked. ICMP is always tracked.
  • Change a rule → tracked connections continue; untracked ones drop at once.
  • Always tracked: through NAT gateway, NLB, Global Accelerator, interface endpoints, Network Firewall, egress-only IGW, Lambda ENIs, DynamoDB gateway endpoint.

Timeouts (per ENI, Nitro)

  • TCP established idle: 60–432,000 s; default 432,000 s (5 days), or 350 s on Nitro v6 instances.
  • UDP: 30–60 s (default 30). UDP stream: 60–180 s (default 180).
  • NLB-tracked flows: 350 s TCP, 120 s UDP.
  • Set with --connection-tracking-specification on the ENI or at launch.

Operational impact

  • To kill an attacker's session now: NACL deny on the subnet.
  • Conntrack table is finite per instance: watch conntrack_allowance_exceeded.
  • Asymmetric routing with tracked flows lowers performance; prefer short TCP timeouts.
  • Long-lived DB pools on Nitro v6: send TCP keepalives under 5 minutes.
Task 4.1 · SG referencing

Reference security groups instead of IPs

An SG rule can name another SG as the source: "allow 5432 from members of sg-app". Membership changes, rules do not.

Where referencing works

PathNotes
Same VPCAlways
Shared VPC (RAM)Participants can reference SGs in the shared VPC
VPC peeringSame Region only; peer SG by ID
Transit Gateway (Sep 2024)Enable on the TGW and the VPC attachments
Cloud WAN (Jun 2025)Enable on the core network attachments

Limits for TGW and Cloud WAN

  • Inbound rules only (no outbound SG references).
  • Same Region only.
  • Not through GWLB or Network Firewall inspection paths — the source must be seen unchanged.
  • Reference by SG ID; the referenced SG must belong to an attached VPC.
  • Before 2024 the only option across TGW was CIDR-based rules.
Task 4.1 · SG patterns

Three-tier with SG chaining

Each tier accepts traffic only from the tier above, by SG ID.

Internetsg-albin: 443 from 0.0.0.0/0sg-appin: 8080 from sg-albsg-dbin: 5432 from sg-app80805432sg-bastion → removeduse Session ManagerNo CIDRs inside the VPC: scaling the app tier needs no rule changes.Outbound: tighten to the next tier only, or leave default all-outbound and rely on inbound rules.
Task 4.1 · VPC Block Public Access

VPC BPA: one switch to block internet paths Region-wide

An account-level (or Organization declarative policy) control that overrides route tables and SGs for IGW / egress-only IGW traffic.

Modes

  • Bidirectional: block all traffic to and from IGWs and egress-only IGWs.
  • Ingress-only: block inbound; NAT gateway and egress-only IGW outbound still work.
  • Exclusions per VPC or subnet: allow-bidirectional or allow-egress (egress applies when BPA is bidirectional).
  • 50 exclusions per account per Region (adjustable).
  • Enforce across accounts with an Organizations declarative policy (it can allow/deny exclusions).

What it blocks / not

  • Blocked: IGW, egress-only IGW, NAT gateway internet egress, internet-facing ALB/NLB, CloudFront VPC origins, Global Accelerator inbound, DX public VIF traffic, GWLB/Network Firewall unless their subnets are excluded.
  • Not affected: Site-to-Site VPN, Client VPN, Transit Gateway, Cloud WAN, Verified Access, Outposts local gateway.
  • Ingress routing via an appliance: exclude the workload subnet.
aws ec2 \
 modify-vpc-block-public-access-options \
 --internet-gateway-block-mode \
   block-bidirectional

aws ec2 \
 create-vpc-block-public-access-exclusion \
 --subnet-id subnet-0pub \
 --internet-gateway-exclusion-mode \
   allow-bidirectional

aws ec2 \
 describe-vpc-block-public-access-options
# test first with Network Access
# Analyzer / Reachability Analyzer
Exam traps

Filtering traps

Wrong answers that look right.

Deny in an SG

  • Security groups cannot deny. To block one IP, use a NACL deny (or WAF / Network Firewall).

Stateless return

  • A NACL allowing 443 in but not 1024–65535 out breaks HTTPS.

Isolation by SG

  • Swapping an instance's SG does not drop tracked connections. NACL deny does.

Cross-Region SG reference

  • Not possible over peering, TGW or Cloud WAN. Use CIDRs or prefix lists.

Prefix list sizing

  • A prefix list counts its max entries against the SG rule quota, even if half empty.

BPA and VPN

  • VPC BPA does not block Site-to-Site VPN or TGW traffic — it is about internet gateways.
Exam-style scenarios

Filtering scenarios

Scenario → answer → why.

ScenarioAnswerWhy
Block a malicious /24 from reaching a public subnet nowNACL deny rule with a low rule numberSGs cannot deny; NACL also cuts live flows
DB must accept only from app instances that scale in and outSG rule source = app tier SG IDMembership-based, no IP maintenance
Security team: no VPC in any account may ever be internet-reachable, except one egress VPCVPC BPA bidirectional via declarative policy + exclusion for the egress VPCOrganization-wide preventive control
Firewall cluster drops connections under load; conntrack_allowance_exceeded risingLower TCP established timeout, bigger instances, or untracked rulesConntrack table full
App in spoke VPC must allow traffic only from SG of a shared-services VPC via TGWEnable SG referencing on TGW and attachments; inbound rule by SG IDSupported same Region, inbound only
Chapter 1 · Part 1.4

Network interfaces and instance networking

ENIs, prefix delegation, ENA / ENA Express / EFA, bandwidth limits and MTU.

Task 3.3 · ENI

The elastic network interface is the unit of networking

An ENI carries IPs, MAC, security groups and source/dest check. It lives in one subnet and one AZ.

Attributes

  • Primary private IPv4 (fixed for the ENI's life) + secondary private IPv4s.
  • Optional public IPv4 / EIP per private IP; IPv6 addresses; IPv4/IPv6 prefixes.
  • Up to 5 SGs by default; MAC address; source/destination check flag.
  • Connection-tracking timeouts; ENA Express setting (per attachment).

Attach and move

  • Primary ENI (eth0) cannot be detached.
  • Secondary ENIs: hot (running), warm (stopped) or cold (at launch) attach.
  • Moving an ENI moves its IPs and MAC: a classic licence / failover trick.
  • ENIs per instance and IPs per ENI depend on instance type.
  • Multi-homed instances: ENIs in different subnets of the same AZ.

Requester-managed ENIs

  • Created by services in your subnets: NAT gateway, ALB/NLB, interface endpoints, RDS, Lambda (Hyperplane), EKS control plane, TGW attachments.
  • You see them, can tag them, cannot modify them.
  • They consume subnet IPs — budget for them.
Task 3.3 · Prefix delegation

Prefix delegation: more IPs per ENI slot

Assign an IPv4 /28 (16 addresses) or IPv6 /80 to an ENI instead of single secondary IPs.

Why

  • Each ENI has a fixed number of IP slots per instance type.
  • One slot can hold a /28 prefix: 16× more addresses per slot.
  • EKS VPC CNI with ENABLE_PREFIX_DELEGATION=true raises max pods per node.
  • Faster pod start: the CNI hands out IPs from an already attached prefix.

Caveats

  • The subnet needs contiguous free /28 blocks; fragmented subnets fail allocation.
  • Use subnet CIDR reservations (prefix type) to keep space for prefixes.
  • Nitro instances only.
  • Many pods per node still hit instance limits (pps, conntrack).
# reserve space in the subnet for /28 prefixes
aws ec2 \
 create-subnet-cidr-reservation \
  --subnet-id subnet-0app \
  --cidr 10.20.48.0/20 \
  --reservation-type prefix

# assign two /28 prefixes to an ENI
aws ec2 assign-private-ip-addresses \
  --network-interface-id eni-0abc \
  --ipv4-prefix-count 2

# EKS
kubectl set env ds aws-node \
  -n kube-system \
  ENABLE_PREFIX_DELEGATION=true
Task 4.1 · Source/destination check

Source/destination check: why an appliance must turn it off

By default an ENI drops packets that are neither from nor to its own IPs. Routers, NAT instances and firewalls forward other hosts' traffic.

App 10.0.2.10dst 8.8.8.8NAT / firewall ENIcheck ON → dropsNAT / firewall ENIcheck OFF → forwardsNext hopIGW / TGWsrc ≠ me, dst ≠ meaws ec2 modify-network-interface-attribute --network-interface-id eni-… --no-source-dest-check

GWLB-based and Network Firewall designs avoid this by using endpoints instead of instance ENIs as route targets.

Task 3.3 · ENA, ENA Express, EFA

Pick the network adapter for the job

All current-generation instances use ENA. ENA Express and EFA are add-ons for specific workloads.

ENAENA ExpressEFA
WhatEnhanced networking (SR-IOV) driver, default on NitroENA + AWS SRD transport: multipath, congestion-awareENA + OS-bypass (libfabric) for HPC/ML
Single-flow bandwidth5 Gbps (10 Gbps in a cluster placement group)Up to 25 Gbps (same Region, up to instance limit)OS-bypass traffic, very low latency
RequirementsNitro instance, ENA driverSupported large types on both sides, enabled on both attachments, no middlebox in pathSupported types, EFA ENI at launch, MPI / NCCL apps
TrafficEverythingTCP; UDP optional; same or cross-AZ in a RegionMPI, NCCL collective ops
Use forGeneral workloadsBig data shuffles, storage, media between instancesTightly coupled HPC, distributed training

ENA Express falls back to plain ENA silently if either side is not configured or the path crosses a middlebox (NLB, NAT, firewall).

Task 3.3 · Bandwidth limits

Where instance throughput really stops

Published "up to N Gbps" is the best case. Several other ceilings apply first.

Limits

  • Single flow (5-tuple): 5 Gbps outside a cluster placement group; 10 Gbps inside; 25 Gbps with ENA Express.
  • Via IGW: 5 Gbps if < 32 vCPUs; 50% of instance bandwidth above.
  • "Up to" sizes (≤ 16 vCPU) burst on network I/O credits for 5–60 min, then drop to baseline.
  • Other ceilings: pps, conntrack entries, link-local 1024 pps.
  • GRE / IPsec flows are hashed on 3-tuple: one tunnel = one flow.

Prove it with ENA metrics

  • bw_in_allowance_exceeded / bw_out_allowance_exceeded
  • pps_allowance_exceeded
  • conntrack_allowance_exceeded, conntrack_allowance_available
  • linklocal_allowance_exceeded (DNS / IMDS / NTP)
  • Read with ethtool -S eth0; publish via CloudWatch agent.

Get more

  • Multiple flows (parallel streams, MPTCP).
  • Cluster placement group for single-flow 10 Gbps + low latency.
  • ENA Express between supported instances.
  • Bigger instance or network-optimized "n" families.
  • Jumbo frames inside the VPC.
Task 3.3 · MTU

MTU along each path

Inside a VPC the MTU is 9001. Every path that leaves the VPC is smaller.

Maximum MTU by path

PathMTU
Within a VPC, intra-Region peering9001
Direct Connect private VIF9001 (when enabled)
Transit Gateway (VPC, DX, peering attachments)8500
Inter-Region VPC peering8500
Direct Connect transit VIF8500 (when enabled)
NAT gateway8500
Internet gateway, public VIF, between Regions without peering or TGW1500
Site-to-Site VPN (inside the tunnel)Max 1446 / MSS 1406; 1422 / 1382 with AES-CBC + NAT-T

Path MTU discovery

  • IPv4: router sends ICMP type 3 code 4 "fragmentation needed"; IPv6: ICMPv6 type 2 "packet too big".
  • If an SG or NACL blocks that ICMP, large packets vanish: an MTU black hole.
  • Allow inbound ICMP type 3 code 4 (or ICMPv6 type 2) where the path shrinks.
  • Over VPN the effective TCP MSS is lower again because of IPsec overhead.
  • Mixed paths: lower MTU per route (ip route … mtu 1500) or a second ENI.
Task 3.2 · MTU black hole

Diagnose an MTU problem in five commands

Classic symptom: SSH and small HTTP work; large downloads or TLS handshakes with big certificates hang.

ip link show eth0 | grep mtu        # 9001 on most Nitro instances
tracepath -n 10.50.0.20             # reports pmtu per hop

# largest unfragmented payload: MTU - 28 (IPv4 + ICMP headers)
ping -M do -s 8972 10.50.0.20       # tests 9000-byte packets
ping -M do -s 1472 10.50.0.20       # tests 1500-byte packets

# fix for one destination (e.g. on-prem via VPN)
sudo ip route add 10.50.0.0/16 via 10.0.1.1 mtu 1500

# or the whole interface
sudo ip link set dev eth0 mtu 1500

Reading the results

  • 8972 fails but 1472 works → path MTU is 1500 somewhere.
  • No ICMP error at all, just silence → ICMP is filtered: open type 3 code 4.
  • Make the change persistent via /etc/dhcp/dhclient.conf / systemd-networkd, not just ip link.
  • In containers, the CNI MTU must match the host path too.
  • Chapter 8 repeats this as a troubleshooting case study.
Exam traps

Interface and performance traps

Wrong answers that look right.

"Bigger instance" for one flow

  • A single TCP stream outside a placement group caps at 5 Gbps regardless of size. Use more flows, placement group or ENA Express.

EFA for web apps

  • EFA is for MPI / NCCL. General throughput is ENA or ENA Express.

Jumbo to the internet

  • IGW forwards only 1500-byte packets. Jumbo frames help only inside AWS paths.

Moving eth0

  • The primary ENI cannot be detached. Fail over a secondary ENI or use Route Server / NLB.

Source/dest check

  • A NAT or firewall instance that "receives nothing" usually still has the check on.

ENA Express via NLB

  • Middleboxes in the path disable SRD; it falls back silently.
Exam-style scenarios

Interface and performance scenarios

Scenario → answer → why.

ScenarioAnswerWhy
HPC job needs lowest inter-node latency for MPIEFA + cluster placement groupOS-bypass and close physical placement
Large file copy between two m7i.16xlarge in different AZs tops out at 5 GbpsParallel streams, or enable ENA Express on bothSingle-flow limit; SRD raises it to 25 Gbps
EKS nodes hit max pods long before CPUVPC CNI prefix delegation/28 per slot multiplies IPs per ENI
Uploads to on-premises over VPN hang, small requests fineAllow ICMP fragmentation-needed; set MTU/MSS for the VPN pathVPN tunnels carry at most 1446-byte packets (AES-GCM, no NAT-T) and do not support PMTUD; clamp MSS to ≤1406 (1382 with AES-CBC + SHA-256 + NAT-T)
Licensed app is tied to a MAC address and must survive instance failureSecondary ENI moved to a standby instanceENI keeps its MAC and IPs
Chapter 1 · Part 1.5

DHCP options and VPC DNS basics

DHCP option sets, DNS attributes, the Route 53 VPC Resolver, hostnames and link-local services.

Task 2.2 · DHCP option sets

DHCP option sets: what instances are told at boot

One option set per VPC at a time. Option sets are immutable: create a new one and associate it.

Options

OptionMeaning
domain-nameSearch suffix (e.g. corp.example.com). Use a domain you control; one value for Windows.
domain-name-serversAmazonProvidedDNS or up to 4 IPv4 + 4 IPv6 custom servers. Do not mix both.
ntp-serversUp to 4 IPv4 + 4 IPv6; Amazon Time Sync is 169.254.169.123 / fd00:ec2::123.
netbios-name-servers, netbios-node-typeWindows; node type 2 (P-node) recommended.
ipv6-address-preferred-lease-time140 s default, up to ~68 years.

Behaviour

  • Existing instances pick up a new option set at their next DHCP renewal (within hours) — no reboot needed.
  • No option set at all disables name resolution in the VPC.
  • Pointing domain-name-servers to on-premises DNS bypasses Route 53 private zones — use Resolver endpoints instead (Chapter 5).
  • Custom DNS servers must resolve AWS names themselves (forward to VPC+2).
Task 1.2 · DNS attributes

enableDnsSupport and enableDnsHostnames

Two VPC attributes control whether the Amazon DNS server works and whether instances get DNS names.

enableDnsSupportenableDnsHostnamesResult
truetrueResolver works; instances with public IPs get public DNS names; private hosted zones resolve. Needed for PHZ and interface endpoint private DNS.
truefalseResolver works; no public DNS hostnames for instances.
falseanyNo Amazon DNS server at VPC+2 or 169.254.169.253. Custom DNS only.

Defaults and CLI

  • Default VPC: both true. New VPC via CLI/API: enableDnsSupport true, enableDnsHostnames false.
  • Console "VPC and more" wizard turns both on.
  • aws ec2 modify-vpc-attribute --vpc-id vpc-0a --enable-dns-hostnames "{\"Value\":true}"
  • One attribute per call.
Task 1.2 · Amazon DNS server

Route 53 VPC Resolver: the DNS server built into every VPC

Renamed from "Route 53 Resolver" in Mar 2026. Same addresses, same behaviour.

VPCInstanceresolv.conf → 10.0.0.2VPC Resolver10.0.0.2 · 169.254.169.253Private hosted zonesForwarding rules → on-premPublic DNS (recursive)queryOrder of answers:1 forwarding rule match2 private hosted zone3 VPC / AWS internal names4 public internet recursion

Facts

  • Addresses: primary CIDR base + 2, 169.254.169.253, and fd00:ec2::253 (Nitro).
  • SGs and NACLs do not filter traffic to it; use DNS Firewall instead.
  • 1,024 packets/s per ENI to link-local services (DNS + IMDS + NTP + Windows licensing), not adjustable.
  • Over the limit: drops show in linklocal_allowance_exceeded. Cache locally (systemd-resolved, nscd, NodeLocal DNSCache in EKS).
  • Not reachable from on-premises directly: use an inbound Resolver endpoint.
Task 1.2 · Hostnames

IP-based vs resource-based hostnames

Subnet setting --private-dns-hostname-type-on-launch decides the guest OS hostname for new instances.

IP nameResource name
Formatip-10-0-1-20.ap-northeast-1.compute.internali-0123456789abcdef0.ap-northeast-1.compute.internal
IPv6-only subnetsNot availableRequired
Changes when IP changesYesNo
DNS recordsA recordOptional A and AAAA records (enable-resource-name-dns-a/aaaa-record)

Notes

  • us-east-1 uses ec2.internal instead of <region>.compute.internal.
  • Public DNS names (ec2-203-0-113-25.compute-1.amazonaws.com) resolve to the private IP inside the VPC and the public IP outside.
  • That split-horizon behaviour is why EC2 public names keep traffic inside AWS when used from the VPC.
  • Kubernetes node names follow the hostname type — choose it before building an EKS cluster.
Task 4.1 · Link-local services

Services every instance reaches on link-local addresses

They never leave the host; they share the 1,024 pps limit per ENI.

ServiceIPv4IPv6 (Nitro)
Instance Metadata Service169.254.169.254fd00:ec2::254
Amazon DNS (VPC Resolver)169.254.169.253 (and VPC+2)fd00:ec2::253
Amazon Time Sync (NTP)169.254.169.123fd00:ec2::123

IMDSv2 in practice

  • Session token: PUT /latest/api/token with X-aws-ec2-metadata-token-ttl-seconds (up to 21,600 s).
  • Hop limit: default 1 blocks containers (one extra hop); set 2 for containers that need IMDS, or pass config directly.
  • Require IMDSv2 (HttpTokens=required) to stop SSRF-based credential theft.
  • IPv6 endpoint must be enabled explicitly.
TOKEN=$(curl -sX PUT http://169.254.169.254/latest/api/token -H "X-aws-ec2-metadata-token-ttl-seconds: 300")
curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/network/interfaces/macs/
Exam traps

DHCP and DNS basics traps

Wrong answers that look right.

PHZ not resolving

  • Private hosted zones need both enableDnsSupport and enableDnsHostnames on the VPC.

Custom DNS in DHCP

  • Pointing DHCP to on-premises DNS breaks PHZ and endpoint private DNS unless those servers forward back to VPC+2 via an inbound endpoint.

SG for DNS

  • Opening UDP 53 in an SG does not matter for VPC+2; it is not filtered. A NACL also does not filter it.

DNS throttling

  • Busy hosts hitting 1,024 pps to VPC+2 need local caching, not a quota increase (there is none).

Editing an option set

  • You cannot edit one. Create a new set and associate it.

IMDS from containers

  • Hop limit 1 blocks IMDSv2 from pods. That can be intended (use IRSA / Pod Identity).
Exam-style scenarios

DHCP and DNS basics scenarios

Scenario → answer → why.

ScenarioAnswerWhy
Instances must use corp.example.com as search domain and still resolve a private hosted zoneOption set: domain-name corp.example.com, servers AmazonProvidedDNSKeeps the VPC Resolver in the path
Interface endpoint private DNS name does not resolve in a new VPCEnable enableDnsHostnames (and DNS support)Private DNS for endpoints relies on PHZs
DNS timeouts on a log-shipping fleet under load; linklocal_allowance_exceeded risingLocal DNS cache on each host (or NodeLocal DNSCache)1,024 pps per ENI limit is fixed
IPv6-only subnet; instances need stable namesResource-based hostnames with AAAA recordsIP names are unavailable in IPv6-only subnets
Pods must not read node credentials from IMDSIMDSv2 required + hop limit 1; use Pod Identity / IRSAExtra hop blocks pod access
Self-check

VPC foundations: can you answer these?

Explain the answer out loud and name the exact setting or service.

Addressing and routing

  • Why can a 10.0.0.0/16 VPC not add 192.168.0.0/24?
  • How many usable hosts in a /27, and why?
  • What two features let an IPv6-only host reach an IPv4 API?
  • What makes a subnet public?
  • When must you still use a zonal NAT gateway?

Filtering and interfaces

  • Which flows are untracked, and why does it matter during an incident?
  • What NACL rule fixes "SYN arrives, no reply"?
  • Three limits of SG referencing over Transit Gateway?
  • Why does one TCP stream stop at 5 Gbps?
  • When do you disable source/destination check?

MTU and DNS

  • MTU over TGW, inter-Region peering, VPN and the IGW?
  • Which ICMP message must pass for PMTUD on IPv4?
  • Which two VPC attributes do private hosted zones need?
  • Three addresses of the Amazon DNS server?
  • What does the 1,024 pps link-local limit include?
Chapter 2

Private access to services

Tasks 1.6, 3.1, 3.3 — reach AWS services, partner services and your own services without the internet.

Chapter 2 · Part 2.1

VPC endpoints

Gateway endpoints, interface endpoints, endpoint policies, and private DNS for AWS services.

Task 1.6 · Why endpoints

Without an endpoint, "private" traffic to S3 leaves through NAT

An instance in a private subnet that calls an AWS API uses the public service endpoint. The path decides cost and exposure.

VPC 10.0.0.0/16Private subnetPublic subnetEC2 app10.0.10.25NAT gatewayper-GB processingGateway endpointroute: pl-xxxx → vpceInternet gatewayAmazon S3regional servicewithout endpointwith gateway endpoint: no NAT, no IGW, no chargepublic IPsS3 sees 10.0.10.25

Endpoints keep traffic on the AWS network, remove NAT data processing charges (gateway type), and let you write policies on aws:SourceVpce / aws:SourceVpc.

Task 1.6 · Endpoint types

Gateway vs interface vs Gateway Load Balancer endpoints

Three different mechanisms behind one console page.

Gateway endpointInterface endpoint (PrivateLink)GWLB endpoint
ServicesS3 and DynamoDB onlyMost AWS services, S3 too, partner and your own servicesYour inspection appliances behind a GWLB
MechanismRoute table entry: AWS prefix list → vpce-…ENI with a private IP in each chosen subnetENI used as a route target (next hop)
DNSNo change: public name, public IPs, routed privatelyRegional + zonal names; optional private DNS overrides the public nameNone — it is a routing target
Reach from on-premises / peered VPC / TGWNoYes (routed to the ENI IPs)Via routing design
Security groupNo (use the prefix list in instance SGs)Yes, on the endpoint ENINo
CostNo chargePer hour per AZ + per GB processedPer hour per AZ + per GB processed
Quota (default)20 per Region (adjustable), max 255 per VPC50 interface + GWLB endpoints per VPC (adjustable)(shared with interface)
Task 1.6 · Gateway endpoints

Gateway endpoints in detail

Free, simple, and limited to the VPC that owns them.

How it works

  • You associate the endpoint with route tables.
  • AWS adds a route: destination = AWS-managed prefix list (com.amazonaws.<region>.s3), target = vpce-….
  • Longest prefix match still applies: the prefix list beats 0.0.0.0/0 → NAT.
  • Creating or changing it breaks open TCP connections to S3; plan for reconnects.

Limits

  • Same Region only.
  • Not usable from on-premises, from a peered VPC, or through a Transit Gateway — gateway endpoints cannot be extended out of the VPC.
  • Requires DNS resolution of the public S3 names (VPC DNS or a resolver that returns AWS IPs).
  • Allow the prefix list in SG outbound rules; NACLs need the CIDRs (no prefix lists).

Policy conditions

  • Bucket policy: aws:SourceVpce (one endpoint) or aws:SourceVpc (any endpoint in a VPC).
  • aws:SourceIp does not work through an endpoint: S3 sees the private IP. Use aws:VpcSourceIp.
  • Endpoint policy: limit which buckets can be reached (s3:ResourceAccount, bucket ARNs).
aws ec2 create-vpc-endpoint --vpc-id vpc-0abc --vpc-endpoint-type Gateway \
  --service-name com.amazonaws.ap-northeast-1.s3 \
  --route-table-ids rtb-0priv-a rtb-0priv-c --policy-document file://s3-endpoint-policy.json
Task 1.6 · Interface endpoints

Interface endpoints: an ENI that fronts a service

One ENI per selected subnet (one subnet per AZ). The service is reached at that private IP.

DNS names you get

  • Regional: vpce-0123-abcd.ec2.ap-northeast-1.vpce.amazonaws.com → all ENI IPs.
  • Zonal: vpce-0123-abcd-ap-northeast-1a.ec2… → the ENI in that AZ only (keeps traffic AZ-local, avoids cross-AZ charges).
  • Private DNS (optional): a hidden private hosted zone makes the normal name ec2.ap-northeast-1.amazonaws.com resolve to the ENIs inside the VPC.
  • Private DNS needs enableDnsSupport and enableDnsHostnames on the VPC.

Data path facts

  • TCP only.
  • MTU 8500; larger packets are dropped; no Path MTU Discovery (MSS clamping instead).
  • Bandwidth: 10 Gbps per AZ by default, scales automatically up to 100 Gbps per AZ.
  • SG on the endpoint must allow 443 from the clients.
  • Endpoint policy max 20,480 characters.

Choose the subnets well

  • One subnet per AZ where clients run.
  • An AZ with no endpoint ENI still works through the regional name, but crosses AZs.
  • Use dedicated small "endpoint" subnets so IPs are predictable and NACLs simple.
  • IP address type: IPv4, IPv6, or dualstack, matching the subnets.
Task 1.6 · Private DNS

How private DNS for an interface endpoint resolves

Same name, different answer depending on who asks.

VPC with interface endpoint (private DNS on)App: sts.ap-northeast-1.amazonaws.com ?VPC Resolver (.2)hidden private zoneEndpoint ENIs10.0.20.11 / 10.0.21.11answers 10.0.20.11On-premises / other VPC (no forwarding)Clientsame name ?Public DNSpublic IPsOn-premises with forwarding ruleClientconditional forwarderInbound Resolverendpoint in the VPC

From outside the VPC, either forward the service domain to a Resolver inbound endpoint, or use the endpoint-specific vpce-… name. For S3, "private DNS only for inbound endpoint" sends on-premises clients to the interface endpoint and keeps VPC clients on the free gateway endpoint.

Task 4.1 · Endpoint policies

Endpoint policy: a resource policy on the network path

It does not grant anything; it is one more policy that must allow the request (with IAM and the resource policy).

{
  "Version": "2012-10-17",
  "Statement": [{
    "Sid": "OnlyOurOrgBuckets",
    "Effect": "Allow",
    "Principal": "*",
    "Action": ["s3:GetObject", "s3:PutObject", "s3:ListBucket"],
    "Resource": "*",
    "Condition": {
      "StringEquals": {
        "aws:PrincipalOrgID": "o-a1b2c3d4e5",
        "aws:ResourceOrgID": "o-a1b2c3d4e5"
      }
    }
  }]
}

Data perimeter pattern

  • Endpoint policy: only our principals, only our resources (aws:PrincipalOrgID, aws:ResourceOrgID). Stops data exfiltration to an attacker’s bucket.
  • Bucket policy: deny unless aws:SourceVpce is ours. Stops use of stolen credentials from outside.
  • SCP / RCP: enforce both org-wide.
  • Remember AWS services that read your buckets (yum repos, ECR layers on S3, SSM agents): allow their buckets explicitly or they break.
  • Default endpoint policy = full access.
Task 1.6 · Cross-Region PrivateLink

Reach a service in another Region through a local endpoint

The consumer creates the endpoint in its own Region and names the service Region; AWS carries the traffic over its backbone.

Your own endpoint services (Nov 2024)

  • Provider adds consumer Regions to the endpoint service ("supported Regions").
  • Consumer creates an interface endpoint in its Region with --service-region.
  • The provider cannot remove the Region that hosts the service.
  • Removing a supported Region blocks new endpoints, not existing ones.

AWS services (Nov 2025)

  • Interface endpoints to selected AWS services in another Region.
  • Use case: a regulated workload in one Region needs an AWS service only offered elsewhere, without internet or a VPC in that Region.
  • Gateway endpoints remain same-Region only.
  • Check the live list of supported services and Regions before designing on it.

Design notes

  • Inter-Region data transfer charges apply on top of PrivateLink charges.
  • Alternative for many services across Regions: inter-Region TGW peering or Cloud WAN to a shared-services VPC with endpoints.
  • Exam questions written before 2024 assume PrivateLink is Regional only.
aws ec2 create-vpc-endpoint --vpc-id vpc-0abc --vpc-endpoint-type Interface \
  --service-name com.amazonaws.vpce.us-east-1.vpce-svc-0123456789abcdef0 --service-region us-east-1 \
  --subnet-ids subnet-a subnet-c --security-group-ids sg-0endpoint
Task 1.6 · Centralized endpoints

Share interface endpoints from one hub VPC

Hundreds of VPCs × dozens of services × 3 AZs becomes expensive. Centralize, then make DNS point at the hub.

Spoke VPCsSpoke APHZ associatedSpoke BPHZ associatedSpoke C (other account)PHZ via Profiles / RAMTransit GatewayShared-services VPCInterface endpointsprivate DNS OFFPrivate hosted zonessqs.ap-northeast-1.amazonaws.com → alias vpceResolver inbound endpointfor on-premises

Turn private DNS off on the hub endpoints, create one private hosted zone per service name with an alias to the endpoint, and associate the zones with every spoke VPC. Trade-off: TGW processing charges per GB vs per-VPC endpoint hours.

Exam traps

Endpoint traps

Wrong answers that look right.

Gateway endpoint reach

  • On-premises over DX or VPN cannot use a gateway endpoint. Use an S3 interface endpoint.
  • A peered VPC cannot use your gateway endpoint either.

aws:SourceIp

  • A bucket policy with aws:SourceIp = VPC CIDR never matches through an endpoint. Use aws:VpcSourceIp, aws:SourceVpc or aws:SourceVpce.

Private DNS scope

  • Private DNS works only for clients using the VPC Resolver of that VPC.
  • On-premises needs a forwarder to an inbound endpoint.

Endpoint SG

  • Interface endpoint timeouts are usually its SG missing inbound 443 from the client CIDR.

Policy ≠ grant

  • An endpoint policy that allows * does not give a principal any permission it did not have.

Cost

  • Gateway endpoint is free; interface endpoint bills per AZ-hour. For S3 inside the VPC, gateway wins.
Exam-style scenarios

Endpoint scenarios

Scenario → answer → why.

ScenarioAnswerWhy
Private subnets download large S3 objects; NAT bill is highS3 gateway endpoint on the private route tablesFree, removes NAT processing
On-premises servers must upload to S3 over Direct Connect privatelyS3 interface endpoint + Resolver inbound endpoint (or endpoint DNS names)Gateway endpoints are not reachable from outside the VPC
Stop developers copying data to personal buckets from the VPCEndpoint policy with aws:ResourceOrgID / s3:ResourceAccountRestricts which resources the path can reach
Bucket must only be readable from one VPCBucket policy deny unless aws:SourceVpc matchesIdentity-independent network condition
100 VPCs each need 15 interface endpointsCentral endpoints in a shared-services VPC + PHZs associated with spokesCuts endpoint hours; DNS steers to the hub
App in AZ a must not pay cross-AZ for endpoint trafficUse the zonal endpoint DNS nameResolves only to the ENI in that AZ
Chapter 2 · Part 2.2

PrivateLink for your own services

Publish a service behind an NLB or GWLB to other VPCs and accounts — one-way and overlap-tolerant.

Task 1.6 · PrivateLink model

Provider, consumer, and a one-way door

The consumer gets an ENI in its own VPC; the provider sees connections from its NLB. Nothing else is routable.

Consumer VPC 10.0.0.0/16 (account B)Client10.0.1.50Interface endpointENI 10.0.5.10Can overlap with the provider CIDRno routes between VPCs existProvider VPC 10.0.0.0/16 (account A)Endpoint servicevpce-svc-…NLBinternalTargetsEC2 / IP / ALBAWS PrivateLinkOnly the consumer can open connections.Provider cannot reach the consumer VPC.
Task 2.2 · Build an endpoint service

Publish a service in five commands

Provider side, then consumer side. The NLB must be in the AZs the consumer wants to use.

# provider (account A)
aws ec2 create-vpc-endpoint-service-configuration \
  --network-load-balancer-arns $NLB_ARN \
  --acceptance-required \
  --supported-ip-address-types ipv4 \
  --private-dns-name api.example.com
aws ec2 modify-vpc-endpoint-service-permissions \
  --service-id vpce-svc-0123 \
  --add-allowed-principals arn:aws:iam::222233334444:root

# consumer (account B)
SVC=com.amazonaws.vpce.ap-northeast-1.vpce-svc-0123
aws ec2 create-vpc-endpoint --vpc-endpoint-type Interface \
  --vpc-id vpc-0consumer --service-name $SVC \
  --subnet-ids subnet-a subnet-c --security-group-ids sg-0ep

# provider accepts
aws ec2 accept-vpc-endpoint-connections \
  --service-id vpce-svc-0123 --vpc-endpoint-ids vpce-0abc

Controls on the provider side

  • Allowed principals: accounts, roles or users who may create an endpoint. Default: nobody.
  • Acceptance required: manual approval per connection.
  • Allowing * with auto-accept makes the NLB effectively public.
  • Removing a principal does not cut existing connections — reject the endpoint connection.
  • Event notifications via SNS on connect/accept/reject.
  • A new NLB AZ must also be enabled on the endpoint service.
Task 1.6 · Private DNS name

Give consumers a friendly name: domain verification

Without it consumers use vpce-… names or create their own zone.

Steps

  • Provider sets a private DNS name on the service, e.g. api.example.com (wildcards allowed, lowercase).
  • AWS returns a TXT record name and value.
  • Provider publishes the TXT record in the public DNS of example.com.
  • Run start-vpc-endpoint-service-private-dns-verification; status becomes verified.
  • Consumers enable private DNS on their endpoint: api.example.com resolves to their ENIs.

If verification lapses

  • New connection requests are denied.
  • Existing connections keep working.
  • Keep the TXT record in place; automate a check.

Consumer side

  • Needs enableDnsSupport and enableDnsHostnames.
  • Only one endpoint per VPC can own a given private DNS name.
  • On-premises consumers: forward api.example.com to a Resolver inbound endpoint.
Task 3.1 · Source IP and health

What the provider sees, and how to keep it highly available

The NLB performs source NAT for PrivateLink traffic; the client IP is hidden unless you ask for it.

Client identity

  • Targets see the NLB private IP, not the consumer IP.
  • Enable proxy protocol v2 on the target group to get the source IP and the endpoint ID (vpce-… in a TLV).
  • Targets must parse proxy protocol, or connections fail.
  • Authorize per consumer by endpoint ID, not by IP (consumer CIDRs can overlap).

Availability

  • Consumer endpoint ENIs only exist in AZs that the NLB also serves.
  • Enable cross-zone load balancing on the NLB if targets are not in every AZ.
  • Health checks happen on the provider side; with cross-zone off, an AZ with no healthy targets can black-hole that AZ’s endpoint traffic.

Scale and IPv6

  • Endpoint bandwidth scales to 100 Gbps per AZ.
  • IPv6 consumers: the NLB must be dualstack; targets may stay IPv4.
  • GWLB-backed endpoint services carry traffic to inspection appliances (Chapter 7).
Task 1.6 · Resource and service-network endpoints

Newer endpoint types (Dec 2024)

PrivateLink can now reach a single resource (a database, an IP, a domain) without an NLB, or a whole Lattice service network.

Endpoint typeConnects toProvider buildsTypical use
InterfaceAn endpoint service (NLB / GWLB) or an AWS serviceNLB + endpoint serviceSaaS APIs, AWS APIs
ResourceOne resource configuration: IP, domain-name target, or RDS databaseResource gateway + resource configuration, shared via RAMGive a partner access to one database port without an NLB
Service networkA VPC Lattice service network (its services and resources)Lattice service networkOn-premises or TGW-connected clients reaching Lattice services
Gateway Load BalancerA GWLB endpoint serviceGWLB + appliancesTransparent inspection

Quotas: 200 resource endpoints and 50 service-network endpoints per VPC by default (both adjustable).

Task 3.3 · Choose the pattern

PrivateLink vs peering vs Transit Gateway for "service access"

PrivateLink exposes one service; peering and TGW connect whole networks.

RequirementPrivateLinkVPC peeringTransit Gateway
Overlapping CIDRsWorksNot possibleOnly with NAT tricks
Consumer initiates onlyYes, by designNo — both sides routableNo — routing decides
Hundreds of consumers / SaaSYesQuota-bound (50, up to 125 per VPC)Yes, but full network access
Many ports and protocols, east-westTCP via NLB listeners onlyYesYes
Cost modelEndpoint hours + per GBData transfer onlyAttachment hours + per GB
Chapter 2 · Part 2.3

Service-to-service networking

VPC Lattice (beyond the exam guide), AWS Cloud Map, and life after App Mesh.

Beyond the exam guide · VPC Lattice

VPC Lattice: application networking without routing

A managed L7 proxy (and L4 for resources) between clients in associated VPCs and services anywhere in the organization.

Client VPC A (10.0.0.0/16)Pod / EC2calls billing.example.internalSN–VPC associationsecurity groupService network "prod"Service: billingHTTPS listener · auth policy (IAM SigV4)Service: ordersHTTP / gRPC listenerResource configurationRDS :5432 (TCP)Service-network auth policy: coarse ("only our org").Service VPC B 10.0.0.0/16Target groupECS / EKS / Lambda / IP / ALBResource gatewayingress to RDSSame CIDR as VPC A: fine.
Beyond the exam guide · Lattice mechanics

How Lattice works

Components, traffic path and security layers.

Components

  • Service: listeners (HTTP, HTTPS, TLS passthrough), rules, target groups.
  • Target types: EC2, IP, Lambda, ALB, ECS tasks, Kubernetes pods (AWS Gateway API Controller).
  • Service network: logical boundary; associate services, resource configurations and VPCs.
  • Resource configuration + resource gateway: TCP access to an RDS database, IP or domain.
  • Share services and service networks across accounts with RAM.

Traffic path

  • Clients use the service DNS name; the VPC Resolver answers with a Lattice address.
  • Lattice addresses come from a link-local range published as an AWS-managed prefix list; allow it in client SGs and NACLs.
  • No route tables, TGW or peering involved; overlapping CIDRs are fine.
  • AZ affinity: clients get an address in their own AZ; no inter-AZ data transfer charge in Lattice.
  • On-premises or TGW clients: a service-network VPC endpoint.

Security layers

  • 1 · Association: no association or endpoint, no access.
  • 2 · Security group on the VPC association.
  • 3 · Auth policy on the service network (coarse, e.g. "only our org").
  • 4 · Auth policy on the service (fine, e.g. role X may POST /charge).
  • Auth policies do not apply to resource configurations.
Beyond the exam guide · Lattice auth policy

An auth policy that allows one caller role

Auth type AWS_IAM requires clients to sign requests with SigV4 (SDKs, or a sidecar/proxy).

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {
      "AWS": "arn:aws:iam::111122223333:role/orders-task"
    },
    "Action": "vpc-lattice-svcs:Invoke",
    "Resource":
      "arn:aws:vpc-lattice:*:444455556666:service/svc-0b/*",
    "Condition": {
      "StringEquals": {
        "vpc-lattice-svcs:RequestMethod": "POST" },
      "StringLike": {
        "vpc-lattice-svcs:RequestPath": "/charge*" }
    }
  }]
}
aws vpc-lattice create-service-network --name prod --auth-type AWS_IAM
aws vpc-lattice create-service --name billing --auth-type AWS_IAM
aws vpc-lattice create-service-network-service-association \
  --service-network-identifier sn-0prod \
  --service-identifier svc-0billing
aws vpc-lattice create-service-network-vpc-association \
  --service-network-identifier sn-0prod \
  --vpc-identifier vpc-0client \
  --security-group-ids sg-0lattice
aws vpc-lattice put-auth-policy \
  --resource-identifier svc-0billing \
  --policy file://auth.json
Task 2.2 · Service discovery

AWS Cloud Map

A registry of service instances, discoverable by DNS or by API, with health status.

Namespaces

  • HTTP: API discovery only (DiscoverInstances).
  • Private DNS: API + records in a Route 53 private hosted zone for chosen VPCs.
  • Public DNS: API + records in a public hosted zone.
  • Each service holds instances with attributes (IP, port, custom key/values).

Health

  • Route 53 health checks (public DNS namespaces).
  • Custom health status set by your code (UpdateInstanceCustomHealthStatus).
  • Unhealthy instances are dropped from DNS answers and API results.

Who uses it

  • ECS service discovery and ECS Service Connect register tasks automatically.
  • EKS via external tools; Lattice and App Mesh integrate.
  • Pick the API when clients can filter by attributes (version, stage); DNS when they cannot.
Task 2.2 · Choose the pattern

Lattice vs PrivateLink vs TGW, and life after App Mesh

App Mesh support ended on 30 Sep 2026; it is still on the ANS-C01 in-scope list.

NeedBest fit
L7 routing, auth per path, many services across accountsVPC Lattice
Expose one TCP service to many consumers, overlap-safePrivateLink endpoint service
Full network reachability, many protocols, inspectionTransit Gateway / Cloud WAN
ECS-only service mesh features (retries, metrics)ECS Service Connect
One database port to a partnerResource endpoint (Lattice resource configuration)

App Mesh migration

  • App Mesh was an Envoy-based mesh for ECS, EKS and EC2.
  • ECS workloads → ECS Service Connect (Cloud Map namespaces, managed proxy).
  • Cross-compute or cross-account → VPC Lattice.
  • EKS-only meshes may also move to open-source meshes (Istio, Linkerd).
  • Exam: an answer naming App Mesh can still be correct for older scenarios.
Exam traps and scenarios

PrivateLink, Lattice and service discovery

Scenario → answer → why.

ScenarioAnswerWhy
SaaS provider must serve 400 customer VPCs with overlapping CIDRsNLB + endpoint service, allowed principals per customerOne-way, overlap-tolerant, scales without routing
Provider app must log the real client IP behind PrivateLinkProxy protocol v2 on the NLB target groupNLB hides the source IP otherwise
Consumers want api.vendor.com to resolve privatelyPrivate DNS name on the endpoint service + TXT verificationDomain ownership must be proven
Removing a customer from allowed principals did not cut them offReject the existing endpoint connectionPermission changes affect only new connections
Microservices in 30 accounts need per-path IAM authorization, no routing changesVPC Lattice with service-level auth policiesL7 auth without TGW or peering
ECS team is still on App Mesh in Oct 2026Migrate to ECS Service ConnectApp Mesh support ended 30 Sep 2026
Self-check

Private access: can you answer these?

Explain the answer and name the exact feature.

Endpoints

  • Why can on-premises clients not use an S3 gateway endpoint?
  • What changes in DNS when you enable private DNS on an interface endpoint?
  • Why does aws:SourceIp fail behind an endpoint?
  • When do you use the zonal endpoint DNS name?
  • What is the MTU of an interface endpoint and what happens above it?

PrivateLink

  • What do allowed principals and acceptance each control?
  • How does the provider learn the consumer’s endpoint ID?
  • What happens to existing connections when verification of a private DNS name lapses?
  • Why must the NLB be present in the consumer’s AZs?
  • When would you choose a resource endpoint over an endpoint service?

Lattice and discovery

  • Name the four security layers of VPC Lattice.
  • Why do overlapping CIDRs not matter to Lattice?
  • What must client SGs allow for Lattice traffic?
  • Cloud Map: HTTP vs private DNS namespace?
  • Where should an App Mesh workload on ECS migrate, and why?
Chapter 3

Multi-VPC and multi-account

Tasks 1.6, 2.2, 3.1, 3.3 — peering, Transit Gateway, Cloud WAN, sharing, and overlapping IP ranges.

Chapter 3 · Part 3.1

VPC peering

A direct, non-transitive link between two VPCs: cheapest and simplest, until the mesh grows.

Task 1.6 · Peering model

Peering is a point-to-point link, never a hub

A peers with B and with C. B cannot reach C through A, and B cannot use A’s gateways.

VPC A10.1.0.0/16VPC B10.2.0.0/16VPC C10.3.0.0/16pcx-abpcx-acB → C via A: dropped (not transitive)A’s IGW, NAT, VGWand gateway endpointsB cannot use themEdge-to-edge routing is not supported: internet gateway, NAT, VPN, Direct Connect and gateway endpoints of A serve only A.

Unicast reverse path forwarding is not supported on peering connections: if a reply would come back over a different path, it is dropped.

Task 2.2 · Peering rules

What you must know about a peering connection

Most exam distractors break one of these rules.

Addressing

  • No matching or overlapping IPv4 or IPv6 CIDRs — any CIDR of either VPC, even secondary ones you will not use.
  • Only one peering connection between the same two VPCs.
  • IPv6 works if both VPCs have IPv6 CIDRs and routes.
  • Routes are manual on both sides, and can be more specific than the peer CIDR (partial access).

Cross-account and Region

  • Requester → accepter; the request expires after 7 days if not accepted.
  • Inter-Region peering runs on the AWS backbone, encrypted.
  • MTU: 9001 within a Region, 8500 between Regions.
  • SG referencing of the peer’s groups: same Region only.
  • Quota: 50 active peerings per VPC by default, adjustable up to 125.

DNS

  • You cannot query the peer VPC’s Amazon DNS server.
  • Enable DNS resolution support on the peering connection so public DNS hostnames of peer instances resolve to private IPs.
  • Required for inter-Region peering even with RFC 1918 CIDRs.
  • Private hosted zones: associate the zone with both VPCs instead.
Task 2.2 · Build a peering

Create, accept, route, allow

Four steps; forgetting the route on one side is the classic outage.

# account A (requester), Region ap-northeast-1
aws ec2 create-vpc-peering-connection --vpc-id vpc-0aaa \
  --peer-vpc-id vpc-0bbb --peer-owner-id 222233334444 \
  --peer-region ap-southeast-1

# account B (accepter), Region ap-southeast-1
aws ec2 accept-vpc-peering-connection \
  --vpc-peering-connection-id pcx-0123 --region ap-southeast-1

# both sides: route + DNS option
aws ec2 create-route --route-table-id rtb-0a \
  --destination-cidr-block 10.2.0.0/16 \
  --vpc-peering-connection-id pcx-0123
aws ec2 modify-vpc-peering-connection-options \
  --vpc-peering-connection-id pcx-0123 \
  --requester-peering-connection-options \
    AllowDnsResolutionFromRemoteVpc=true

Checklist

  • Route in every subnet route table that needs the peer, on both sides.
  • SG inbound: peer CIDR, or the peer SG ID (same Region only).
  • NACLs on both subnets, including ephemeral ports.
  • DNS option set by each owner for its own side.
  • Only VPC owners manage peerings; shared-VPC participants cannot.
  • Verify with Reachability Analyzer: source ENI → destination ENI.
Task 3.3 · Peering vs alternatives

When peering is still the right answer

n VPCs in a full mesh need n(n−1)/2 connections. Beyond a handful, the hub wins on operations.

Peering compared

PeeringTransit Gateway
TopologyPoint-to-pointHub-and-spoke, transitive
Hourly chargeNonePer attachment
Data processingNone (data transfer only; free within the same AZ)Per GB processed
BandwidthNo aggregate limit (instance limits apply)Up to 100 Gbps per VPC attachment per AZ
MTU9001 (8500 inter-Region)8500
Central inspectionNoYes (appliance mode)
Max scale125 per VPC5,000 attachments per TGW

Rules of thumb

  • A few VPCs, high-volume, latency-sensitive pair (e.g. analytics ↔ data lake): peering.
  • Dozens of VPCs, on-premises, segmentation, inspection: Transit Gateway.
  • Mixed: TGW for everything, plus a direct peering for one heavy pair (more specific route wins).
  • Proxy pattern (NLB/PrivateLink or an L7 proxy) when only one service must be exposed, or CIDRs overlap.
  • Migrating peering → TGW: watch the MTU drop from 9001 to 8500; change both sides together.
Chapter 3 · Part 3.2

AWS Transit Gateway deep dive

A Regional router you program with route tables: attachments, association, propagation, segmentation and inspection.

Task 1.6 · TGW building blocks

Attachments and route tables

Every packet enters on an attachment, is looked up in that attachment’s associated route table, and leaves on another attachment.

Attachment types

  • VPC — one ENI in one subnet per AZ you enable.
  • Site-to-Site VPN (and VPN Concentrator).
  • Direct Connect gateway (via a transit VIF).
  • Connect — GRE + BGP to SD-WAN appliances.
  • Peering — another TGW (same or other Region) or a Cloud WAN core network edge.
  • Network function — AWS Network Firewall attached natively (Jun 2025).

Route tables

  • Association: each attachment uses exactly one route table for lookups.
  • Propagation: an attachment can install its routes into many route tables.
  • Static routes, blackhole routes, and prefix list references.
  • Default route table: auto-association and auto-propagation (turn off for segmentation).
  • You cannot filter propagated routes; use separate tables instead.

Key quotas (default)

  • 5,000 attachments per TGW.
  • 20 route tables per TGW (adjustable).
  • 10,000 routes across all route tables of a TGW.
  • 50 peering attachments per TGW; only one between the same two TGWs.
  • One attachment per VPC per TGW; up to 5 TGWs per VPC.
  • MTU 8500 (VPN attachments: 1446 inside the tunnel).
Task 2.2 · VPC attachment details

How a VPC attaches, AZ by AZ

The TGW is reachable only from AZs where the attachment has a subnet.

Spoke VPC 10.1.0.0/16AZ aAZ cApp subnet0.0.0.0/0 → tgw-…App subnet0.0.0.0/0 → tgw-…TGW subnet /28attachment ENITGW subnet /28attachment ENITransit GatewayRegional routerVPC BVPNDX gateway

Best practice: a dedicated small subnet per AZ for attachment ENIs, with its own route table and a permissive NACL. Traffic from an AZ without an attachment subnet cannot reach the TGW; traffic to an attachment missing the source AZ is sent to another AZ at no extra TGW charge.

Task 2.2 · Route evaluation

Which route wins in a TGW route table

Longest prefix first; for the same prefix, the attachment type decides; within one type, BGP attributes.

Same prefix, different sources (high → low)

  • 1 · Static routes (including static VPN routes).
  • 2 · Prefix-list-referenced routes.
  • 3 · VPC-propagated.
  • 4 · Direct Connect gateway-propagated.
  • 5 · Connect-propagated.
  • 6 · VPN over private DX-propagated.
  • 7 · Site-to-Site VPN-propagated.
  • 8 · VPN Concentrator, Client VPN, then TGW peering (Cloud WAN) propagated.

Same prefix, same type (BGP)

  • Shorter AS_PATH wins.
  • Then lower MED.
  • Then eBGP over iBGP where supported.
  • No MED sent: TGW assigns 0 on DX, 100 on VPN and Connect.
  • Only the preferred route is shown; the backup appears when the primary is withdrawn.

Consequences

  • DX and VPN advertising the same prefix → DX wins by type, regardless of AS_PATH.
  • To prefer VPN for a prefix, advertise a more specific prefix over VPN.
  • A static route hides a propagated route with the same CIDR until you delete it.
  • ECMP: across VPN tunnels with dynamic routing (option on the TGW), Connect peers, and DX gateway attachments with identical prefix and AS_PATH; never across different attachment types.
Task 2.2 · Segmentation

Prod, dev and shared services with three route tables

Associate by "who am I", propagate by "who may reach me".

Prod VPC AProd VPC BDev VPC COn-premises (VPN)Transit GatewayRT-prodassociated: A, Broutes: A, B, Shared, VPNRT-devassociated: Croutes: C, SharedRT-sharedassociated: Shared, VPNroutes: all VPCs + VPNShared servicesDNS, AD, endpointsDev → Prod: no route in RT-dev → dropped.
Task 2.2 · Segmentation tables

The same design as route tables

Read each row as "attachments associated here can reach…".

RT-prod (associated: Prod A, Prod B)

DestinationTargetType
10.1.0.0/16Prod A attachmentpropagated
10.2.0.0/16Prod B attachmentpropagated
10.100.0.0/16Shared attachmentpropagated
192.168.0.0/16VPN attachmentpropagated
10.3.0.0/16—blackhole (explicit)

RT-dev and RT-shared

TableRoutes
RT-dev (Dev C)10.3.0.0/16 → Dev C; 10.100.0.0/16 → Shared
RT-shared (Shared, VPN)All VPC CIDRs propagated + on-premises from VPN
Spoke VPC route tables10.0.0.0/8 and 192.168.0.0/16 → tgw-…
aws ec2 create-transit-gateway --options DefaultRouteTableAssociation=disable,DefaultRouteTablePropagation=disable,VpnEcmpSupport=enable
aws ec2 associate-transit-gateway-route-table --transit-gateway-route-table-id tgw-rtb-prod --transit-gateway-attachment-id tgw-attach-a
aws ec2 enable-transit-gateway-route-table-propagation --transit-gateway-route-table-id tgw-rtb-shared --transit-gateway-attachment-id tgw-attach-a
Task 4.1 · Appliance mode

Stateful inspection needs symmetric AZs

Without appliance mode the TGW keeps each direction in its originating AZ — the reply may hit a firewall that never saw the request.

VPC A (AZ a)ClientVPC B (AZ c)ServerTGWInspection VPCFirewall AZ aFirewall AZ crequestreply (no appliance mode)drops: unknown flowWith appliance mode on the inspection VPC attachment,a flow hash picks one AZ for both directions.

Enable it on the inspection VPC attachment only (--options ApplianceModeSupport=enable). Use exactly one TGW per inspection VPC: TGWs do not share flow state. Native Network Firewall attachments have appliance mode on automatically.

Task 2.2 · Inspection routing

Route tables for centralized inspection

Spokes send everything to the inspection VPC; the inspection VPC sends traffic back to the TGW after the firewall.

TGW route tables

TableAssociatedRoutes
RT-spokesAll spoke VPCs0.0.0.0/0 → inspection attachment (static)
RT-inspectionInspection VPCAll spoke CIDRs propagated; 0.0.0.0/0 → egress VPC or local NAT

Inspection VPC subnet route tables

SubnetRoutes
TGW attachment subnet (per AZ)0.0.0.0/0 → firewall endpoint in the same AZ
Firewall subnet (per AZ)10.0.0.0/8 → tgw-…; 0.0.0.0/0 → NAT gateway
Public subnet (NAT)10.0.0.0/8 → firewall endpoint (return path); 0.0.0.0/0 → IGW

With the native Network Firewall attachment (Jun 2025) the inspection VPC disappears: point routes in the TGW route tables at the network function attachment (static routes only). Details in Chapter 7.

Task 2.2 · TGW peering

Inter-Region and intra-Region peering

Peering connects TGWs over the AWS backbone; routing across it is static only.

ap-northeast-1us-east-1VPC 10.1.0.0/16VPN 192.168.0.0/16TGW-1ASN 64512TGW-2ASN 64513VPC 10.20.0.0/16VPC 10.21.0.0/16peeringTGW-1 static: 10.20.0.0/15 → peeringTGW-2 static: 10.1.0.0/16, 192.168.0.0/16 → peeringUse a different ASN per TGW.

Limits: no BGP and no ECMP over peering; one peering between the same two TGWs; no PMTUD on peering, VPN or DX attachments (the TGW clamps MSS); SG referencing does not cross Regions. Many Regions? Cloud WAN builds the mesh and dynamic routing for you.

Task 2.2 · More TGW features

Multicast, Connect, SG referencing, cost allocation

Features with a specific exam trigger.

Multicast

  • Create the TGW with multicast support, then a multicast domain; associate subnets.
  • Groups by static registration or IGMPv2.
  • Limits: 1 Gbps and 75,000 pps per flow (fewer than 10 receivers); 20 Gbps aggregate per AZ.
  • Not recommended for high-frequency trading.
  • Only Nitro instances as senders; non-Nitro need source/destination check off.

Connect (SD-WAN)

  • Built on a VPC or DX attachment as transport.
  • GRE tunnel + BGP to the appliance; up to 4 Connect peers per attachment.
  • Up to 5 Gbps per peer (20 Gbps per Connect attachment); ECMP across peers.
  • Appliance can advertise up to 1,000 routes per peer.
  • Detail in Chapter 4.

Operations

  • SG referencing (Sep 2024): enable on the TGW and the VPC attachments; inbound rules only, same Region, not through inspection.
  • Flexible cost allocation (Nov 2025): charge data processing to source, destination or TGW owner account.
  • Flow logs per TGW or attachment.
  • Network Manager + Route Analyzer for path checks (Chapter 8).
Exam traps

Peering and Transit Gateway traps

Wrong answers that look right.

Transitivity

  • "Peer B to A and A to C" never connects B to C.
  • Sharing A’s NAT, IGW, VPN or gateway endpoint with peers is impossible.

Missing AZ

  • Instances in an AZ with no TGW attachment subnet cannot reach the TGW at all.

Asymmetric firewall

  • Inspection VPC drops replies: enable appliance mode on that attachment, not on spokes.

Peering routing

  • TGW peering is static only; "propagate routes across peering" is wrong.

Route priority

  • DX-propagated beats VPN-propagated for the same prefix whatever the AS_PATH; use a more specific prefix on VPN.

MTU

  • Peering in-Region 9001, TGW 8500, VPN 1446 inside the tunnel. Jumbo frames break after a migration from peering to TGW.
Exam-style scenarios

Peering and Transit Gateway scenarios

Scenario → answer → why.

ScenarioAnswerWhy
Two VPCs exchange 200 TB/month in the same AZ; cost must be minimalVPC peeringNo hourly or processing charge; free within the AZ
80 VPCs and two data centers; dev must never reach prodTGW with separate prod/dev route tables, no cross propagationTransitive hub with segmentation
Inspection VPC firewalls drop return traffic intermittentlyEnable appliance mode on the inspection VPC attachmentKeeps both directions in one AZ
Same prefix learned over DX and VPN; VPN should carry 10.50.0.0/24Advertise 10.50.0.0/24 over VPN (longer prefix)Type priority prefers DX for equal prefixes
Connect TGWs in Tokyo and VirginiaInter-Region TGW peering with static routesNo BGP over peering
Need more than 1.25 Gbps over IPsec to the TGW from one siteMultiple VPN tunnels with BGP + ECMP (or 5 Gbps tunnels, Ch 4)ECMP needs dynamic routing
Chapter 3 · Part 3.3

AWS Cloud WAN

Beyond the exam guide — a global network defined by a policy document instead of per-Region route tables.

Beyond the exam guide · Cloud WAN model

Core network, edges and segments

One core network spans Regions; each Region has a core network edge (CNE); segments are global isolated routing domains.

Core network (global, policy-driven)CNE ap-northeast-1ASN 64520CNE eu-west-1ASN 64521CNE us-east-1ASN 64522AWS backbone, dynamicSegment: prodVPCs tagged env=prod in every RegionSegment: devisolated from prodSegment: hybridVPN, Connect, DX gatewayNetwork function group"inspection"firewall VPCs in each RegionSegment actionsshare: prod ↔ hybridsend-via inspection: dev → prodsend-to inspection: internet egress
Beyond the exam guide · Policy document

A minimal core network policy

Policies are versioned; each change produces a change set you review and then execute (rollback = execute an older version).

{
  "version": "2021.12",
  "core-network-configuration": {
    "asn-ranges": ["64520-64530"],
    "edge-locations": [{"location": "ap-northeast-1"},
                       {"location": "us-east-1"}]
  },
  "segments": [
    {"name": "prod", "require-attachment-acceptance": true},
    {"name": "dev", "isolate-attachments": false},
    {"name": "hybrid"}
  ],
  "segment-actions": [
    {"action": "share", "mode": "attachment-route",
     "segment": "hybrid", "share-with": ["prod", "dev"]}
  ],
  "attachment-policies": [
    {"rule-number": 100,
     "conditions": [{"type": "tag-value", "operator": "equals",
                     "key": "env", "value": "prod"}],
     "action": {"association-method": "constant",
                "segment": "prod"}}
  ]
}

Building blocks

  • segments: routing domains; isolate-attachments blocks traffic between attachments of the same segment.
  • attachment-policies: rules (by tag, account, Region, type) that map attachments to segments automatically.
  • segment-actions: share routes between segments, create-route static routes, send-via / send-to steer traffic through a network function group.
  • network-function-groups: inspection or egress VPCs used by send-via / send-to.
  • Routing policies (Nov 2025): filter, summarize and manipulate BGP routes; route information base (RIB) view.
Beyond the exam guide · Attachments

What attaches to Cloud WAN

Attachments land in a Region’s CNE and are mapped to a segment by the attachment policy.

AttachmentNotes
VPCSubnet per AZ, like TGW; appliance mode option; tags drive segment mapping
Site-to-Site VPNDynamic routing; 5 Gbps large-bandwidth tunnels supported (Nov 2025)
ConnectGRE or tunnel-less (Oct 2023, BGP straight to the appliance) over a VPC transport
Transit Gateway route tableVia TGW ↔ CNE peering; lets existing TGW estates join gradually
Direct Connect gatewayNative attachment; DX routes propagate into segments without a TGW

Security group referencing across Cloud WAN VPC attachments arrived in Jun 2025 (same Region, inbound rules).

Beyond the exam guide · Decision

Cloud WAN vs Transit Gateway with peering

Both are valid; the choice is about scale and how you want to operate.

FactorTGW + peeringCloud WAN
RegionsOne TGW per Region, peer in a meshOne core network, CNE per Region
Inter-Region routingStatic routes you maintainDynamic, automatic
SegmentationRoute tables per TGW, repeated per RegionGlobal segments in one policy
OnboardingAssociate + propagate per attachmentTag the attachment; policy maps it
Change controlAPI calls / IaCVersioned policy, change sets, rollback
Exam (ANS-C01)In scopeNot on the in-scope list

Pick Cloud WAN when

  • Three or more Regions with many VPCs.
  • Global segmentation must be identical everywhere.
  • SD-WAN or many branch VPNs land in multiple Regions.
  • Migration path: peer existing TGWs to CNEs and move segments over time.
Chapter 3 · Part 3.4

VPC sharing and AWS RAM

Many accounts, one network: the owner runs routing, participants run workloads.

Task 1.6 · VPC sharing model

Owner and participants in one VPC

The network account owns the VPC and shares subnets through AWS RAM with accounts or OUs in the same organization.

Shared VPC 10.10.0.0/16 (owner: network account)app-a → OU Workloadsdata-a → account CEC2 · account AECS · account BEach participantown SGs, ENIs, flow logsRDS · account CSame VPCimplicit local route between accountsOwner keepsSubnets and route tablesNetwork ACLsInternet and NAT gatewaysTGW attachments, peeringGateway / interface endpointsResolver endpointsFlow logs for subnets
Task 2.2 · Sharing rules

Who can do what in a shared VPC

Participants consume the network; they cannot change it.

Participants can

  • Create, modify, delete their own resources in shared subnets (EC2, RDS, Lambda, ENIs).
  • Create their own security groups and reference others’ SGs as account-id/sg-id.
  • Use owner SGs that the owner explicitly shares with them.
  • Create flow logs for ENIs they own.
  • Describe VPC, subnets, route tables, NACLs, IGW.

Participants cannot

  • Modify subnets, route tables, NACLs or the VPC.
  • Create or describe NAT gateways; create IGWs.
  • Attach a TGW, create peering, or flow logs for the subnet.
  • Use the VPC’s default security group (owner’s).
  • See owner tags (tags are not shared).

Requirements and tips

  • Same AWS Organization; enable RAM sharing with Organizations once.
  • Default-VPC subnets cannot be shared.
  • AZ names differ per account; plan with AZ IDs (e.g. apne1-az1).
  • Participant resources count against the participant’s quotas.
  • Use one shared VPC per environment and trust boundary, not one for everything.
aws ram enable-sharing-with-aws-organization            # management account, once
aws ram create-resource-share --name prod-subnets \
  --resource-arns arn:aws:ec2:ap-northeast-1:111122223333:subnet/subnet-0app-a \
  --principals arn:aws:organizations::999988887777:ou/o-a1b2c3d4e5/ou-ab12-workloads
Task 2.2 · AWS RAM for networking

Network resources you share with RAM

Build once in the network account, consume everywhere.

ResourceWhat the consumer getsTypical pattern
SubnetsLaunch resources into the owner’s VPCVPC sharing per environment
Transit GatewayCreate attachments from their own VPCs (owner may need to accept)Central TGW in the network account
Customer-managed prefix listsReference in SGs and route tables; updates flow automaticallyCorporate CIDRs, partner ranges
VPC Resolver rules / Route 53 ProfilesForwarding rules or a bundle of DNS settings to associate with VPCsHybrid DNS (Chapter 5)
IPAM poolsAllocate CIDRs from a central planNo overlapping VPCs (Chapter 1)
Network Firewall policiesUse a central policy on local firewallsDistributed inspection
Cloud WAN core network, VPC Lattice service networks / services / resource configurationsAttach or associate across accountsGlobal network, app networking

Not RAM: Direct Connect gateways are shared across accounts with association proposals; VPC peering uses requests and acceptance.

Exam traps and scenarios

Cloud WAN, VPC sharing and RAM

Scenario → answer → why.

ScenarioAnswerWhy
50 application accounts, one network team, apps need low-latency east-west in the same VPCVPC sharing from a network accountImplicit local routing, central control, separate billing
Participant cannot add a route to the NAT gatewayExpected: only the owner manages route tablesParticipants consume, owner configures
Every account must use the corporate CIDR list in SGs, kept currentCustomer-managed prefix list shared with RAMOne update propagates everywhere
Five Regions, identical prod/dev isolation, frequent onboardingCloud WAN with segments and tag-based attachment policiesGlobal policy, automatic mapping
Team in account B must attach its VPC to the central TGWShare the TGW with RAM; B creates the attachmentCross-account attachments need the share
Subnet placement differs between accounts though both chose "AZ a"Use AZ IDsAZ names are mapped per account
Chapter 3 · Part 3.5

Overlapping IP ranges

Acquisitions, partners and Kubernetes pods: how to connect networks that reuse the same addresses.

Task 1.6 · Options

Five ways to live with overlapping CIDRs

Routing cannot choose between two identical destinations; something must translate or avoid routing.

OptionHowGood forCost / limits
PrivateLinkExpose a service via NLB endpoint service; consumer gets a local IPClient → specific service, many consumersOne-way, TCP, per-service
VPC LatticeService network; no routing between VPCsService-to-service across accountsL7 / TCP resources; not on the exam
Private NAT gateway + TGWTranslate a non-routable range to a small routable rangeMany-to-one, initiated from the NATed sideNAT + TGW processing; inbound needs an LB
IPv6Give workloads globally unique IPv6 addressesNew builds, EKS IPv6 clustersAll components must support IPv6
Re-IPRenumber one side (IPAM to prevent recurrence)Long-term fix after an acquisitionProject effort, downtime risk

Prevent it first: allocate every VPC from Amazon VPC IPAM pools (Chapter 1).

Task 1.6 · Private NAT pattern

Routable and non-routable ranges with a private NAT gateway

Both VPCs reuse 10.0.0.0/16 internally; each also gets a small unique range that the TGW routes.

VPC A · 10.0.0.0/16 + 100.64.1.0/24App10.0.5.20Private NAT GW100.64.1.10TGW attachmentin 100.64.1.0/24src → 100.64.1.10TGWstatic routesVPC B · 10.0.0.0/16 + 100.64.2.0/24TGW attachmentin 100.64.2.0/24ALB100.64.2.xTargets10.0.7.30

Routes: A non-routable subnet 100.64.2.0/24 → private NAT; A routable subnet 100.64.2.0/24 → TGW; TGW (propagation off) static 100.64.1.0/24 → A, 100.64.2.0/24 → B; B routable subnet 100.64.1.0/24 → TGW.

Task 1.6 · Private NAT details

Private NAT gateway facts

Same service as a public NAT gateway, without an Elastic IP or internet path.

Behavior

  • Source NAT to its private IP (plus optional secondary private IPs for more connections).
  • Only outbound-initiated flows: the far side cannot open connections back to the NATed hosts.
  • Works toward TGW, VGW (VPN/DX) or peering destinations.
  • Also solves "partner only allows 100.64.1.0/28": present one allow-listed address.

For inbound

  • Put an ALB or NLB in the routable subnet of the server side.
  • Register non-routable targets by IP.
  • Combine both directions for bidirectional services.

Kubernetes pattern

  • EKS nodes and load balancers in a small routable CIDR.
  • Pods in a large secondary CIDR such as 100.64.0.0/16 (VPC CNI custom networking), reused in every cluster VPC.
  • Pod egress to corporate networks via a private NAT gateway; to the internet via a public NAT.
  • Or run IPv6 clusters to avoid the problem.
Exam traps

Overlap traps

Wrong answers that look right.

Peering will not do it

  • Peering refuses any CIDR overlap, even on secondary CIDRs you never use.

TGW alone will not do it

  • Two attachments propagating the same CIDR do not load-balance; VPC attachments do not support ECMP.

Private NAT is one-way

  • Servers behind a private NAT gateway cannot be reached from the other VPC; add an LB.

Public NAT is not the answer

  • Public NAT goes to the internet, not to a partner over TGW.

Gateway endpoints

  • Overlap does not matter for S3/DynamoDB gateway endpoints; they are not routed between VPCs.

Lattice / PrivateLink

  • Both tolerate overlap because nothing is routed between the VPCs.
Exam-style scenarios

Overlap scenarios

Scenario → answer → why.

ScenarioAnswerWhy
Acquired company uses the same 10.0.0.0/16; their app needs one API in our VPCPrivateLink endpoint service for the APIOverlap-tolerant, least change
Many workloads in overlapping VPCs must call on-premises systemsRoutable /24 per VPC + private NAT gateway + TGW static routesTranslates many-to-one into a unique range
Partner firewall only allows 100.64.10.0/28Private NAT gateway in a subnet from that rangePresent allow-listed source IPs
EKS pods exhaust routable IPs in every clusterSecondary non-routable CIDR for pods + custom networking (or IPv6)Reuse the same pod range everywhere
Need permanent fix and avoid future overlapsRe-IP one side; allocate all VPCs from IPAM poolsRemoves the root cause
Self-check

Multi-VPC and multi-account: can you answer these?

Explain the answer and name the exact feature.

Peering and sharing

  • Why can VPC B not use VPC A’s NAT gateway over peering?
  • What changes when you enable DNS resolution support on a peering?
  • Name three things a shared-VPC participant cannot do.
  • Why plan shared subnets with AZ IDs?
  • When is peering still better than a TGW?

Transit Gateway

  • Association vs propagation — what does each control?
  • Same prefix from DX and VPN: which wins and how do you override it?
  • What problem does appliance mode fix, and on which attachment?
  • Why must every AZ have an attachment subnet?
  • What routing does TGW peering support?

Cloud WAN and overlap

  • What do share, send-via and send-to do?
  • How do attachment policies place a VPC in a segment?
  • Draw the private NAT pattern for overlapping VPCs.
  • Why is a private NAT gateway not enough for inbound connections?
  • Which two services tolerate overlap because they do not route?
Chapter 4

Hybrid connectivity

Tasks 1.5, 2.1, 2.2, 3.1, 4.1, 4.3 — BGP, Site-to-Site VPN, Direct Connect, resiliency, SD-WAN and multicloud.

Chapter 4 · Part 4.1

BGP for AWS engineers

The one routing protocol every hybrid design depends on, and the AWS-specific knobs that steer it.

Task 1.5 · BGP refresher

What BGP does on an AWS hybrid link

Every Direct Connect VIF, dynamic VPN tunnel and Connect peer is an eBGP session between your router and AWS.

Session basics

  • TCP port 179 between two peers; each side has an ASN.
  • eBGP: different ASNs (the normal AWS case). iBGP: same ASN (possible only on Transit Gateway Connect).
  • Peers exchange prefixes plus attributes (AS_PATH, MED, communities, next hop).
  • A prefix is only usable if the session is Established.

ASN ranges (Direct Connect)

  • Your side: 1 to 4,294,967,294 (2-byte and 4-byte ASNs).
  • Private: 64,512–65,534 and 4,200,000,000–4,294,967,294.
  • You cannot use the same ASN on both ends of a VIF.
  • Public VIF with a public ASN: you must own it; AWS verifies.
  • AWS side: the VGW, DX gateway or Transit Gateway ASN (default 64512 for VGW and TGW).

Two directions, two decisions

  • AWS → on-premises: AWS chooses the path. You influence it with what you advertise (prefix length, communities, AS_PATH, MED).
  • On-premises → AWS: your router chooses. You control it with your own local preference, weight or policy.
  • Always design both directions, or you get asymmetric routing.
Task 3.1 · Route priority in a VPC

How a VPC route table picks a route for the same destination

Applies to virtual private gateway (VGW) designs. Longest prefix always wins first.

VPC route table order

#RuleExample
1Longest prefix match10.10.2.0/24 beats 10.10.0.0/16
2Static routesIGW, NAT, peering, TGW, ENI, gateway endpoint, VGW static
3Prefix list routesA route referencing a managed prefix list
4aPropagated: Direct Connect BGPLearned over a private VIF on the VGW
4bPropagated: VPN staticStatic routes on a VPN connection
4cPropagated: VPN BGPDynamic routes over the VPN tunnels

What this means

  • Same prefix over DX and VPN on the same VGW → DX wins.
  • A static route to anything beats a propagated route to the VGW.
  • To force VPN for one subnet, advertise a more specific prefix over the VPN.
  • Propagation must be enabled per route table; otherwise no BGP routes appear at all.
Task 3.1 · Route priority in Transit Gateway

How a Transit Gateway route table picks a route

Most specific first. For the same CIDR from different attachment types, a fixed order applies; within one type, BGP attributes decide.

Same CIDR, different attachment types

PriorityRoute source
1Static routes (including VPN static)
2Prefix list referenced routes
3VPC-propagated routes
4Direct Connect gateway-propagated
5Transit Gateway Connect-propagated
6Site-to-Site VPN over private Direct Connect
7Site-to-Site VPN-propagated
8VPN Concentrator, then Client VPN, then TGW peering (Cloud WAN)

Same CIDR, same attachment type

  • Shorter AS_PATH wins.
  • Then lower MED.
  • Then eBGP over iBGP (where supported).
  • No MED sent? TGW assigns 0 on DX attachments and 100 on VPN and Connect.
  • TGW shows only the preferred route; the backup appears only when the primary is withdrawn.
  • BGP Multipath AS-Path Relax is not supported: no ECMP across different ASNs.
Task 1.5 · Steering AWS → on-premises

The four levers AWS honours, in evaluation order

On private and transit VIFs, AWS evaluates prefix length, then local preference communities, then AS_PATH, then MED.

OrderLeverHow you set itUse it for
1Longest prefixAdvertise /24s on the primary VIF and the /16 on bothStrong active/passive; AWS recommends it
2Local preference community7224:7300 high, 7224:7200 medium, 7224:7100 low on each prefixActive/passive with identical prefixes; beats AS_PATH
3AS_PATH lengthPrepend your ASN on the backup pathWhen prefix and local pref are equal
4MEDLower MED preferredLast resort; AWS does not recommend it
—ECMPSame prefix, same attributes (same AS_PATH length) on several VIFsActive/active load sharing

Default without communities: a Region prefers Direct Connect locations associated with that Region (treated as medium, 7224:7200); other locations get a lower value. That is why a remote DX location can lose to a local one even with a shorter AS_PATH.

Task 1.5 · Active/passive design

Two Direct Connect connections, one primary

Primary tagged high preference, backup tagged low. Your router mirrors the choice for the return direction.

On-premises AS 65010Core router10.20.0.0/16Edge Alocal-pref 200Edge Blocal-pref 100Direct Connect locationsDX location 1connection 1 · primaryDX location 2connection 2 · backup7224:73007224:7100AWS RegionDX gatewayASN 64512Transit GWVPCstransit VIFtransit VIFAWS → on-prem: high pref path winson-prem → AWS: Edge A local-pref 200

Without the on-premises local preference, traffic to AWS can leave through Edge B while replies come back through Edge A. Stateful firewalls on the edges then drop it.

Task 2.1 · BGP communities

Every AWS community you need to know

Communities 7224:1–7224:65535 are reserved by Direct Connect; unsupported communities on a public VIF are removed.

Private and transit VIFs (you → AWS)

CommunityMeaning
7224:7100Low local preference
7224:7200Medium local preference
7224:7300High local preference
same tag on all pathsECMP active/active

Public VIF (you → AWS): scope

CommunityYour prefix reaches
7224:9100Local AWS Region only
7224:9200All Regions on the continent
7224:9300All public Regions (global)
noneGlobal (default)

Public VIF (AWS → you): origin tags

CommunityAWS prefix originates in
7224:8100Same Region as the DX location
7224:8200Same continent
no tagOther continents
NO_EXPORTOn every route AWS advertises

Filter on 7224:8100 to accept only local-Region AWS prefixes over a public VIF and send everything else to the internet. Local preference communities are evaluated before AS_PATH.

Task 3.1 · Timers and BFD

Detect failure in under a second, not in 90 seconds

Default BGP hold time on Direct Connect is 90 seconds. BFD cuts detection to roughly 300 ms × 3.

Direct Connect BGP settings

SettingValue
Hold timerdefault 90 s, minimum 3 s
Keepalivedefault 30 s, minimum 1 s
BFD minimum interval300 ms
BFD minimum multiplier3
Graceful restart timer120 s (do not combine with BFD)
BGP TTL1: no multihop BGP
MD5 authenticationAlways on; key yours or AWS-generated

By connection type

  • Direct Connect: configure BFD on your router; BGP timers negotiate down to the lowest value.
  • Site-to-Site VPN: no BFD. Failure is detected by Dead Peer Detection (DPD, default 30 s) and BGP.
  • Transit Gateway Connect: no BFD, no graceful restart; keepalive 10 s, hold 30 s; eBGP multihop TTL 2.
  • Faster detection means faster failover to the backup path — test it with the failover test (Part 4.4).
Task 2.1 · Router config

FRRouting: primary and backup transit VIFs

Tag the primary with high preference, the backup with low; prefer the primary for outbound too.

router bgp 65010
 neighbor 169.254.10.1 remote-as 64512      # VIF 1 (primary)
 neighbor 169.254.10.1 password <md5-key>
 neighbor 169.254.10.1 bfd
 neighbor 169.254.20.1 remote-as 64512      # VIF 2 (backup)
 neighbor 169.254.20.1 password <md5-key>
 neighbor 169.254.20.1 bfd
 address-family ipv4 unicast
  network 10.20.0.0/16
  neighbor 169.254.10.1 route-map TO-AWS-PRI out
  neighbor 169.254.20.1 route-map TO-AWS-BAK out
  neighbor 169.254.10.1 route-map FROM-AWS-PRI in
 exit-address-family
!
route-map TO-AWS-PRI permit 10
 set community 7224:7300                    # high: AWS -> us via VIF 1
route-map TO-AWS-BAK permit 10
 set community 7224:7100                    # low: backup
route-map FROM-AWS-PRI permit 10
 set local-preference 200                   # us -> AWS via VIF 1

Checks

  • show bgp summary: both neighbors Established.
  • show bgp ipv4 unicast 10.0.0.0/16: best path via 169.254.10.1.
  • AWS side: aws directconnect describe-virtual-interfaces → bgpPeers[].bgpStatus = up.
  • Transit Gateway: search-transit-gateway-routes shows the DX gateway attachment.
  • Shut VIF 1 BGP (or run a failover test) and confirm traffic moves within BFD time.
  • Prepending instead of communities also works but is evaluated later.
Exam traps

BGP traps

Wrong answers that look right.

AS_PATH vs local pref

  • On private/transit VIFs, local preference communities beat AS_PATH prepending.
  • Prepending alone does not override a 7224:7300 tag on another path.

Private ASN on a public VIF

  • AWS replaces it with 7224 toward other networks, so your prepends are stripped.
  • Use a public ASN if prepending must work beyond AWS.

Wrong direction

  • Communities steer AWS → on-premises only.
  • On-premises → AWS is your router’s decision (local pref, weight).

ECMP needs identical attributes

  • Same prefix and same AS_PATH length on each path.
  • ECMP across different ASNs is not supported on TGW.

Too many prefixes

  • Over 100 prefixes on a private/transit VIF (default) → session goes idle (DOWN).
  • Summarize, or raise the allocation with prefix controls (max 1,000).

MED is weak

  • MED is evaluated last and AWS does not recommend it.
  • Pick prefix length or communities in exam answers.
Exam-style scenarios

BGP scenarios

Scenario → answer → why.

ScenarioAnswerWhy
Two DX connections; all AWS → on-prem traffic must use connection 1 unless it fails; same prefixes on bothTag prefixes on VIF 1 with 7224:7300 and VIF 2 with 7224:7100Local pref communities are the recommended active/passive lever
Same requirement, but the team may only change what it advertises, not tagsAdvertise more specific prefixes on VIF 1, the summary on bothLongest prefix is evaluated first
Load share across two DX connections in different locationsSame prefixes, same community (e.g. 7224:7200), same AS_PATHECMP needs identical attributes
Public VIF: on-prem prefixes must be reachable only from the local RegionTag with 7224:9100Scope communities limit propagation
Failover from DX to VPN takes over a minuteEnable BFD on the DX BGP sessionsDefault hold time is 90 s; BFD detects in about 900 ms
On-prem router advertises 150 prefixes and the VIF keeps going downSummarize routes (or request higher prefix allocation)Default limit is 100 per session
Chapter 4 · Part 4.2

AWS Site-to-Site VPN

IPsec over the internet or over Direct Connect: tunnels, options, routing, scale, and remote-user access.

Task 1.5 · VPN architecture

One VPN connection = two IPsec tunnels

Each tunnel has its own AWS public IP in a different AZ. Configure both: AWS maintenance takes one down at a time.

On-premisesCustomer gatewaypublic or private outside IPAWS RegionTunnel 1 endpointAZ a · 203.0.113.10Tunnel 2 endpointAZ b · 198.51.100.20IPsec + BGP 169.254.11.0/30IPsec + BGP 169.254.12.0/30Virtual private gwone VPCTransit Gatewaymany VPCs, ECMPCloud WANcore network edgeno ECMP, no 5 GbpsECMP, 5 Gbps, accelerated5 Gbps tunnels

Traffic from on-premises to AWS can use both tunnels. Traffic from AWS to on-premises prefers one tunnel (unless ECMP on Transit Gateway or Cloud WAN) and fails over automatically.

Task 2.1 · Tunnel options

What you can tune per tunnel (and the defaults)

AWS evaluates proposals starting from the lowest configured value; restrict the lists with modify-vpn-tunnel-options.

OptionDefaultAllowed
Inside tunnel CIDR (IPv4)Assigned by AWS/30 from 169.254.0.0/16; 169.254.0.0/30–169.254.5.0/30 and 169.254.169.252/30 reserved
Inside tunnel CIDR (IPv6)Assigned by AWS/126 from fd00::/8 (TGW only)
Pre-shared keyGenerated8–64 chars, alphanumeric, . and _, not starting with 0 (or certificate auth)
IKE versionsikev1, ikev2Use ikev2 only where the device supports it
Phase 1 / Phase 2 lifetime28,800 s / 3,600 s900–28,800 s / 900–3,600 s (Phase 2 < Phase 1)
Rekey margin / fuzz270 s / 100%60 s to half of Phase 2 / 0–100%
Replay window1,024 packets64–2,048
DPD timeout / action30 s / clear≥ 30 s / clear, none, restart
Startup actionadd (your device initiates)start: AWS initiates IKE

Encryption: AES128, AES256, AES128-GCM-16, AES256-GCM-16. Integrity: SHA1, SHA2-256/384/512. DH groups 2 and 14–24 (Phase 2 also allows 5). Tunnel endpoint lifecycle control lets you schedule AWS maintenance replacements yourself.

Task 2.1 · Routing over VPN

Static vs dynamic routing, and the route limits

Dynamic (BGP) is the default choice: failover, ECMP and route updates without tickets.

Static

  • You list on-premises prefixes on the VPN connection.
  • Failover between tunnels uses DPD only.
  • No ECMP on Transit Gateway.
  • Only for devices without BGP.

Dynamic (BGP)

  • One BGP session inside each tunnel (inside /30).
  • Required for ECMP on Transit Gateway.
  • Withdrawn routes fail over immediately.
  • Propagate into VPC (VGW) or TGW route tables.

Route quotas (not adjustable)

DirectionVGWTGW
CGW → AWS, dynamic1001,000
CGW → AWS, static100—
AWS → CGW1,0005,000

Advertised routes on TGW come from the route table associated with the VPN attachment; on a VGW, from the VPC. Exceeding 100 routes on a VGW is a classic reason to move to a Transit Gateway.

Task 3.3 · VPN throughput

Bandwidth, packets per second, and MTU

A tunnel is the unit of scale. Go wider with ECMP, or bigger with Large Bandwidth Tunnels.

Per-tunnel limits

Tunnel typeBandwidthPPS
Standardup to 1.25 Gbpsup to 140,000
Large Bandwidth Tunnel (Nov 2025)up to 5 Gbpsup to 400,000
VPN Concentrator tunnelup to 100 Mbpsup to 10,000

Scaling out

  • ECMP on Transit Gateway: several VPN connections, dynamic routing, VPN ECMP enabled on the TGW.
  • Example: 2 LBT connections × 2 tunnels × 5 Gbps = 20 Gbps aggregate.
  • A single flow still rides one tunnel.
  • LBT: TGW or Cloud WAN only, both tunnels same size, fixed CGW IP, no Accelerated VPN.

MTU

  • Max packet inside the tunnel: 1,446 bytes; MSS 1,406.
  • AES-CBC and NAT-T reduce it: 1,422 / 1,382 for AES-CBC + SHA-256 + NAT-T.
  • No jumbo frames and no Path MTU Discovery.
  • Clamp MSS on the customer gateway, or large transfers stall.
Task 2.1 · VPN variants

Accelerated VPN, VPN Concentrator, and private IP VPN

Three options that change where the tunnel lands or how many sites share it.

VariantWhat it doesRules and limits
Accelerated VPNTwo AWS-managed Global Accelerator accelerators (one per tunnel) bring traffic onto the AWS backbone at the nearest edgeTransit Gateway only; set at creation (cannot toggle later); NAT-T required; CGW must initiate IKE; not with a DX public VIF; not with LBT
VPN Concentrator (Nov 2025)One TGW / Cloud WAN attachment aggregates many low-bandwidth sitesUp to 100 remote sites per concentrator; up to 100 Mbps per tunnel; no ECMP; aimed at 25+ small sites
Private IP VPNIPsec over a Direct Connect transit VIF with private outside IPs (RFC 1918 or RFC 6598)Needs TGW + DX gateway; the TGW CIDR block must be in the DX gateway allowed prefixes; higher route scale than DX alone
VPN over a public VIFStandard VPN whose AWS endpoints are reached over a public VIFOlder pattern: exposes public endpoints; private IP VPN is preferred

Private IP VPN solves "encrypt everything on Direct Connect without public IPs". MACsec (Part 4.3) is the alternative at layer 2, only on dedicated 10/100/400 Gbps connections.

Task 2.1 · Private IP VPN

IPsec over Direct Connect without public IP addresses

Encrypted and unencrypted traffic can share the same transit VIF, using different TGW route tables.

On-premisesCustomer gatewayoutside IP 10.200.0.10Direct ConnectTransit VIFto DX gatewayAWS RegionDX gatewayallowed: 192.0.2.0/24Transit GatewayCIDR block 192.0.2.0/24VPC AVPC BDX attachmentgreen: IPsec tunnels to 192.0.2.xride the transit VIF and DX gateway

Order of build: give the TGW a CIDR block → associate the DX gateway with the TGW and put that CIDR in allowed prefixes → create the VPN with outside IP type PrivateIpv4 and transport attachment = the DX gateway attachment.

Task 2.1 · Build it

Create a dynamic VPN on a Transit Gateway

CLI on the AWS side, strongSwan + FRR on a Linux "on-premises" router (used in Lab 4).

aws ec2 create-customer-gateway --type ipsec.1 \
  --bgp-asn 65010 --ip-address 198.51.100.7
aws ec2 create-vpn-connection --type ipsec.1 \
  --customer-gateway-id cgw-0abc \
  --transit-gateway-id tgw-0def \
  --options file://vpn-opts.json
# vpn-opts.json: StaticRoutesOnly=false,
#   TunnelOptions[].TunnelInsideCidr =
#   169.254.11.0/30 and 169.254.12.0/30, IKEv2 only
aws ec2 describe-vpn-connections \
  --vpn-connection-ids vpn-0123 --query \
  'VpnConnections[0].VgwTelemetry[].[OutsideIpAddress,Status]'
# download a device config (strongSwan, Cisco, Palo Alto...)
aws ec2 get-vpn-connection-device-sample-configuration \
  --vpn-connection-id vpn-0123 \
  --vpn-connection-device-type-id <id>
# customer gateway (Linux), tunnel 1 of 2
# xfrm interface carries the BGP session
ip link add xfrm1 type xfrm dev ens5 if_id 1
ip addr add 169.254.11.2/30 dev xfrm1
ip link set xfrm1 mtu 1422 up   # AES-CBC+SHA-256+NAT-T
iptables -t mangle -A FORWARD -o xfrm1 -p tcp \
  --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1382

# FRR (vtysh): BGP over the tunnel inside /30
router bgp 65010
 neighbor 169.254.11.1 remote-as 64512
 address-family ipv4 unicast
  network 10.200.0.0/24

# strongSwan config: next slide
Task 2.1 · Build it

strongSwan route-based tunnel for the VPN

One key = value per line (the swanctl.conf grammar has no ; separator). if_id binds the child SA to xfrm1.

# /etc/swanctl/conf.d/aws.conf (tunnel 1)
connections {
  aws1 {
    version = 2
    local_addrs = 10.200.0.10
    remote_addrs = 203.0.113.10
    proposals = aes256-sha256-modp2048
    local {
      auth = psk
      id = 198.51.100.7
    }
    remote {
      auth = psk
      id = 203.0.113.10
    }
    children {
      aws1 {
        local_ts = 0.0.0.0/0
        remote_ts = 0.0.0.0/0
        if_id_in = 1
        if_id_out = 1
        start_action = start
        dpd_action = restart
      }
    }
  }
}
secrets {
  ike-aws1 {
    id-1 = 198.51.100.7
    id-2 = 203.0.113.10
    secret = "<tunnel 1 pre-shared key>"
  }
}

swanctl --load-all
swanctl --list-sas
#  IKE_SA ESTABLISHED

# tunnel 2: copy as aws2 with the second
# outside IP, its PSK, if_id 2, xfrm2 and
# 169.254.12.2/30

Why it looks like this

  • Behind NAT (an EIP), local_addrs is the private IP and the IKE id is the public IP AWS knows.
  • NAT-T moves IKE and ESP to UDP 4500.
  • 10.200.0.10 = cgw private IP; 198.51.100.7 = its public IP (the customer gateway); 203.0.113.10 = AWS tunnel 1 outside IP.
  • esp_proposals can be added; the defaults cover the AWS tunnel proposals.
  • 0.0.0.0/0 selectors + if_id = route-based: BGP, not IPsec policy, picks the tunnel.
  • The full two-tunnel build is Lab 4.
Task 4.3 · Client VPN

AWS Client VPN for remote users

Managed OpenVPN-based endpoint: users connect to a VPC, then reach peered VPCs, TGW, on-premises or the internet through it.

Authentication

  • Mutual certificate (ACM-imported server and client certs).
  • Active Directory via AWS Directory Service.
  • SAML 2.0 federation (Okta, Entra ID, IAM Identity Center) with group claims.
  • Combine mutual cert + AD/SAML for two factors.

Network model

  • Client CIDR pool: must not overlap the VPC or routes you add.
  • Target network associations: subnets (one per AZ for HA); IPv4 traffic is source-NATed to the association ENIs.
  • Route table entries say which destinations the endpoint can reach.
  • Authorization rules (per CIDR, per AD/SAML group) say who may reach them.

Choices the exam asks about

  • Split tunnel: only VPC/on-prem routes go to the VPN; internet stays local. Off by default.
  • Full tunnel to internet: add a 0.0.0.0/0 route via a subnet with NAT and an authorization rule.
  • Connection logs to CloudWatch Logs; client connect handler (Lambda) for posture checks.
  • For identity- and device-aware access to apps without a VPN: Verified Access (Ch 7).
Task 3.2 · Operate VPN

Monitor and troubleshoot tunnels

Most VPN outages are a mismatched proposal, a DPD timeout, or MTU.

Visibility

  • CloudWatch metrics per tunnel: TunnelState (1 up / 0 down), TunnelDataIn, TunnelDataOut.
  • Alarm when TunnelState max < 1 for either tunnel.
  • Site-to-Site VPN logs to CloudWatch Logs: IKE and BGP negotiation details.
  • TGW flow logs or Network Manager for route and topology views.

Symptom → cause

  • Phase 1 fails → PSK, IKE version, or proposal mismatch.
  • Phase 2 fails → traffic selectors / encryption domain mismatch (use 0.0.0.0/0 route-based).
  • Tunnel flaps every few minutes → DPD or NAT-T issues, lifetime mismatch.
  • BGP Idle → wrong inside IP or ASN, or > 100 prefixes on a VGW.
  • Ping works, HTTPS hangs → MTU / MSS.

Maintenance

  • AWS replaces tunnel endpoints during maintenance: one tunnel at a time.
  • Tunnel endpoint lifecycle control: get notified and choose when to apply.
  • Keep both tunnels up and BGP on both.
Exam traps

VPN traps

Wrong answers that look right.

More bandwidth on a VGW

  • A VGW cannot do ECMP, Large Bandwidth Tunnels or Accelerated VPN.
  • The answer is a Transit Gateway (or Cloud WAN).

ECMP with static routes

  • ECMP on TGW needs dynamic (BGP) VPN connections.

Accelerated after the fact

  • You cannot enable acceleration on an existing VPN. Create a new one.

Jumbo over VPN

  • VPN has no jumbo frames and no PMTUD; 1,446-byte packets max.

Encrypting DX with a public VIF

  • Private IP VPN over a transit VIF removes public endpoints.
  • MACsec only on dedicated 10/100/400 Gbps.

One tunnel configured

  • AWS maintenance will take it down. Always both tunnels.
Exam-style scenarios

VPN scenarios

Scenario → answer → why.

ScenarioAnswerWhy
Need 4 Gbps of encrypted bandwidth to 30 VPCs over the internetTGW + dynamic VPN with ECMP (or Large Bandwidth Tunnels)Standard tunnels are 1.25 Gbps; ECMP needs BGP on TGW
Branch in another continent sees jitter on VPN to the RegionAccelerated Site-to-Site VPN on TGWEnters the AWS backbone at the nearest edge
Compliance: all DX traffic must be IPsec-encrypted, no public IPsPrivate IP VPN over transit VIFIPsec with private outside addresses
80 retail stores, each needing ~20 Mbps to AWSVPN Concentrator on TGWBuilt for many low-bandwidth sites, up to 100 per concentrator
VPN up, SSH works, file copies hangLower MTU / clamp MSS on the CGWPackets over 1,446 bytes are dropped; no PMTUD
Remote staff need access to a VPC with Okta groups controlling which subnetsClient VPN with SAML + authorization rules per groupGroup-based authorization on Client VPN
Chapter 4 · Part 4.3

AWS Direct Connect — physical and logical

A private circuit into AWS: ports, cross connects, LAGs, MACsec, virtual interfaces, and the DX gateway.

Task 1.5 · Building blocks

From your router to a VPC

A Direct Connect location is a colocation facility where AWS has routers. You (or a partner) bring a fiber there.

Customer data centerCustomer routerBGP, 802.1QDirect Connect location (colo)Customer / partnercage or partner portAWS DX routeryour portcross connectlast mileAWSPrivate VIF → VGW / DX gatewayVPC private IPsTransit VIF → DX gatewayTransit Gateway / Cloud WANPublic VIFAWS public endpoints (S3, ...)one port, many VLANs= many VIFs

A connection is layer 1–2 (a port and a fiber). A virtual interface (VIF) is layer 3: a VLAN plus a BGP session. Nothing flows until a VIF exists.

Task 2.1 · Ordering a dedicated port

Request → LOA-CFA → cross connect → BGP

The physical steps are the slow part; plan weeks, not hours.

1 · Request

  • Pick location and port speed (cannot change later).
  • Use the Connection wizard (resiliency models) or Classic.
  • AWS reviews: up to 72 business hours.
  • Answer any info request within 7 days or the request is deleted.
→

2 · LOA-CFA

  • Letter of Authorization – Connecting Facility Assignment.
  • Names your patch panel and ports; digitally signed.
  • Give it to the colo or partner to order the cross connect.
→

3 · Cross connect

  • Fiber between your (or partner) equipment and the AWS port.
  • Single-mode fiber: 1G 1000BASE-LX, 10G 10GBASE-LR, 100G 100GBASE-LR4, 400G 400GBASE-LR4.
  • Check light levels when the link comes up.
→

4 · VIFs and BGP

  • Create private, transit or public VIF.
  • VLAN 1–4094, BGP ASN, MD5 key, peer IPs.
  • Bring up BGP; advertise prefixes.

Billing starts when the port is active or 90 days after the LOA is issued, whichever comes first. Delete the port before then to avoid charges.

Task 1.5 · Dedicated vs hosted

Two ways to get a connection

Hosted connections come from a Direct Connect Partner who already has a port at the location.

Dedicated connectionHosted connection
Speeds1, 10, 100, 400 Gbps50, 100, 200, 300, 400, 500 Mbps; 1, 2, 5, 10, 25 Gbps (higher tiers only from qualified partners)
Who ordersYou, in the console or APIThe partner creates it; you accept it
VIFs50 private/public + up to 4 transit (51 total)1 VIF of any type
Change speedNo: create a new connectionPartner can upgrade or downgrade (if supported)
LAGYesNo
MACsec10, 100, 400 Gbps at selected locationsNot supported
Jumbo framesPer VIFOnly if the partner’s parent connection has it
Rate behaviourPort speedPoliced at the purchased rate: bursts are dropped

"Need 200 Mbps quickly from a city without our own colo" → hosted connection. "Need 100 Gbps with MACsec" → dedicated.

Task 1.5 · LAG and MACsec

Bundle ports, and encrypt them at layer 2

A link aggregation group (LAG) uses LACP to make several dedicated ports act as one.

Link aggregation group

  • Up to 4 connections per LAG below 100 Gbps; up to 2 at 100 Gbps.
  • All members: same speed, same location, same AWS device.
  • VIFs attach to the LAG, not to members.
  • Minimum links: the LAG goes down if fewer members are up — set it to avoid an overloaded remainder.
  • A LAG is one device, so it is not a resiliency design by itself.

MACsec (IEEE 802.1AE)

  • Point-to-point encryption between your router and the AWS DX device.
  • Dedicated 10, 100, 400 Gbps at selected locations; not hosted.
  • Ciphers: GCM-AES-256 or GCM-AES-XPN-256 on 10G; XPN-256 required on 100G/400G.
  • Static CAK mode only: you generate CKN/CAK (64 hex chars) and associate them.
  • Keychain holds up to 3 CKN/CAK pairs for rotation.

Encryption modes

  • should_encrypt (default): falls back to clear if negotiation fails.
  • must_encrypt: no MACsec, no traffic.
  • Secure Channel Identifier (SCI) must be on.
  • Not end to end: only the first hop. Use IPsec or TLS for end-to-end.
Task 2.1 · Virtual interfaces

Private, transit and public VIFs

Pick the VIF type by what you need to reach. Each is one VLAN plus one BGP session per address family.

Private VIFTransit VIFPublic VIF
ReachesOne VPC via VGW, or VPCs via DX gateway + VGWsTransit Gateways or a Cloud WAN core network via DX gatewayAll AWS public prefixes (S3, DynamoDB, APIs, EC2 public IPs)
AddressesPrivate (or public) prefixesPrivate (or public) prefixesPublic prefixes you own (RIR-registered)
MTU1500 or 90011500 or 85001500
Prefixes you → AWS100 default, up to 1,000 (prefix controls)100 default, up to 1,0001,000 (fixed)
Prefixes AWS → youVPC CIDRs (filtered by allowed prefixes)Allowed prefixes; 200 per TGW (v4+v6)All Region and edge prefixes (NO_EXPORT)
Per dedicated connection50 private + public combinedUp to 4Counts in the 50
SiteLinkYes, if via DX gatewayYesNo

BGP peer IPs: IPv4 /30 or /31 (you choose or AWS assigns); IPv6 /125 always assigned by Amazon. Multiple BGP sessions of the same address family on one VIF are not allowed.

Task 1.6 · Direct Connect gateway

A global, route-reflecting hub for VIFs

A DX gateway is not in the data path. It is a distributed set of BGP route reflectors, so one is enough.

On-premisesRouterAS 65010DX gatewayglobal · ASN 64600transit VIFprivate VIFus-east-1ap-northeast-1Transit Gateway Aallowed 10.0.0.0/12VGW · VPC 10.32.0.0/16Transit Gateway Ballowed 10.16.0.0/12Cloud WAN edge (optional)Limits per DX gateway20 VGWs · 6 TGWs · 30 VIFsNot a transit routerVGW ↔ VGW traffic blocked

A DX gateway can associate VGWs and TGWs in any Region (and accounts) except China. Use private VIFs for VGW associations and transit VIFs for TGW associations.

Task 1.6 · Allowed prefixes

Allowed prefixes mean different things for VGW and TGW

A frequent exam detail: filter on one, advertisement on the other.

VGW association (filter)

Allowed prefixVPC 10.0.0.0/16 advertised?
22.0.0.0/24No route
10.0.0.0/24No (narrower than the VPC CIDR)
10.0.0.0/15Yes: 10.0.0.0/16

TGW association (exactly what is advertised)

Allowed prefixYou receive
22.0.0.0/2422.0.0.0/24 (even though no VPC uses it)
10.0.0.0/2410.0.0.0/24
10.0.0.0/810.0.0.0/8 (a summary)

Rules

  • TGW: allowed prefixes are your summarization tool (max 200 per TGW toward on-premises).
  • Allowed prefixes cannot overlap across multiple TGWs on one DX gateway (so no 0.0.0.0/0 with two TGWs).
  • Cross-account: the VGW/TGW owner sends an association proposal; the DX gateway owner accepts and can edit the allowed prefixes.
  • Changing a prefix only affects traffic using that prefix.
Task 1.5 · Public VIF

Reach AWS public services privately

Traffic to S3, DynamoDB or public endpoints goes over your circuit instead of the internet.

Inbound rules (you → AWS)

  • You must own the public prefixes (RIR-registered); AWS validates.
  • Traffic must be destined to Amazon prefixes; no transit between your connections.
  • AWS filters packets whose source is not in your advertised prefixes.
  • Up to 1,000 prefixes; any length /1–/32 (IPv4).

Outbound rules (AWS → you)

  • AWS advertises all local and remote Region prefixes plus edge (CloudFront, Route 53) prefixes.
  • Every route carries NO_EXPORT; do not leak them to the internet.
  • AWS advertises with a minimum path length of 3.
  • Filter with communities 7224:8100 / 7224:8200 to keep only nearby prefixes.

Design notes

  • A public VIF reaches every AWS public prefix, including other customers’ public IPs: put a firewall in front.
  • For only S3/DynamoDB from on-premises, an interface endpoint over a private/transit VIF is often cleaner (Ch 2).
  • Private ASN: prepends are stripped (replaced by 7224).
Task 1.5 · SiteLink

Site-to-site over the AWS backbone

SiteLink lets two Direct Connect locations talk to each other directly, without going through a Region.

Office in TokyoRouterAS 65010Office in SingaporeRouterAS 65020DX locationTokyoDX locationSingaporeDX gatewaySiteLink on both VIFsVIFVIFshortest path on AWS backboneSupported: transit VIF; private VIF on a DX gatewayNot supported: private VIF on a VGW, public VIFNot in GovCloud or China · separate per-hour + data fee

With SiteLink enabled, Regions prefer the path with the shorter AS_PATH regardless of associated Region. SiteLink fails if your router advertises the same route on multiple VIFs. Jumbo up to 8500 or 9001 depending on VIF type.

Task 3.3 · Jumbo frames on DX

MTU per VIF type, and what silently drops you to 1500

Ethernet frames of 1522 or 9023 bytes at layer 2; the IP MTU is what you configure.

MTU

PathMTU
Private VIF1500 or 9001
Transit VIF1500 or 8500
Transit Gateway (any jumbo)8500
Public VIF1500
Site-to-Site VPN1,446 inside the tunnel

Gotchas

  • Enabling jumbo on a VIF can update the physical connection: up to 30 s outage for all VIFs on it.
  • Two private VIFs advertising the same route with different MTUs → 1500 used.
  • Same route also advertised over VPN → 1500 used.
  • Jumbo applies to routes propagated via DX and static routes via TGW.
  • Hosted: only if the partner’s parent connection is jumbo-capable.
  • Check "Jumbo frame capable" on the connection first.
Task 2.1 · Build it

Create a transit VIF and associate a Transit Gateway

The minimal AWS-side commands; the BGP side is the FRR config from Part 4.1.

# 1. DX gateway (private ASN) and TGW association
aws directconnect create-direct-connect-gateway \
  --direct-connect-gateway-name corp-dxgw --amazon-side-asn 64600
aws directconnect create-direct-connect-gateway-association \
  --direct-connect-gateway-id dxgw-111 --gateway-id tgw-0def \
  --add-allowed-prefixes-to-direct-connect-gateway cidr=10.0.0.0/12

# 2. Transit VIF on the connection (or LAG)
aws directconnect create-transit-virtual-interface \
  --connection-id dxcon-abc --new-transit-virtual-interface \
  'virtualInterfaceName=tvif-1,vlan=301,asn=65010,mtu=8500,
   addressFamily=ipv4,amazonAddress=169.254.10.1/30,
   customerAddress=169.254.10.2/30,authKey=<md5>,
   directConnectGatewayId=dxgw-111'

# 3. Verify
aws directconnect describe-virtual-interfaces \
  --virtual-interface-id dxvif-123 \
  --query 'virtualInterfaces[0].[virtualInterfaceState,bgpPeers[0].bgpStatus,mtu]'
aws ec2 search-transit-gateway-routes --transit-gateway-route-table-id tgw-rtb-1 \
  --filters Name=type,Values=propagated

Then

  • Propagate the DX gateway attachment into the TGW route tables that need on-premises routes.
  • Associate the DX attachment with a route table that contains the VPC CIDRs (or uses allowed-prefix summaries).
  • Add 10.20.0.0/16 → tgw-0def in VPC subnet route tables.
  • Alarm on ConnectionState and on VirtualInterfaceBpsEgress.
  • For a second account’s TGW: it creates an association proposal; you accept.
Exam traps

Direct Connect traps

Wrong answers that look right.

DX is not encrypted

  • A private circuit is not encryption. Use MACsec (dedicated 10/100/400G) or IPsec (private IP VPN).

DX gateway transit

  • Two VPCs on the same DX gateway cannot talk to each other through it (VGW ↔ VGW).
  • Use TGW or peering for VPC-to-VPC.

Wrong VIF for TGW

  • A Transit Gateway needs a transit VIF to a DX gateway, not a private VIF.

Hosted limits

  • A hosted connection has exactly one VIF; no LAG, no MACsec.

Allowed prefixes

  • VGW: a filter that must be equal or wider than the VPC CIDR.
  • TGW: exactly what is advertised.

Speed changes

  • A dedicated port speed cannot be changed; order a new one (or add LAG members).
Exam-style scenarios

Direct Connect scenarios

Scenario → answer → why.

ScenarioAnswerWhy
One 10G DX must reach 40 VPCs in 3 RegionsTransit VIF → DX gateway → one TGW per RegionDX gateway is global; up to 6 TGWs per DX gateway
Need 30 Gbps on one logical link at one locationLAG of 4 × 10G dedicated (minimum links 3)LAG bundles same-speed ports
Compliance needs layer 2 encryption on a 100G linkMACsec with GCM-AES-XPN-256XPN required at 100G
On-prem must reach S3 public endpoints without the internetPublic VIF (or interface endpoint via private VIF)Public VIF carries AWS public prefixes
Offices in Tokyo and Sydney must talk over AWS without a Region hopSiteLink on both VIFs to a DX gatewayBackbone path between DX locations
After enabling jumbo, all VIFs on the port dropped brieflyExpected: connection update, up to 30 sPlan MTU changes in a window
Chapter 4 · Part 4.4

Hybrid resiliency and routing design

Resiliency models, active/active vs active/passive, VPN backup, route limits, asymmetric routing, and failover tests.

Task 1.5 · Resiliency models

The three models in the Direct Connect Resiliency Toolkit

Pick the model from the SLA you need; the toolkit orders the right number of ports and keeps them on separate devices.

Maximum resiliency · 99.99% SLADX location 1device ADX location 1device BDX location 2device CDX location 2device D4 connections, 2 locations, separate devicesSurvives device, link and full-location failureHigh resiliency · 99.9% SLADX location 1one connectionDX location 2one connection2 connections, 2 locationsSurvives fiber cut, device or one locationDevelopment and testDX location 1device ADX location 1device B2 connections, 1 location, 2 devicesSurvives device failure onlyNo SLA target; non-critical workloads

SLAs apply only when you meet every requirement in the Direct Connect SLA. A "Classic" connection is ordered one by one without a model. Add a VPN as a low-cost backup on top of any model.

Task 1.5 · Active/active vs active/passive

Choose how traffic uses redundant paths

Both designs need redundancy; they differ in capacity planning and predictability.

Active/activeActive/passive
AWS → on-premSame prefixes, same community (e.g. 7224:7200), same AS_PATH → ECMPPrimary 7224:7300, backup 7224:7100 (or more specifics on primary)
On-prem → AWSEqual local pref, ECMP on your routersHigher local pref on the primary edge
Capacity ruleEach path must carry 100% if the other fails (keep each under 50%)Backup sized for full load, or accept degraded mode
ProsUses all bandwidth; failures are smaller eventsPredictable path, simple troubleshooting, easier for stateful firewalls
ConsFlows hash across paths: stateful devices must see both directionsBackup capacity idle; failover is a bigger change
TypicalTwo DX connections in the same associated RegionDX primary + DX or VPN backup; cross-Region DX
Task 1.5 · VPN as DX backup

Direct Connect primary, Site-to-Site VPN backup

Cheapest resilient design. AWS prefers DX for the same prefix; your side must do the same.

On-premises AS 65010DX routerlocal-pref 200VPN routerlocal-pref 100, prepend x3AWS RegionTransit GatewayDX gw route beats VPN routeDirect Connect · transit VIF · 10.20.0.0/16VPN over internet · 10.20.0.0/16 (same prefix)

Why it works

  • TGW order: DX gateway-propagated beats VPN-propagated for the same CIDR.
  • VGW order: DX BGP beats VPN static and VPN BGP.
  • Do not advertise more specifics over VPN: longest prefix would pull traffic to the VPN.

On your side

  • Prefer routes learned over DX (local pref) for traffic to AWS.
  • Size the VPN for the critical subset of traffic; 1.25 Gbps per standard tunnel.
  • Test it: run a DX failover test and watch TGW routes switch.
Task 3.1 · Route limits and summarization

Stay under the prefix limits on every session

Exceeding a Direct Connect limit is not a soft warning: the BGP session goes idle.

Limits that matter

SessionLimit
Private/transit VIF, you → AWS100 per family (prefix controls up to 1,000)
Public VIF, you → AWS1,000 (fixed)
Transit VIF, AWS → you per TGW200 (IPv4 + IPv6)
Cloud WAN DX gateway attachment → you5,000
VPN on VGW, you → AWS100
VPN on TGW, you → AWS / AWS → you1,000 / 5,000
Private IP VPN1,000 in / 5,000 out

Techniques

  • Summarize on-prem prefixes on the edge routers before advertising.
  • Use TGW allowed prefixes as summaries toward on-prem (10.0.0.0/12 instead of 40 VPC CIDRs).
  • Plan VPC CIDRs from contiguous IPAM pools so they summarize.
  • Need more than DX allows? Private IP VPN over DX has higher route scale.
  • Avoid overlapping on-prem and VPC ranges: summaries hide overlaps until they break.
Task 3.2 · Asymmetric routing

When the request and the reply take different paths

Routing still "works", but stateful firewalls, NAT and Network Firewall drop the half they never saw.

On-premisesServer10.20.5.10Firewall ADX edgeFirewall BVPN edgeAWSTransit GatewayVPC 10.0.0.0/16request: out via DX (on-prem prefers DX)reply: via VPN, which advertises a more specific /24Firewall B never saw the SYN → drops the reply

Fixes: advertise identical prefixes on both paths and steer with communities/local pref, keep stateful devices on a shared session table, or place inspection where both directions pass (TGW appliance mode for inspection VPCs, Ch 7).

Task 4.1 · Failover testing

Prove the backup works before a real outage

The Resiliency Toolkit failover test shuts down BGP sessions on chosen VIFs for a set time.

# bring BGP down on one VIF for 60 minutes
aws directconnect start-bgp-failover-test \
  --virtual-interface-id dxvif-primary \
  --test-duration-in-minutes 60

# watch: routes move, traffic shifts
aws ec2 search-transit-gateway-routes \
  --transit-gateway-route-table-id tgw-rtb-1 \
  --filters Name=route-search.exact-match,Values=10.20.0.0/16

# end early and review history
aws directconnect stop-bgp-failover-test \
  --virtual-interface-id dxvif-primary
aws directconnect list-virtual-interface-test-history \
  --virtual-interface-id dxvif-primary

What to know

  • Works on public, private, transit and hosted VIFs.
  • Default duration 180 minutes; maximum 4,320 minutes (72 hours).
  • BGP status shows DOWN during the test; traffic must move to the redundant VIF or VPN.
  • Measure: time to reroute (BFD helps), packet loss, application errors.
  • Also test the reverse direction from your routers.
  • Document results: auditors ask for failover evidence (Task 4.1).
Task 3.2 · Monitor Direct Connect

CloudWatch metrics for connections and VIFs

Alarm on the physical layer and on BGP, not only on application symptoms.

MetricLevelUse
ConnectionStateConnection1 up / 0 down: alarm when < 1
ConnectionBpsEgress / Ingress, ConnectionPpsEgress / IngressConnectionCapacity planning; keep active/active paths under 50%
ConnectionLightLevelTx / RxConnection (dedicated)Optical health; dropping dBm predicts a failing fiber
ConnectionErrorCountConnectionMAC-level errors incl. CRC (Sum): dirty optics or bad patching
ConnectionEncryptionStateConnection (MACsec)Alarm if encryption drops in should_encrypt mode
VirtualInterfaceBpsEgress / IngressVIFPer-VIF usage, chargeback

Alarm on VirtualInterfaceBgpStatus for BGP down. Maintenance appears in AWS Health events; route changes are visible in Transit Gateway Network Manager (Ch 8).

Exam traps

Resiliency traps

Wrong answers that look right.

Two ports, one device

  • Two connections on the same device or LAG do not give location or device resiliency.
  • Maximum resiliency = separate devices in two locations.

More specifics over VPN

  • They make the VPN primary. Advertise the same prefixes over both.

Overloaded active/active

  • Two paths at 70% each cannot absorb a failure.

SLA without requirements

  • The 99.99% SLA applies only if you meet all model requirements.

Untested failover

  • "Verify the design meets resiliency requirements" → Resiliency Toolkit failover test.

Stateful edges

  • Asymmetric paths break stateful firewalls; symmetric steering or shared state fixes it.
Exam-style scenarios

Resiliency scenarios

Scenario → answer → why.

ScenarioAnswerWhy
Critical workload, must survive loss of a whole DX location, 99.99%Maximum resiliency: 2 locations × 2 connections on separate devicesOnly model with 99.99% SLA
Cost-sensitive, one DX already, needs a backupSite-to-Site VPN on the same TGW as backupDX routes win; VPN takes over on withdrawal
Prove to auditors that failover worksDirect Connect failover test on the primary VIFControlled BGP shutdown with history
Firewall logs show half-open sessions after a DX changeAsymmetric routing: align prefixes and local prefRequest and reply on different edges
50 VPC CIDRs exceed 200 prefixes per TGW toward on-premSummarize with allowed prefixes on the DX gateway associationTGW allowed prefixes are advertised exactly
Failover to backup DX takes 90 sEnable BFD on both VIFsDetects in about 900 ms
Chapter 4 · Part 4.5

SD-WAN, MPLS and multicloud

Overlay networks into Transit Gateway and Cloud WAN, existing WANs, and private links to other clouds.

Task 1.5 · Transit Gateway Connect

GRE + BGP from SD-WAN appliances to Transit Gateway

A Connect attachment rides on a transport attachment (a VPC attachment or a DX gateway attachment).

Appliance VPCSD-WAN appliance AGRE outer 10.1.1.10SD-WAN appliance BGRE outer 10.1.2.10VPC attachmenttransportTransit GatewayCIDR block 192.0.2.0/24GRE to 192.0.2.1GRE to 192.0.2.2Connect peer ABGP /29 × 2 sessionsConnect peer BBGP /29 × 2 sessions
ItemRule
Inside CIDR/29 from 169.254.0.0/16 (first IP on the appliance); 169.254.0.0/29–169.254.5.0/29 and 169.254.169.248/29 reserved; optional /125 from fd00::/8
BGPTwo sessions per Connect peer (configure both); eBGP needs multihop TTL 2; MP-BGP carries IPv6 over IPv4 peering
Not supportedStatic routes, BFD, graceful restart
ECMPSame prefixes, same AS_PATH and ASN across peers; routes propagate by default
MTUGRE tunnel MTU = outer MTU − 24 (e.g. 1476 over 1500)
Task 2.1 · Build it

Create a Connect attachment and peer

AWS side first, then a Linux GRE tunnel and FRR on the appliance (a stand-in for an SD-WAN box).

aws ec2 modify-transit-gateway --transit-gateway-id tgw-0def \
  --options AddTransitGatewayCidrBlocks=192.0.2.0/24
aws ec2 create-transit-gateway-connect \
  --transport-transit-gateway-attachment-id tgw-attach-vpc \
  --options Protocol=gre
aws ec2 create-transit-gateway-connect-peer \
  --transit-gateway-attachment-id tgw-attach-connect \
  --peer-address 10.1.1.10 --transit-gateway-address 192.0.2.1 \
  --inside-cidr-blocks 169.254.6.0/29 \
  --bgp-options PeerAsn=65050
# appliance subnet route table: 192.0.2.0/24 -> tgw-0def
# on the appliance (Linux)
ip tunnel add gre1 mode gre local 10.1.1.10 remote 192.0.2.1 ttl 255
ip addr add 169.254.6.1/29 dev gre1
ip link set gre1 mtu 8476 up      # 8500 TGW MTU - 24
# FRR: two sessions, to .2 and .3 of the /29
router bgp 65050
 neighbor 169.254.6.2 remote-as 64512
 neighbor 169.254.6.2 ebgp-multihop 2
 neighbor 169.254.6.3 remote-as 64512
 neighbor 169.254.6.3 ebgp-multihop 2
 address-family ipv4 unicast
  network 172.16.0.0/12

Verify: describe-transit-gateway-connect-peers shows both BGP sessions up; on-premises branch prefixes appear as Connect-propagated routes in the TGW route table.

Task 1.5 · Overlays and WAN integration

GRE, IPsec, Cloud WAN Connect, MPLS and the legacy transit VPC

Choose the overlay by what the existing WAN already runs.

OptionHow it worksPick it when
TGW Connect (GRE)GRE + BGP over VPC or DX transport; higher throughput than IPsec, no encryptionSD-WAN appliances in a VPC or on-prem over DX
Cloud WAN ConnectSame GRE model on a core network edge; tunnel-less Connect (Oct 2023) peers BGP directly without GREGlobal SD-WAN on Cloud WAN segments
IPsec VPNEncrypted; 1.25 or 5 Gbps per tunnelEncryption required over the internet
MPLS providerThe carrier is a Direct Connect partner and hands off a hosted connection or VIF at a DX locationAn enterprise WAN already on MPLS: AWS becomes another site
Transit VPC (legacy)VPN from every VPC to router instances in a hub VPCOnly for exam history: replaced by Transit Gateway (managed, scalable, no instances)
Third-party appliancesMarketplace firewalls/routers behind GWLB or as Connect peersExisting vendor tooling and skills

GRE is not encryption. If the exam says "encrypt", the overlay must be IPsec (or MACsec on the DX hop), not Connect.

Task 1.5 · Multicloud

AWS Interconnect – multicloud

A managed private link from your VPCs to another cloud provider, carried on the AWS backbone and handed off directly.

Status (Oct 2026)

  • GA with Google Cloud: 14 Apr 2026.
  • GA with Oracle Cloud Infrastructure: Jul 2026 (us-east-1).
  • Microsoft Azure: preview since Aug 2026.
  • Region pairs are limited; check the live list.

How it works

  • Created from the Direct Connect console: pick the provider, Regions and bandwidth.
  • Provide the other cloud’s project ID; AWS issues an activation key for the partner side.
  • No customer routers, colo cages or cross connects.
  • Bandwidth can be changed without recreating the connection.
  • Hourly price by bandwidth tier and Region pair.

Before it existed

  • Colocate routers in a facility where both clouds have ports (DX + partner interconnect).
  • Or IPsec VPN between cloud VPN gateways over the internet.
  • Both are still valid answers when a provider or Region pair is not supported.

Not in the ANS-C01 guide. For the exam, multicloud and branch problems still map to DX + partner, VPN, or SD-WAN via TGW Connect.

Task 1.5 · Decision guide

Which hybrid connection for which requirement

Read the constraint words in the stem.

RequirementChooseNot
Fast to set up, encrypted, < ~1 GbpsSite-to-Site VPN (TGW for scale)Direct Connect (weeks to provision)
Consistent latency, high throughput, lower data transfer rateDirect Connect (dedicated or hosted)VPN over internet
Encrypted and private at high speedDX + MACsec, or private IP VPN over transit VIFPublic VIF + VPN
Many VPCs and Regions over one DXTransit VIF + DX gateway + TGWs (or Cloud WAN)One private VIF per VPC
Many small branchesVPN Concentrator, or SD-WAN via ConnectOne VPN per branch on a VGW
Remote employeesClient VPN or Verified AccessSite-to-Site VPN
Branch-to-branch over AWSDX SiteLink, or Cloud WANHairpin through a VPC
Exam traps

SD-WAN and multicloud traps

Wrong answers that look right.

Connect is not encrypted

  • GRE carries no encryption. Use IPsec when encryption is required.

Static routes on Connect

  • Connect supports BGP only; no static routes.

One BGP session per peer

  • Configure both sessions of each Connect peer, or AWS maintenance causes a blip.

Wrong transport

  • A Connect attachment needs an existing VPC or DX gateway attachment as transport.

Transit VPC

  • Recommending EC2 router instances in a hub VPC is the legacy pattern; Transit Gateway replaces it.

MTU on GRE

  • Forgetting the 24-byte GRE overhead causes fragmentation and drops.
Exam-style scenarios

SD-WAN and multicloud scenarios

Scenario → answer → why.

ScenarioAnswerWhy
SD-WAN appliances in a VPC must exchange routes with 50 VPCs dynamically at high throughputTGW Connect attachment with Connect peers (GRE + BGP)No IPsec overhead; BGP propagation
Same, but the SD-WAN head-ends are on-premises behind DXConnect attachment using the DX gateway attachment as transportConnect supports DX transport
Existing MPLS WAN; add AWS as a site with minimum changeAsk the MPLS carrier (DX partner) for a hosted connection / VIFCarrier hands off at a DX location
Private link from AWS VPCs to Google Cloud VPCs without managing routersAWS Interconnect – multicloudManaged, backbone-carried link (GA Apr 2026)
Company runs EC2-based transit VPC; routing scale and HA are problemsMigrate to Transit GatewayManaged regional router, ECMP, route tables
Two Connect peers; traffic uses only oneAdvertise the same prefixes with the same AS_PATH and ASN from bothECMP requires identical attributes
Self-check

Hybrid connectivity: can you answer these?

Explain the answer out loud and name the exact feature or value.

BGP and VPN

  • In what order does AWS evaluate prefix length, local pref, AS_PATH and MED?
  • Which community makes a VIF the backup path?
  • Why might prepending fail on a public VIF?
  • Name two things a VGW cannot do that a TGW VPN can.
  • Why do large file copies hang over a working VPN?

Direct Connect

  • Dedicated vs hosted: speeds, VIF count, MACsec?
  • What does an allowed prefix do on a VGW vs a TGW association?
  • MTU of a private VIF, a transit VIF and a public VIF?
  • What happens when you advertise 101 prefixes on a transit VIF?
  • When do you need a public VIF instead of a private one?

Design and operations

  • Requirements of the maximum resiliency model?
  • How do you make a VPN a pure backup for DX on a TGW?
  • How do you prove failover works to an auditor?
  • How do you encrypt DX traffic without public IPs?
  • What does a Connect peer need (CIDR, sessions, TTL)?
Chapter 5

DNS with Route 53

Tasks 1.2, 2.3, 3.3, 4.3 — public and private zones, routing policies, hybrid resolution, DNSSEC and DNS security.

Chapter 5 · Part 5.1

DNS refresher and public DNS

Records, delegation, alias records, domain registration and DNSSEC signing.

Task 1.2 · Resolution path

How a name is resolved, end to end

Recursive resolvers do the walking; authoritative servers only answer for their own zone.

Clientstub resolverRecursive resolverVPC Resolver / ISPCachekept for TTL secondsRoot servers"ask .com".com TLD servers"ask ns-123.awsdns-…"Route 53 authoritativeexample.com zone1 query5 answer234Delegation: each parent zone holds NS records (plus glue A records) that point to the child zone’s name servers.Route 53 gives every public hosted zone four name servers in different TLDs (.com, .net, .org, .co.uk) for resilience.

Lower the TTL (for example to 60 s) a day before a planned cut-over, then raise it again afterwards. Changing TTL at the moment of the change does not help: resolvers already cached the old value with the old TTL.

Task 2.3 · Record types

Record types you must know

The exam asks which record solves a requirement, and what a record cannot do.

TypePurposeWatch out for
A / AAAAName → IPv4 / IPv6 addressDual-stack clients query both; publish AAAA only if the IPv6 path works
CNAMEName → another nameNot allowed at the zone apex; cannot coexist with other records of the same name
Alias (Route 53 only)Name → AWS resource or another record in the zone, answered as A/AAAAWorks at the apex; no TTL of your own when the target is an AWS resource
MX / TXTMail routing; free text (SPF, DKIM, domain verification)TXT values over 255 characters are split into several strings
PTRIP → name (reverse lookup), in in-addr.arpa / ip6.arpa zonesReverse zones for VPC CIDRs are answered by VPC Resolver automatically
NS / SOADelegation to name servers; zone metadata incl. negative-caching TTLThe NS set in the zone must match the NS set at the registrar
SRV / CAA / DSService location; which CAs may issue certs; DNSSEC chain to the parentCAA must allow amazon.com (or amazontrust.com) for ACM issuance
Task 2.3 · Alias vs CNAME

Alias records: Route 53’s extension to DNS

Use an alias whenever the target is an AWS resource: it is free, works at the apex and follows IP changes.

Alias targets

  • CloudFront distribution, ELB (ALB, NLB, CLB), Global Accelerator.
  • API Gateway custom domain, App Runner, Elastic Beanstalk, AppSync.
  • VPC interface endpoint, OpenSearch Service custom domain.
  • S3 bucket configured as a static website.
  • Another record of the same type in the same hosted zone.

Behaviour

  • Answered as the record type you chose (A or AAAA): dig never shows "ALIAS".
  • Route 53 does not charge for alias queries to AWS resources; CNAME queries are charged.
  • TTL comes from the target resource (or the target record); you cannot set it.
  • Alias can "evaluate target health" so routing policies skip unhealthy targets.

CNAME still needed when…

  • The target is outside AWS or in another DNS provider.
  • You need a name → name mapping that is not an AWS resource.
  • A third party asks for a CNAME for verification.

Exam pattern: "point example.com (the apex) at an ALB" → alias A record. A CNAME at the apex is never a valid answer.

Task 1.2 · Public hosted zones

Public hosted zones and domain registration

Registering a domain and hosting its zone are separate things; Route 53 can do either or both.

Hosted zone

  • Container for the records of one domain (for example example.com).
  • Created with an NS and SOA record and four Route 53 name servers.
  • Initial quota: 500 hosted zones per account; 10,000 records per zone (more for a fee).
  • Reusable delegation sets keep the same four name servers across zones (white-label DNS).

Registration (Route 53 Domains)

  • Registrar function; the API lives in us-east-1.
  • Default quota for new accounts: 20 domains.
  • Registering a domain creates a matching public hosted zone.
  • Transfers in: unlock at the old registrar and supply the authorization code.
  • Turn on transfer lock and auto-renew for production domains.

Delegating a subdomain

  • Create a hosted zone dev.example.com (it can live in another account).
  • Copy its four NS values into an NS record dev in the parent zone.
  • Parent and child are independent: separate permissions, separate owners.
  • Verify with dig NS dev.example.com +trace.
Task 2.3 · Hands-on

Inspect and change DNS from the command line

dig shows what resolvers see; the Route 53 API shows what you configured. Check both.

# Follow the delegation chain from the root
dig +trace www.example.com

# Ask a Route 53 name server directly (bypasses caches)
dig @ns-123.awsdns-15.com www.example.com A +norecurse

# Remaining TTL from a recursive resolver
dig www.example.com +noall +answer

# Create or update a record (UPSERT is idempotent)
aws route53 change-resource-record-sets \
  --hosted-zone-id Z0123456789ABC --change-batch '{
  "Changes": [{"Action": "UPSERT", "ResourceRecordSet": {
    "Name": "app.example.com", "Type": "A",
    "AliasTarget": {
      "HostedZoneId": "Z14GRHDCWA56QT",
      "DNSName": "dualstack.my-alb-123.ap-northeast-1.elb.amazonaws.com",
      "EvaluateTargetHealth": true }}}]}'

aws route53 get-change --id C2682N5HXP0BZ4   # PENDING -> INSYNC

Notes

  • The alias HostedZoneId is the target’s zone ID (the ELB’s canonical zone for that Region), not your own zone.
  • get-change returns INSYNC once every Route 53 name server has the change.
  • A single change batch is atomic: all changes apply or none do.
  • Limits per request: 1,000 ResourceRecord elements and 32,000 characters; UPSERT counts twice.
  • Throttling returns HTTP 400 Throttling; a concurrent change to the same zone returns PriorRequestNotComplete.
Task 1.2 / 4.3 · DNSSEC

DNSSEC signing for public hosted zones

DNSSEC proves a response was not tampered with. It signs data; it does not encrypt queries.

How Route 53 signs

  • You enable signing on a public hosted zone.
  • Route 53 creates a key-signing key (KSK) from a customer managed KMS key.
  • Route 53 manages the zone-signing key (ZSK) and rotates it for you.
  • Up to 2 KSKs per hosted zone (for KSK rotation).

KMS key requirements

  • Must be in US East (N. Virginia).
  • Asymmetric customer managed key, key spec ECC_NIST_P256, sign/verify only.
  • Key policy must let the Route 53 DNSSEC service use the key.
  • Deleting or disabling the key breaks the zone: every validating resolver returns SERVFAIL.

Chain of trust

  • Add the DS record to the parent zone (at the registrar, or in the parent Route 53 zone).
  • Enable signing first, wait, then add DS; to disable, remove DS first and wait for its TTL.
  • Alarm on CloudWatch metrics DNSSECInternalFailure and DNSSECKeySigningKeysNeedingAction.
  • Validation side: turn on DNSSEC validation in VPC Resolver per VPC.
Exam traps

DNS fundamentals traps

Wrong answers that look right.

CNAME at the apex

  • Never valid. Use an alias A/AAAA record to the AWS resource.

TTL timing

  • Lowering TTL at cut-over time is too late: clients cached the old record for the old TTL.

Wrong zone ID

  • The alias HostedZoneId is the target service’s zone ID, not yours.

NS mismatch

  • A new hosted zone has new name servers. If the registrar still lists the old NS set, changes never show up.

DNSSEC ≠ privacy

  • DNSSEC signs answers; it does not encrypt queries. Encryption in transit is DoH/DoT.

DNSSEC key Region

  • The KMS key for signing must be in us-east-1 and asymmetric ECC_NIST_P256.
Chapter 5 · Part 5.2

Routing policies and health checks

Steer users with weights, latency, geography and health; fail over automatically.

Task 1.2 · Routing policies

Eight routing policies and when to use each

Each record set with the same name and type uses one policy; mix policies by nesting alias records.

PolicyDecides byTypical use
SimpleOne record, possibly several values (returned in random order)Single resource; no health checks on the record
WeightedRelative weight 0–255 per recordCanary / blue-green, gradual migration, A/B tests
LatencyLowest measured latency between the user and AWS RegionsMulti-Region active/active for performance
FailoverPrimary while healthy, otherwise secondaryActive/passive DR, static S3 "sorry" page
GeolocationContinent, country or US state of the userCompliance, localized content, licensing
GeoproximityDistance to the resource, adjusted by a bias (−99 to 99)Shift traffic between Regions by expanding or shrinking areas
Multivalue answerUp to eight healthy records, random orderSimple client-side load spreading with health checks
IP-basedClient source CIDR, from a CIDR collectionRoute by ISP or known network; cost or performance
Task 1.2 · Geography

Geolocation, geoproximity and IP-based routing in depth

All three use the resolver’s IP, or the client subnet when the resolver sends EDNS Client Subnet (ECS).

Geolocation

  • Most specific match wins: state → country → continent.
  • Always create a Default record, or users in unmapped places get NOERROR with no answer.
  • Does not move traffic for performance; it maps places to endpoints.
  • Up to 100 records with the same name and type.

Geoproximity

  • Location = AWS Region, Local Zone group, or latitude/longitude for non-AWS resources.
  • Biased distance = actual distance × (1 − bias/100).
  • Positive bias pulls a larger area to the resource; negative shrinks it.
  • Change bias in small steps: users near borders can move in bulk.
  • Up to 30 records with the same name and type.

IP-based

  • Create a CIDR collection with named locations (CIDR blocks).
  • Records refer to a location; a default location * catches the rest.
  • Quotas: 5 CIDR collections per account, 1,000 CIDR blocks each.
  • Use when you know the client networks better than Route 53’s geo data.
Task 3.3 · Health checks

Route 53 health checks

Health checks run from Route 53’s global checkers, outside your VPC. That decides what they can reach.

Endpoint checks

  • HTTP, HTTPS, TCP; optional string match in the first 5,120 bytes.
  • Interval 30 s (standard) or 10 s (fast).
  • Healthy if more than 18% of checkers report healthy.
  • HTTP(S): TCP connect within 4 s, then 2xx/3xx within 2 s.
  • HTTPS checks do not validate the certificate.

Calculated and alarm checks

  • Calculated: parent of up to 255 child checks; healthy if N children are healthy.
  • CloudWatch alarm check: follows the alarm’s data stream (OK = healthy).
  • Choose the status when data is insufficient: healthy, unhealthy, last known.
  • No cross-account CloudWatch alarms.

Private resources

  • Checkers are on the internet: they cannot reach private IPs.
  • For private endpoints: CloudWatch metric (for example from a Lambda probe or the ELB HealthyHostCount) → alarm → alarm-based health check.
  • Allow the Route 53 health checker IP ranges in SGs for public endpoints.
  • Quota: 200 active health checks per account (adjustable).

Alias records to ELB, CloudFront or another record can use "Evaluate target health" instead of a separate health check.

Task 3.3 · Failover design

Active/passive and active/active failover

Combine policies by pointing alias records at other records: build a tree.

app.example.comFailover aliasprimary.app.example.comLatency (2 Regions)S3 static siteFailover SECONDARYALB ap-northeast-1eval target healthALB ap-southeast-1eval target healthPRIMARYSECONDARYPrimary is healthy while at least one latencyrecord is healthy (evaluate target health).Both Regions down → users get the static page.

Active/active: all records healthy and answering (weighted, latency, multivalue). Active/passive: failover policy. If every record in a group is unhealthy, Route 53 treats them all as healthy and keeps answering.

Task 3.3 · Recovery

Traffic Flow and Application Recovery Controller

For large policy trees and for controlled, auditable failover.

Traffic Flow

  • Visual editor for routing trees; versioned traffic policies.
  • A policy record applies a policy to a name in a hosted zone.
  • Quotas: 50 traffic policies per account; 5 policy records (adjustable).
  • Policy records have a monthly fee each.

Application Recovery Controller

  • Routing controls: on/off switches exposed as health checks; flipped through a cluster of five redundant Regional endpoints (retry each in turn).
  • Use for deliberate Regional failover that does not depend on the failing Region.
  • Zonal shift: move traffic away from one AZ for ALB/NLB and other supported resources.
  • Zonal autoshift: AWS shifts traffic for you during an AZ impairment.

Design tips

  • Health-check the dependency, not only the web tier (a calculated check over app + DB).
  • Fail over on an alarm you trust; avoid flapping with sensible thresholds.
  • Test failover regularly (Task 4.1: failover tests).
  • Keep DNS TTLs short on records that fail over.
Exam-style scenarios

Routing policy scenarios

Scenario → answer → why.

ScenarioAnswerWhy
Send 5% of users to the new stack, then increaseWeighted records (95/5), health checks on bothWeights shift share without changing clients
EU users must be served only from eu-central-1 for complianceGeolocation (Europe → EU stack) + Default recordLatency could pick a non-EU Region
Users should hit the fastest of three RegionsLatency-based alias records with evaluate target healthLatency is measured, not distance
Move more traffic to Tokyo without moving users by countryGeoproximity with a positive bias for TokyoBias resizes the catchment area
Health check an internal ALB behind a private hosted zone recordCloudWatch alarm on HealthyHostCount + alarm-based health checkCheckers cannot reach private IPs
Return several healthy EC2 IPs without a load balancerMultivalue answer with a health check per recordUp to eight healthy values, random order
Route a partner’s ISP ranges to a dedicated endpointIP-based routing with a CIDR collectionYou know their CIDRs exactly
Chapter 5 · Part 5.3

Private DNS and hybrid resolution

Private hosted zones, VPC Resolver endpoints, forwarding rules, Profiles and a central DNS design.

Task 2.3 · VPC Resolver

Route 53 VPC Resolver: the DNS server in every VPC

Renamed from "Route 53 Resolver" in March 2026; exam questions and older docs use the old name.

Where it listens

  • VPC CIDR base + 2 (for example 10.0.0.2).
  • 169.254.169.253 and fd00:ec2::253 from any subnet.
  • Needs enableDnsSupport = true; hostnames need enableDnsHostnames.
  • Limit: 1,024 packets per second per ENI for link-local services, not adjustable.

What it answers, in order

  • Forwarding rules (most specific domain wins).
  • Private hosted zones associated with the VPC.
  • VPC-internal names (ip-10-0-1-5.ec2.internal, interface endpoint names).
  • Public DNS via recursive resolution.

What it cannot do alone

  • Answer queries from on-premises → needs an inbound endpoint.
  • Resolve on-premises names → needs an outbound endpoint + forwarding rule.
  • Be reached across peering or TGW at .2 from another VPC (it serves its own VPC).
Task 2.3 · Private hosted zones

Private hosted zones

Authoritative only for the VPCs you associate; invisible from the internet.

Basics

  • Associate one or more VPCs (any Region, any account).
  • Up to 300 VPC associations per private hosted zone; use Route 53 Profiles beyond that.
  • Split-horizon: a public and a private zone with the same name; VPCs see the private one.
  • If the private zone has no matching record, the query is not passed to the public zone: NXDOMAIN.

Cross-account association

  • Zone account: create-vpc-association-authorization.
  • VPC account: associate-vpc-with-hosted-zone.
  • Zone account: delete-vpc-association-authorization (cleanup).
  • CLI, SDK or API only (not the console). Up to 1,000 authorizations.

Overlapping namespaces

  • Zones example.com and dev.example.com both associated: the most specific zone answers names under dev.
  • A forwarding rule and a private zone for the same name: the rule wins.
  • Check with dig from an instance in the VPC, not from your laptop.
aws route53 create-vpc-association-authorization --hosted-zone-id Z0PHZ123 \
  --vpc VPCRegion=ap-northeast-1,VPCId=vpc-0spoke          # in the zone owner account
aws route53 associate-vpc-with-hosted-zone --hosted-zone-id Z0PHZ123 \
  --vpc VPCRegion=ap-northeast-1,VPCId=vpc-0spoke          # in the VPC owner account
Task 1.2 / 2.3 · Hybrid DNS

Hybrid resolution with inbound and outbound endpoints

Inbound: on-premises asks AWS. Outbound: AWS asks on-premises. Both ride your VPN or Direct Connect.

On-premisesCorporate DNS (BIND / AD)corp.example.comLaptop / serverasks corporate DNSAWS VPC 10.0.0.0/16Inbound endpoint2+ ENIs, 2 AZsOutbound endpoint2+ ENIs, 2 AZsVPC Resolver10.0.0.2 + rulesEC2 / EKS podasks 10.0.0.2Private hosted zoneaws.example.comforward aws.example.comrule: corp.example.comVPN or DX

Blue: on-premises → inbound endpoint → private hosted zone. Green: workload → VPC Resolver → forwarding rule → outbound endpoint → corporate DNS. Endpoint ENIs need SG rules for TCP and UDP 53 (and 443 for DoH).

Task 2.3 · Endpoints

Resolver endpoints in detail

Endpoints are ENIs in your subnets; they are billed per ENI hour and per query.

Inbound endpoint

  • Category Default: on-premises resolvers conditionally forward to its IPs.
  • Category Delegation: on-premises delegates a subdomain with NS (glue) records pointing at its IPs (Do53 only).
  • Protocols: Do53, DoH, DoH-FIPS (inbound only), or Do53 combined with one of them.
  • Optional DNS64 for IPv6-only clients.

Outbound endpoint

  • Used by forwarding rules to reach target resolvers.
  • Protocols: Do53 or DoH towards the target.
  • Route from the endpoint subnets to on-premises DNS must exist (VGW/TGW/DX).
  • SG must allow outbound 53 TCP/UDP to the targets.

Sizing and quotas

  • At least 2 IPs; spread across 2+ AZs. Up to 6 IPs per endpoint (adjustable).
  • Up to 10,000 UDP queries per second per IP; as low as 1,500 when connections are tracked.
  • Avoid tracking: allow all traffic in SG rules both ways, or remove restrictive rules.
  • Scale out when an ENI passes ~50% of capacity (CloudWatch per-IP metrics).
  • 4 endpoints per Region per account by default.
Task 2.3 · Forwarding rules

Forwarding rules and system rules

Rules tell VPC Resolver which domains to send elsewhere. They apply only to VPCs they are associated with.

# Forward corp.example.com to two on-premises DNS servers
aws route53resolver create-resolver-rule \
  --name corp-forward --rule-type FORWARD \
  --domain-name corp.example.com \
  --resolver-endpoint-id rslvr-out-0abc \
  --target-ips "Ip=172.16.0.10,Port=53" "Ip=172.16.1.10,Port=53" \
  --creator-request-id corp-2026-10

# Carve out one subdomain that AWS should resolve itself
aws route53resolver create-resolver-rule --name aws-local \
  --rule-type SYSTEM --domain-name aws.corp.example.com \
  --creator-request-id aws-local-2026-10

aws route53resolver associate-resolver-rule \
  --resolver-rule-id rslvr-rr-0123 --vpc-id vpc-0spoke

# Share the rule with the organization via RAM
RULE=arn:aws:route53resolver:ap-northeast-1:1111…:resolver-rule/rslvr-rr-0123
ORG=arn:aws:organizations::1111…:organization/o-abc123
aws ram create-resource-share --name dns-rules \
  --resource-arns $RULE --principals $ORG

Rules of thumb

  • FORWARD: send matching queries out through an outbound endpoint.
  • SYSTEM: resolve locally (override a broader forward rule for a subdomain).
  • Most specific domain name wins across all rules for that VPC.
  • Up to 6 target IPs per rule; 1,000 rules per Region per account.
  • Shared rules: the owner keeps the outbound endpoint; consumers only associate the rule.
  • DELEGATION (new): follow NS delegations to on-premises name servers.
  • Built-in: autodefined system rules (AWS names, reverse zones) and the recursive "Internet Resolver" rule.
Task 2.3 · Centralized DNS

Centralized hybrid DNS in a shared-services VPC

One pair of endpoints for the whole organization; spokes reach on-premises names through shared rules.

Network / shared-services accountInbound EPshared VPCOutbound EPshared VPCForwarding rules → RAM sharecorp.example.com, ad.example.comPrivate hosted zonesassociated with every spoke VPCTransit Gatewayor Cloud WANWorkload accountsSpoke VPC Aassociates shared rulesSpoke VPC Bassociates shared rulesSpoke VPC Cor a Route 53 ProfileSpokes resolve locally at their own .2;shared rules send corp names to theoutbound endpoint in the shared VPC.

Spokes need no endpoints of their own: queries that match a shared rule leave through the owner’s outbound endpoint. On-premises DNS forwards AWS zones to the central inbound endpoint IPs, so only the shared VPC needs a DNS path to on-premises.

Task 2.3 · Route 53 Profiles

Route 53 Profiles: DNS configuration as one shareable unit

Launched in 2024. Bundle DNS settings once, share with RAM, associate with up to 1,000 VPCs per Profile.

What a Profile carries

  • Private hosted zones (up to 5,000 per Profile).
  • Resolver rules, forwarding and system (up to 1,000).
  • DNS Firewall rule groups (up to 5).
  • Interface VPC endpoints and up to 2 query logging configurations.
  • VPC settings: reverse DNS for rules, DNS Firewall failure mode, DNSSEC validation.

Precedence

  • One Profile per VPC.
  • Same domain in the VPC’s own settings and in the Profile → the local VPC setting wins.
  • Different specificity → the most specific domain wins, wherever it comes from.
  • Use local settings for migration and testing.

Why use it

  • Replaces hundreds of per-VPC zone associations and rule associations.
  • New VPC: one association instead of many.
  • Profiles per account per Region: 5 by default.
  • Pairs well with CloudFormation / Terraform for every new VPC.
Exam-style scenarios

Hybrid and private DNS scenarios

Scenario → answer → why.

ScenarioAnswerWhy
On-premises servers must resolve records in a private hosted zoneInbound endpoint; conditional forwarder on-premises to its IPsThe zone is only answered inside associated VPCs
EC2 must resolve corp.example.com hosted in Active DirectoryOutbound endpoint + FORWARD rule to the AD DNS IPsVPC Resolver cannot reach on-premises by itself
50 accounts need the same on-premises forwarding with least effortCentral outbound endpoint; share rules via RAM (or a Profile)One endpoint pair, consumers only associate
Forward all of corp.example.com except aws.corp.example.comFORWARD rule for the parent + SYSTEM rule for the subdomainMost specific rule wins
Associate a private zone in account A with a VPC in account BAuthorization in A, association in B (CLI/API)The console cannot do cross-account associations
Inbound endpoint drops queries at peakAdd IPs/ENIs; avoid connection tracking in SG rulesTracked flows reduce per-IP capacity
Chapter 5 · Part 5.4

DNS security and logging

Query logs, DNS Firewall, encrypted DNS transport and Route 53 Global Resolver.

Task 2.3 / 4.2 · Query logging

Two kinds of DNS query logs

Public logs show who asked your authoritative zone; Resolver logs show what your workloads asked.

Public DNS query logging

  • Per public hosted zone; one configuration per zone.
  • Destination: CloudWatch Logs log group in us-east-1 only.
  • Logs only queries that reach Route 53 (cache hits at resolvers are invisible).
  • Fields: name, type, response code, edge location, resolver IP (and ECS subnet).

VPC Resolver query logging

  • Logs queries from VPCs: workloads, inbound endpoint traffic.
  • Destinations: CloudWatch Logs, S3 or Data Firehose.
  • Share configurations with RAM; 20 configurations and 100 VPC associations per Region (adjustable).
  • Includes DNS Firewall actions (firewall_rule_action, firewall_domain_list_id).

Use the logs for

  • Find malware beaconing and DNS tunneling.
  • Prove which workload resolved which name (incident response).
  • Debug hybrid resolution: did the query use the rule you expect?
  • Feed Security Lake (OCSF) or a SIEM; query with Logs Insights or Athena.
Task 4.1 · DNS Firewall

Resolver DNS Firewall

Filters outbound DNS queries from your VPCs. It acts on names, not on IP traffic.

Building blocks

  • Domain lists: your own, or AWS managed lists (malware, botnet command and control, aggregate threat lists).
  • Rules in a rule group, evaluated by priority.
  • Rule groups associated with VPCs (up to 5 per VPC) or shared with RAM / Firewall Manager.
  • Quota: 100 rules per rule group; 100,000 domains across all lists (adjustable).

Actions

  • Allow: stop inspecting, let it through.
  • Alert: let it through and log — use it to test a block rule first.
  • Block with a response: NODATA, NXDOMAIN, or OVERRIDE with a custom CNAME (for example a walled-garden page).
  • Advanced rules: DGA and DNS tunneling detection, threat and content categories, partner (Palo Alto Networks) lists.

Failure mode

  • Per VPC setting.
  • Fail closed (default): if DNS Firewall cannot evaluate, queries are blocked — security first.
  • Fail open: queries are allowed if evaluation fails — availability first.
  • Pick by the workload: a payment system and a dev VPC answer this differently.

DNS Firewall does not stop a client that uses an IP address directly or another DNS server. Combine it with Network Firewall egress rules and blocking outbound port 53/853 to the internet.

Task 4.3 · Encrypted DNS

Securing DNS in transit and at the edge

Two separate problems: integrity of answers (DNSSEC) and privacy of queries (DoH / DoT).

NeedUseNotes
Prove answers from my public zone are authenticDNSSEC signing on the public hosted zoneKSK from a us-east-1 KMS key; DS record at the parent
Reject forged answers for workloadsDNSSEC validation in VPC ResolverPer VPC, or set by a Route 53 Profile
Encrypt hybrid DNS between on-premises and AWSDoH on inbound and outbound Resolver endpointsAllow TCP 443 in the endpoint SG; DoH-FIPS for inbound only
Encrypt everything at the link layerIPsec VPN or MACsec on Direct ConnectCovers all traffic, not just DNS
Stop DNS bypass from workloadsBlock outbound 53/853 except to VPC Resolver; DNS FirewallClients with hard-coded resolvers otherwise skip controls
Task 1.2 · Global Resolver

Route 53 Global Resolver (GA March 2026)

An internet-reachable anycast resolver for your organization’s clients outside any VPC.

What it is

  • Anycast DNS resolver run by Route 53; preview at re:Invent 2025, GA 9 Mar 2026.
  • Resolves public domains and private hosted zones you attach, for authorized clients.
  • Serves remote users, branch offices and devices without VPN to a VPC.
  • IPv4 and IPv6; available in 30 Regions at GA.

Access and protocols

  • Do53, DoH and DoT.
  • Access sources: allow or deny by client IP / CIDR (all protocols).
  • Access tokens: encrypted authentication for DoH and DoT clients, with expiry.
  • Centralized query logging.

Filtering

  • Block malicious domains and content categories.
  • Detects DNS tunneling and domain generation algorithms (DGA), including dictionary DGA at GA.
  • Compare with DNS Firewall: that filters queries from VPCs; Global Resolver filters clients anywhere.
  • Not on the ANS-C01 exam guide; know it for real designs.
Exam traps

Private, hybrid and secure DNS traps

Wrong answers that look right.

Peering is not DNS

  • Peering or TGW does not make a private hosted zone visible. Associate the zone with the VPC.

Endpoint direction

  • On-premises → AWS needs inbound; AWS → on-premises needs outbound. The names follow the query, not the answer.

Health checks and private IPs

  • Route 53 checkers cannot probe private IPs; use alarm-based checks.

Forwarding to .2

  • On-premises or custom DNS servers must not forward to the VPC CIDR + 2 address; AWS does not support it. Use an inbound endpoint.

Custom DNS servers

  • If a DHCP options set points EC2 at your own DNS servers, private zones and Resolver rules are bypassed unless those servers forward to .2.

DNS Firewall scope

  • It filters names asked through VPC Resolver only; IP-based egress needs Network Firewall.
Self-check

DNS with Route 53: can you answer these?

Explain the answer and name the exact feature.

Public DNS

  • Why can’t you put a CNAME at the zone apex, and what do you use instead?
  • Which hosted zone ID goes into an alias to an ALB?
  • What is the order of steps to enable DNSSEC signing safely?
  • When does geolocation return no answer?
  • How does a positive geoproximity bias change routing?

Private and hybrid

  • What three addresses does VPC Resolver answer on?
  • Inbound vs outbound endpoint: which direction does each serve?
  • How do you associate a private zone with a VPC in another account?
  • FORWARD vs SYSTEM rule: when do you need both?
  • What wins: a Profile setting or the VPC’s own for the same domain?

Health and security

  • How does Route 53 decide an endpoint is healthy?
  • How do you health-check a private resource?
  • Fail open vs fail closed in DNS Firewall: who chooses which?
  • Where can public query logs go, and Resolver query logs?
  • DNSSEC vs DoH: which problem does each solve?
Chapter 6

Load balancing and edge

Tasks 1.1, 1.3, 2.1, 3.3 — ALB, NLB, GWLB, the Load Balancer Controller, CloudFront, Global Accelerator and API Gateway.

Chapter 6 · Part 6.1

Elastic Load Balancing

ALB, NLB and GWLB: layers, target types, attributes, TLS and Auto Scaling.

Task 1.3 · Pick the load balancer

ALB vs NLB vs GWLB

Choose by the OSI layer you need to act on, then check the features the requirement names.

Application LBNetwork LBGateway LB
Layer7 (HTTP/HTTPS, HTTP/2, gRPC, WebSocket)4 (TCP, UDP, TLS, TCP_UDP, QUIC)3 (all IP packets, all ports)
RoutingHost, path, header, method, query, source IP rulesListener port → target group, flow hashAll traffic → appliance target group (GENEVE 6081)
TargetsInstance, IP, LambdaInstance, IP, ALBInstance, IP (virtual appliances)
Static IPNo (use NLB in front, or Global Accelerator)One per AZ; Elastic IPs for internet-facingReached through GWLB endpoints
Client IP to targetX-Forwarded-For headerPreserved (defaults vary), or proxy protocol v2Original packet inside GENEVE
SecuritySGs, AWS WAF, TLS, mTLS, OIDC/Cognito authSGs (only if attached at creation), TLS terminationInspection by the appliances
Typical useWeb apps, microservices, IngressExtreme throughput, non-HTTP, PrivateLink servicesThird-party firewalls, IDS/IPS
Task 1.3 · Packet paths

What the target sees with each load balancer

ALB is a proxy (two connections); NLB forwards flows; GWLB wraps packets for an appliance and returns them.

ALB — terminates the connection, opens a new oneClient203.0.113.7ALB nodeper AZTargetsrc = ALB IPTLS 1HTTP(S) 2 + XFFNLB — passes the flow through (client IP preserved for instance targets)Client203.0.113.7NLB nodestatic IP / AZTargetsrc = 203.0.113.7TCPsame flowGWLB — bump in the wire: route table → GWLB endpoint → appliance → backClient / IGWingress routeGWLB endpointnext hopGWLBGENEVE 6081Applianceinspects, returnsApp subnetoriginal packetTwo TCP connections;ALB idle timeout default 60 s.One flow; TCP idle timeout350 s (60–6000 s).

GWLB adds 68 bytes of GENEVE header and supports packets up to 8,500 bytes, so appliances need an MTU of at least 8,568. GWLB does not fragment and does not send ICMP "fragmentation needed": Path MTU Discovery does not work through it.

Task 1.3 · Attributes

Target group attributes that change behaviour

Most "the load balancer is wrong" tickets are one of these settings.

AttributeALBNLB
Cross-zoneAlways on at the LB; can be turned off per target groupOff by default; turning it on adds inter-AZ data charges
AlgorithmRound robin (default), least outstanding requests, weighted random (+ anomaly mitigation)Flow hash on protocol, IPs, ports (and TCP sequence)
StickinessDuration cookie AWSALB or app cookie (AWSALBAPP-*), 1 s – 7 daysSource IP; not for TLS or QUIC listeners
Deregistration delay300 s default; target drains, then unused300 s default; optional connection termination
Slow start30–900 s ramp; only with round robinNot available
Client IPX-Forwarded-For, X-Forwarded-Proto, X-Forwarded-PortInstance targets: preserved; IP targets with TCP/TLS: off by default
Proxy protocolNov2 header; carries the VPC endpoint ID for PrivateLink consumers
Task 1.3 · NLB client IP

Client IP preservation and proxy protocol on NLB

The single most tested NLB detail. Know the defaults and the cases where preservation cannot work.

Defaults

  • Instance targets: preserved.
  • IP targets with UDP, TCP_UDP, QUIC: preserved and cannot be turned off.
  • IP targets with TCP or TLS: not preserved by default; set preserve_client_ip.enabled.
  • Changes apply to new TCP connections only.

Where it does not work

  • Targets reached through a Transit Gateway.
  • A GWLB endpoint between the NLB and the target.
  • Traffic arriving through PrivateLink: source is always the NLB’s private IP.
  • IPv6 → IPv4 translation.
  • Hairpin: a target calling its own internal NLB can connect to itself; disable preservation.

Then use proxy protocol v2

  • NLB prepends a binary header with the original source and destination.
  • PrivateLink: the header also carries the consumer’s VPC endpoint ID (TLV type 0xEA).
  • Health checks carry the header too: the target must parse it or return 400.
  • Without preservation: about 55,000 simultaneous connections per unique target IP and port.
aws elbv2 modify-target-group-attributes --target-group-arn $TG \
  --attributes Key=preserve_client_ip.enabled,Value=false Key=proxy_protocol_v2.enabled,Value=true
Task 1.3 / 4.3 · TLS

TLS termination, passthrough and mutual TLS

Decide where TLS ends: at the load balancer (inspect, route, offload) or at the target (end-to-end).

Termination

  • ALB HTTPS listener or NLB TLS listener with an ACM certificate.
  • SNI: several certificates on one listener; the default certificate covers clients without SNI.
  • Security policy picks protocols and ciphers (for example ELBSecurityPolicy-TLS13-1-2-2021-06).
  • Re-encrypt to targets with an HTTPS / TLS target group if the requirement says "encrypted end to end".

Passthrough

  • NLB TCP listener on 443: the encrypted stream goes straight to the target.
  • Target holds the certificate and private key.
  • No header inspection, no WAF, no cookie stickiness.
  • Use when the private key must never leave the instances.

Mutual TLS on ALB

  • Verify mode: ALB checks the client certificate against a trust store (CA bundle, optional CRLs).
  • Passthrough mode: ALB forwards the chain in X-Amzn-Mtls-Clientcert; the target verifies.
  • Verify mode sends X-Amzn-Mtls-Clientcert-Subject, -Issuer, -Serial-Number, -Validity, -Leaf.
  • Client CAs can be AWS Private CA or a third-party CA.
Task 1.3 · ALB features

ALB listener rules, authentication and health

An ALB is a programmable reverse proxy; many "add a proxy fleet" answers are wrong because the ALB already does it.

Rule conditions and actions

  • Conditions: host header, path, HTTP header, method, query string, source IP.
  • Actions: forward (weighted across target groups), redirect, fixed response.
  • Authenticate: OIDC provider or Amazon Cognito user pool before forwarding.
  • Weighted forward = blue/green or canary without DNS changes.

Health and resilience

  • Health check per target group: protocol, path, matcher codes, thresholds.
  • Fails open: with no healthy targets the ALB sends to all targets.
  • Target group health thresholds can trigger DNS failover of an AZ.
  • Anomaly mitigation (weighted random) moves traffic off a target with high error rates.

Auto Scaling integration

  • Attach the target group to the Auto Scaling group: instances register automatically.
  • Use ELB health checks in the ASG so failed targets are replaced.
  • Target tracking on ALBRequestCountPerTarget for request-driven scaling.
  • Deregistration delay + lifecycle hooks for clean scale-in.
  • Subnets in every AZ the ASG uses, with spare IPs for scaling.
Task 1.3 · Scaling

How load balancers scale

ELB scales automatically, but the shape of the traffic still matters.

Load balancer capacity units (LCU)

  • Billing and scaling unit: new connections, active connections, processed bytes, rule evaluations (ALB).
  • LCU reservations: pre-provision capacity for a known event (launch, sale).
  • Without a reservation, a sudden 10× spike may see errors while the ALB scales.

Subnets and IPs

  • ALB/NLB nodes are ENIs in your subnets. ALB subnets need at least a /27 and 8 free IPs; the ALB may use more as it scales.
  • GWLB subnets need at least 8 free IPs.
  • Enable every AZ that has targets; an empty AZ with cross-zone off returns 503.

Connection limits

  • NLB without client IP preservation: ~55,000 connections per unique target IP:port.
  • Long-lived connections (WebSocket, gRPC) do not rebalance when you add targets.
  • NLB / GWLB TCP idle timeout must not exceed the target ENI’s connection-tracking timeout (350 s on Nitro v6).
Exam traps

Load balancer traps

Wrong answers that look right.

Static IP for ALB

  • ALB has no static IP. Put an NLB (ALB-type target) or Global Accelerator in front.

NLB security groups

  • SGs on an NLB can only be added if it was created with at least one SG.

Client IP behind PrivateLink

  • Always the NLB’s IP. Use proxy protocol v2 if the provider needs it.

Cross-zone surprises

  • NLB cross-zone is off by default: uneven targets per AZ → uneven load.
  • Turning it on adds inter-AZ charges.

Stickiness conflicts

  • ALB stickiness needs cross-zone on; it does not work with weighted random.

GWLB and PMTUD

  • No fragmentation, no ICMP needed-frag: oversized packets just drop. Fix MTU on the appliance and clients.
Chapter 6 · Part 6.2

Load balancing for Kubernetes

The AWS Load Balancer Controller: Ingress to ALB, Service to NLB, IP mode and TargetGroupBinding.

Task 1.3 · Controller model

AWS Load Balancer Controller: how it works

A controller in the cluster watches Kubernetes objects and calls the ELB API. It needs IAM (IRSA or Pod Identity).

EKS clusterIngressingressClassName: albServicetype: LoadBalancerTargetGroupBindingexisting TGLB ControllerIRSA / Pod IdentityPod10.0.3.21Pod10.0.4.18ALBrules from Ingress paths and hostsNLBfrom Service annotationsTarget groupsIP mode → pod IPs directlyELB API callstraffic to pod IPs

Subnet discovery uses tags: kubernetes.io/role/elb = 1 for public subnets and kubernetes.io/role/internal-elb = 1 for private subnets (or set the subnets annotation). EKS Auto Mode has built-in load balancing: IngressClass controller: eks.amazonaws.com/alb with IngressClassParams, Services loadBalancerClass: eks.amazonaws.com/nlb (the cluster default); the same subnet tags apply.

Task 1.3 · Ingress → ALB

An Ingress that becomes an internet-facing ALB

Annotations carry the ALB settings; group.name lets many Ingresses share one ALB.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: shop
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing  # default internal
    alb.ingress.kubernetes.io/target-type: ip  # default instance
    alb.ingress.kubernetes.io/group.name: public-web
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS": 443}]'
    alb.ingress.kubernetes.io/ssl-redirect: '443'
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:...:certificate/abcd
    alb.ingress.kubernetes.io/healthcheck-path: /healthz
    alb.ingress.kubernetes.io/wafv2-acl-arn: arn:aws:wafv2:...:regional/webacl/...
spec:
  ingressClassName: alb
  rules:
  - host: shop.example.com
    http:
      paths:
      - path: /api
        pathType: Prefix
        backend: { service: { name: api, port: { number: 8080 } } }

What to remember

  • IP mode: ALB targets are pod IPs (VPC CNI); no NodePort hop, works with Fargate.
  • Instance mode: targets are nodes on a NodePort; extra hop, kube-proxy load balances again.
  • IngressGroup (group.name, group.order) merges rules from several Ingresses into one ALB — fewer ALBs, shared cost.
  • Pod readiness gates (label the namespace elbv2.k8s.aws/pod-readiness-gate-inject=enabled) stop rollouts from outrunning ALB registration.
  • Deleting the Ingress deletes the ALB: protect production with deletion_protection.enabled=true in load-balancer-attributes.
Task 1.3 · Service → NLB

A Service that becomes an NLB, and TargetGroupBinding

Use an NLB for TCP/UDP or when you need static IPs; bind to load balancers that Terraform owns.

apiVersion: v1
kind: Service
metadata:
  name: mqtt
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: external
    service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
    service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
spec:
  type: LoadBalancer
  ports: [{ port: 8883, targetPort: 8883, protocol: TCP }]
  selector: { app: mqtt }
---
apiVersion: elbv2.k8s.aws/v1beta1
kind: TargetGroupBinding
metadata: { name: api-tgb }
spec:
  serviceRef: { name: api, port: 8080 }
  targetGroupARN: arn:aws:elasticloadbalancing:...:targetgroup/api/0123
  targetType: ip

Notes

  • aws-load-balancer-type: external hands the Service to the AWS Load Balancer Controller instead of the legacy in-tree controller.
  • IP targets with TCP: client IP preservation is off by default — enable it or use proxy protocol v2.
  • TargetGroupBinding: the ALB/NLB and target group are created outside Kubernetes (Terraform, another team); the controller only keeps pod IPs registered.
  • Good for blue/green across clusters: two clusters register into weighted target groups behind one ALB.
Chapter 6 · Part 6.3

CloudFront and Global Accelerator

Bring users onto the AWS global network: cache at the edge, or accelerate any TCP/UDP flow.

Task 1.1 · Edge services

CloudFront vs Global Accelerator

Both use AWS edge locations. CloudFront is an HTTP cache and proxy; Global Accelerator is an anycast network entry point.

Amazon CloudFrontAWS Global Accelerator
ProtocolsHTTP/HTTPS (HTTP/1.1, 2, 3), WebSocketAny TCP or UDP
CachingYes: cache behaviors, policies, Origin ShieldNo
AddressingDNS name (d111.cloudfront.net); IPs change2 static anycast IPv4 (4 IPs with dual stack); BYOIP
FailoverOrigin groups (GET/HEAD/OPTIONS)Health-checked endpoint groups; fast failover without DNS TTL
Endpoints / originsS3, ALB, NLB, EC2, API Gateway, any HTTP server, VPC originsALB, NLB, EC2, Elastic IP (standard); VPC subnets (custom routing)
SecurityAWS WAF, Shield, OAC, signed URLs/cookies, geo restrictionShield Standard; WAF on the ALB endpoint
Pick whenCacheable or dynamic web content, APIsGaming, VoIP, IoT, MQTT, static IP allow-lists, multi-Region failover
Task 1.1 · CloudFront

CloudFront architecture and origin protection

Users hit the nearest edge location; misses go via regional edge caches (and optional Origin Shield) to the origin.

UsersworldwideEdge locationFunctions run hereRegional edge cacheLambda@Edge hereOrigin Shieldoptional layerOriginsS3 bucketOAC (SigV4)ALB / NLB / EC2VPC origin, privateCustom originHTTPS + secret headermissCache key = cache policy (path + chosen headers, cookies, query strings).Origin request policy adds values sent to the origin without changing the cache key.Viewer certificate from ACM must be in us-east-1.

VPC origins: CloudFront reaches an ALB, NLB (with an SG, no TLS listener) or EC2 in a private subnet through a service-managed ENI. Allow the CloudFront-VPCOrigins-Service-SG group or the com.amazonaws.global.cloudfront.origin-facing prefix list in the origin SG. Not supported: GWLB origins, gRPC, Lambda@Edge origin request/response triggers; inbound NACL rules are not evaluated, outbound NACLs must allow ephemeral ports.

Task 1.1 · CloudFront features

CloudFront features the exam expects you to place

Map each requirement to the feature and where it runs.

Compute at the edge

  • CloudFront Functions: JavaScript, viewer request/response only, sub-millisecond, no network calls — header rewrites, redirects, URL normalization.
  • Lambda@Edge: Node.js/Python, all four events (viewer + origin request/response), network access — auth, origin selection, larger logic.

Access control

  • Origin access control (OAC) for S3; legacy OAI cannot use SSE-KMS.
  • Signed URLs (one object) and signed cookies (many objects) with trusted key groups.
  • Geo restriction (whole distribution) vs WAF geo match (per path).
  • Field-level encryption for selected POST fields.

Performance and resilience

  • Origin Shield: one extra cache layer per Region to cut origin load.
  • Origin groups: fail over to a secondary origin on chosen status codes.
  • HTTP/3 (QUIC) to viewers; compression; long TTLs for static assets.
  • Continuous deployment: test config on a staging distribution first.
Task 1.1 / 3.3 · Global Accelerator

Global Accelerator in depth

Two anycast IPs announced from every edge location; TCP is terminated at the edge and carried over the AWS backbone.

Building blocks

  • Accelerator → listeners (ports, protocol) → endpoint groups (one per Region) → endpoints.
  • Traffic dial per endpoint group: % of the traffic already directed to that Region (0–100).
  • Weight per endpoint: 0–255, default 128.
  • Client affinity: none (5-tuple) or source IP (2-tuple) for stateful apps.

Behaviour

  • Health checks for standard accelerators; new connections move to healthy endpoints at once — no DNS caching delay.
  • Idle timeout: 340 s TCP, 30 s UDP (not configurable; TCP keep-alives do not count).
  • Client IP preserved for supported endpoint types; always for custom routing.
  • TCP fragments are dropped at the edge; allow ICMP on endpoints for PMTUD.

Use cases

  • Static IPs that partners allow-list, in front of ALBs that have no static IP.
  • Multi-Region active/active or active/passive for TCP/UDP apps.
  • Blue/green between Regions with the traffic dial.
  • Custom routing accelerator: map users to a specific EC2 instance and port (game sessions, VoIP), IPv4 only.
Exam-style scenarios

Edge and load balancing scenarios

Scenario → answer → why.

ScenarioAnswerWhy
Partners need two fixed IPs to allow-list for an ALB-based app in two RegionsGlobal Accelerator with ALB endpointsStatic anycast IPs, health-based Regional failover
UDP game servers need lower jitter for players worldwideGlobal Accelerator (standard or custom routing)CloudFront does not carry UDP
Serve a private ALB through CloudFront with no public IPsCloudFront VPC originPrivate subnet, service-managed ENI
App needs the client IP; NLB with IP targets over TCPEnable client IP preservation (same VPC) or proxy protocol v2Off by default for IP/TCP targets
Inspect all inbound traffic with a third-party firewall fleet that scalesGWLB + GWLB endpoints in ingress routingTransparent bump in the wire
TLS private keys must stay on the instancesNLB TCP listener (passthrough)ALB and NLB TLS listeners terminate TLS
Blue/green on EKS with a gradual shiftALB weighted forward to two target groups (TargetGroupBinding)No DNS TTL delay
Chapter 6 · Part 6.4

API Gateway networking

Endpoint types, private APIs through interface endpoints, and VPC links to private back ends.

Task 1.1 · Endpoint types

API Gateway endpoint types

For REST APIs you choose edge-optimized, Regional or private. HTTP APIs and WebSocket APIs are Regional.

Edge-optimized (REST default)

  • Requests enter at the nearest CloudFront PoP, managed by API Gateway.
  • Custom domain applies across Regions; certificate from ACM in us-east-1.
  • Header names are capitalized by CloudFront.
  • You cannot attach your own WAF/cache settings to that hidden distribution.

Regional

  • Endpoint in one Region; certificate in that Region.
  • Put your own CloudFront distribution in front when you need edge caching, WAF on CloudFront or custom behaviors.
  • Same custom domain in several Regions + Route 53 latency or failover records = multi-Region API.

Private

  • Reachable only through an execute-api interface VPC endpoint.
  • A resource policy is required: allow aws:SourceVpce or aws:SourceVpc.
  • Custom domain names for private APIs are supported (domain name access associations).
  • From on-premises: over DX/VPN, with a Resolver inbound endpoint for the private DNS names.
Task 2.1 · Private APIs

Calling a private REST API from a VPC

The interface endpoint and its private DNS setting decide which URL works.

# A) Private DNS enabled on the endpoint
curl https://01234567ab.execute-api.\
ap-northeast-1.amazonaws.com/prod/pets

# B) Private DNS disabled: endpoint name + header
curl https://vpce-0abc-def.execute-api.\
ap-northeast-1.vpce.amazonaws.com/prod/pets \
  -H 'x-apigw-api-id: 01234567ab'

# C) Endpoint associated with the API:
#    Route 53 alias form {api-id}-{vpce-id}
curl https://01234567ab-vpce-0abc.execute-api.\
ap-northeast-1.amazonaws.com/prod/pets
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Deny", "Principal": "*",
    "Action": "execute-api:Invoke",
    "Resource": "execute-api:/*",
    "Condition": { "StringNotEquals": {
      "aws:SourceVpce": "vpce-0abc" } }
  }, {
    "Effect": "Allow", "Principal": "*",
    "Action": "execute-api:Invoke",
    "Resource": "execute-api:/*"
  }]
}

Trap: with private DNS enabled on the execute-api endpoint, that VPC can no longer reach the default endpoints of public APIs (all *.execute-api names resolve to the endpoint). Use custom domains for public APIs, a separate VPC without private DNS, or disable private DNS and create a private hosted zone per private API.

Task 2.1 · VPC links

Private integrations: from API Gateway into your VPC

The API stays public (or private); the back end stays in private subnets.

API typeVPC linkBack-end targetsNotes
REST APIVPC link V1 (legacy)Network Load BalancerStill supported; AWS recommends not creating new V1 links
REST APIVPC link V2 (Nov 2025)Application Load Balancer, directlyRemoves the NLB hop; API, link and ALB in the same account
HTTP APIVPC link V2ALB, NLB, or AWS Cloud Map servicesLink ENIs live in your subnets with a security group
AnyNo VPC linkPublic HTTP endpoint, Lambda, AWS serviceLambda in a VPC needs no VPC link

Exam pattern: "expose a private ALB through API Gateway REST API with least cost and latency" → VPC link V2 to the ALB (since Nov 2025). Older answer keys say NLB + VPC link; both are valid designs, the newer one has one less hop.

Exam traps

Edge and API traps

Wrong answers that look right.

Certificates

  • CloudFront and edge-optimized API custom domains use ACM certificates from us-east-1; ALB and Regional APIs use the local Region.

Private API needs policy

  • A private API without a resource policy cannot be invoked.

GA is not a cache

  • Global Accelerator speeds up the path but never caches; for static content choose CloudFront.

Geo restriction scope

  • CloudFront geo restriction covers the whole distribution; WAF geo match can target a path.

Lambda@Edge vs Functions

  • Functions cannot call the network or run on origin events.

VPC origin limits

  • No GWLB, no NLB with TLS listener, NLB must have an SG.
Self-check

Load balancing and edge: can you answer these?

Explain the answer and name the exact setting.

Load balancers

  • Which load balancer types support Lambda, ALB and appliance targets?
  • When is NLB client IP preservation off by default?
  • Why can an ALB with stickiness not use weighted random?
  • How do you give an ALB static IPs?
  • What MTU must a GWLB appliance support, and why?

Kubernetes

  • IP mode vs instance mode: what is the extra hop?
  • What does group.name change?
  • When do you use TargetGroupBinding?
  • Which subnet tags does the controller look for?
  • Why use pod readiness gates?

Edge and API

  • CloudFront vs Global Accelerator for a UDP app?
  • Traffic dial vs endpoint weight?
  • What can VPC origins front, and what can they not?
  • Edge-optimized vs Regional + own CloudFront: when each?
  • Which VPC link reaches an ALB from a REST API today?
Chapter 7

Network security

Domain 4 · 24% — threat models, inspection architectures, governance at scale, and encryption in transit.

Chapter 7 · Part 7.1

Perimeter protection

Threat models for inbound, outbound and east-west flows; AWS WAF and AWS Shield from the network point of view.

Task 4.1 · Threat models

Start from the flow, then pick the control

The exam describes a flow and a threat. Name the layer, then the cheapest control that stops it there.

FlowTypical threatsNetwork controlsWhere they sit
Inbound (internet → app)DDoS, L7 exploits (SQLi, XSS), bots, scanningShield, WAF, CloudFront, ALB, SG, Network Firewall (ingress)Edge, then public subnets
Outbound (app → internet)Data exfiltration, malware callbacks (C2), crypto-mining poolsNetwork Firewall domain / IP rules, DNS Firewall, proxies, NATEgress VPC or per-VPC firewall subnet
East-west (VPC ↔ VPC)Lateral movement after a compromiseSG referencing, NACLs, Transit Gateway route segmentation, Network FirewallTGW route tables, inspection VPC
Hybrid (on-premises ↔ AWS)Untrusted branch networks, sniffing on the circuitInspection of VPN / DX traffic, IPsec, MACsec, prefix filteringTGW + inspection VPC, DX location
To AWS servicesData copied to a bucket outside the companyVPC endpoint policies, aws:PrincipalOrgID, aws:ResourceOrgIDEndpoint ENIs, resource policies
Name resolutionDNS tunneling, lookups of malicious domainsRoute 53 Resolver DNS Firewall, query logsVPC Resolver (VPC+2)

Defense in depth: one control per layer, never one control for everything. Security groups stay on every ENI even when a firewall sits in front.

Task 4.1 · Layered perimeter

A three-tier VPC with a control at every hop

Each arrow crosses a control. Only the ALB is reachable from the internet; the database tier has no route out at all.

UsersinternetCloudFront+ WAF + ShieldVPC 10.0.0.0/16Public subnetsPrivate app subnetsIsolated data subnetsALBSG: CloudFront prefix listNAT gatewayegress onlyApp (ECS / EC2)SG: from ALB SGAuroraSG: from app SGInterface endpointsendpoint policyRoute table: local onlyNACL: deny all but app CIDRNo NAT, no IGW routeOrigin: VPC originor prefix list + header

Exam pattern: "only CloudFront can reach the ALB" → SG allows the managed prefix list com.amazonaws.global.cloudfront.origin-facing, or use CloudFront VPC origins with an internal ALB.

Task 4.1 · Securing inbound

Where AWS WAF and AWS Shield attach

WAF is a layer 7 filter on specific resources. Anything that is not HTTP needs Shield, Network Firewall or security groups.

AWS WAF attaches to

  • CloudFront (web ACL created in us-east-1, scope CLOUDFRONT).
  • Regional: ALB, API Gateway REST API, AppSync, Cognito user pool, App Runner, Verified Access, Amplify.
  • Not NLB, not GWLB, not EC2 directly, not API Gateway HTTP APIs.
  • Firewall Manager pushes one web ACL to every account.

AWS Shield

  • Standard: free, automatic L3/L4 protection for every AWS customer.
  • Advanced: US$3,000/month per organization (payer account), 1-year commitment.
  • Advanced adds: Shield Response Team (needs Business or Enterprise Support), cost protection, health-based detection, L7 mitigation with WAF.
  • Protects CloudFront, Route 53, Global Accelerator, ALB, CLB, and EC2 Elastic IPs (which covers NLB through its EIPs).

Network angle

  • Absorb at the edge: CloudFront and Global Accelerator spread a flood across many edge locations.
  • Do not expose an EC2 directly; put an ALB or NLB in front.
  • Keep the origin private so attackers cannot bypass the edge.
  • Rule and bot details: see the AWS Security & Governance deck, Chapter 5.
Task 4.1 · Securing outbound and east-west

Controls for traffic leaving a workload

Outbound and east-west threats are the ones WAF never sees.

RequirementControlNotes
Allow only listed internet domainsNetwork Firewall stateful domain list (HTTP Host, TLS SNI)No decryption needed for SNI matching; TLS inspection for full URLs
Block lookups of malicious domainsResolver DNS FirewallCheap first layer; does not stop direct-to-IP traffic
Block known C2 / malware IPsNetwork Firewall managed threat signature and domain list rule groupsManaged by AWS, updated automatically
Inspect with a third-party NGFWGateway Load Balancer + appliance fleetPalo Alto, Fortinet, Check Point, …
Stop app tier talking to other VPCsSeparate TGW route tables, blackhole routesRouting is the coarse control
Limit which services a tier can callSecurity groups referencing other SGsWorks across TGW and Cloud WAN for inbound rules in the same Region
Keep S3 traffic inside the companyGateway endpoint policy with aws:ResourceOrgIDStops copying data to a personal bucket
Task 4.1 · Architecture patterns

Perimeter VPC, three-tier, and untrusted networks

Names the exam uses for common designs.

Perimeter (DMZ) VPC

  • A dedicated VPC owns all internet ingress and egress.
  • Workload VPCs have no IGW; they reach the internet through TGW → perimeter VPC.
  • One place for WAF, Network Firewall, NAT and logging.
  • Enforce with an SCP that denies ec2:CreateInternetGateway outside the network account.

Three-tier

  • Public: load balancers and NAT only.
  • Private: application; outbound through NAT or a firewall.
  • Isolated: databases; no route outside the VPC.
  • SG chaining: ALB SG → app SG → DB SG.

Untrusted networks

  • Treat partners, branches and acquired companies as untrusted.
  • Land them on a separate TGW route table or VRF-like segment.
  • Inspect before they reach workloads.
  • Expose only specific services with PrivateLink instead of full routing.

Testing compliance (Task 4.1): re-run Reachability Analyzer paths and failover tests after every change, and alert on drift with AWS Config.

Exam traps

Perimeter traps

Wrong answers that look right.

WAF on NLB

  • WAF cannot attach to an NLB. For TCP/UDP apps use Shield Advanced, Network Firewall or SGs.

Origin bypass

  • WAF on CloudFront is useless if the ALB still accepts traffic from anywhere.

DNS Firewall alone

  • Malware that connects straight to an IP never sends a DNS query.
  • Pair DNS Firewall with IP or SNI filtering.

NACL for app rules

  • NACLs are stateless and coarse. Fine-grained app rules belong in SGs.

Shield Advanced for SQLi

  • Shield is DDoS protection. SQL injection needs WAF rules.

One firewall for all

  • A central firewall without SGs on each ENI leaves east-west flows inside a VPC uninspected.
Exam-style scenarios

Perimeter scenarios

Scenario → answer → why.

ScenarioAnswerWhy
Workload VPCs must never have direct internet access; central team controls egressEgress / perimeter VPC behind Transit Gateway + SCP denying IGW creationRouting forces the path; SCP stops bypass
Game server on UDP behind an NLB is hit by floodsShield Advanced on the NLB Elastic IPs (or Global Accelerator)WAF is HTTP-only; Shield covers L3/L4
Only allow outbound HTTPS to *.github.com and *.amazonaws.comNetwork Firewall stateful domain allow list with default dropSNI-based filtering needs no decryption
Partner network connects over VPN and must reach only one APIPrivateLink endpoint service for the API (or a dedicated TGW route table)Exposes one service, not the whole network
Developers copy data to personal S3 buckets from EC2S3 gateway endpoint policy with aws:ResourceOrgIDEndpoint policy limits which buckets are reachable
Chapter 7 · Part 7.2

AWS Network Firewall and inspection architectures

Stateless and stateful engines, Suricata rules, TLS inspection, and the distributed, centralized and GWLB deployment models.

Task 4.1 · Building blocks

AWS Network Firewall: what you create

A managed, horizontally scaling firewall built on Gateway Load Balancer technology, with a Suricata-compatible engine.

Resources

  • Rule groups: stateless or stateful; fixed capacity set at creation.
  • Firewall policy: ordered rule groups + default actions + optional TLS inspection configuration.
  • Firewall: binds the policy to a VPC and to one subnet per AZ.
  • Firewall endpoint: one per AZ; you route traffic to it like a VPC endpoint (vpce-…).
  • Logging: alert, flow and TLS logs.

Placement rules

  • Dedicated firewall subnet per AZ; at least one free IP.
  • An endpoint cannot inspect traffic of its own subnet, so put nothing else there.
  • Up to 100 Gbps per AZ, shared by all endpoint associations.
  • VPC endpoint associations: extra endpoints for the same firewall in other VPCs or the same AZ.
  • Can be shared across accounts with AWS RAM.

Not supported

  • Inspecting traffic over VPC peering.
  • Global Accelerator traffic with client IP preservation on.
  • Queries to AmazonProvidedDNS (use DNS Firewall).
  • VPCs in VPC Encryption Controls enforce mode.
Task 4.1 · Packet path

How a packet moves through the two engines

Stateless runs first on every packet; stateful sees whole flows and application protocols.

Route to endpoint

  • A route table sends traffic to vpce-….
  • Ingress routing on the IGW for inbound.
  • Symmetric routing in both directions is required.
→

Stateless engine

  • Per-packet, 5-tuple + TCP flags.
  • Ordered by priority, first match wins.
  • Actions: Pass, Drop, Forward to stateful.
  • Default action for unmatched packets: usually Forward to stateful.
→

Stateful engine

  • Suricata-compatible, flow-aware.
  • Standard 5-tuple, domain list and Suricata IPS rules.
  • Action order or strict order.
  • Pass, Drop, Reject, Alert.
→

Out

  • Passed traffic returns to the VPC route table of the firewall subnet.
  • Alerts and flows go to S3, CloudWatch Logs or Firehose.

Fragmented and asymmetric traffic breaks stateful inspection. If only one direction of a flow crosses the firewall, the stateful engine drops it.

Task 4.1 · Rule types

Choosing the right rule group

Pick the simplest rule type that meets the requirement; stateful rules cost more capacity.

Rule group typeMatches onUse for
StatelessSource/destination IP and port, protocol, TCP flagsFast drops of obvious junk; forwarding everything else to stateful
Stateful 5-tuple (standard)IP, port, protocol, directionSimple allow/deny lists without Suricata syntax
Stateful domain listHTTP Host header and TLS SNIEgress allow lists such as .github.com
Suricata-compatible IPS rulesSuricata keywords such as tls.sni, http.host, http.uri, geoipDetailed L7 policy and intrusion signatures
AWS managed rule groupsThreat signatures, malware/botnet domain lists, active threat defenseBaseline protection maintained by AWS
Partner managed rule groupsVendor threat intelligence from AWS MarketplaceExtra feeds without running appliances

Geographic IP filtering matches the source or destination country of IPv4 and IPv6 traffic, for example to block all traffic to or from a sanctioned country.

Task 4.1 · Rule order

Action order vs strict order

AWS recommends strict order. Rule groups and the policy must use the same ordering.

Action order (default)

  • Evaluation by action, not by position: pass → drop → reject → alert.
  • Inside one action group, the Suricata priority keyword (1–65535) orders rules.
  • A matching pass stops inspection of the rest of that flow.
  • Surprise: a broad pass rule beats a specific drop rule.

Strict order (recommended)

  • Rule groups run by priority (lowest first); rules run in the order written.
  • You choose default actions for unmatched traffic:
  • Drop all, Drop established, or Application drop established (bidirectional or server-directed only).
  • Alert all / Alert established log what a drop would do, without dropping.

Default-deny egress

  • Use strict order + Drop established with domain list allow rules.
  • Drop established lets the TCP handshake through so the firewall can read the SNI or Host header.
  • For UDP and ICMP, Drop established drops everything: write explicit pass rules (for example NTP).
  • Start with Alert established, review logs, then switch to drop.
Task 4.1 · Suricata examples

Stateful rules in Suricata syntax

Each rule needs a unique sid. In strict order these run top to bottom.

# Allow HTTPS only to approved domains (match on TLS SNI)
pass tls $HOME_NET any -> $EXTERNAL_NET 443 (tls.sni; dotprefix;
  content:".github.com"; endswith; nocase; flow:to_server;
  msg:"allow github"; sid:100001; rev:1;)
pass tls $HOME_NET any -> $EXTERNAL_NET 443 (tls.sni;
  content:"api.example-partner.com"; startswith; endswith;
  nocase; flow:to_server; sid:100002; rev:1;)

# Allow plain HTTP to the OS package mirror by Host header
pass http $HOME_NET any -> $EXTERNAL_NET 80 (http.host;
  content:"deb.debian.org"; startswith; endswith;
  flow:to_server; sid:100003; rev:1;)

# Block SSH to the internet and log it
drop tcp $HOME_NET any -> $EXTERNAL_NET 22 (flow:to_server;
  msg:"egress ssh blocked"; sid:100010; rev:1;)

# Alert on TLS to a non-standard port (possible tunnel)
alert tls $HOME_NET any -> $EXTERNAL_NET !443 (flow:to_server;
  msg:"tls on odd port"; sid:100020; rev:1;)

How to read them

  • Header: action, protocol, source, port, direction, destination, port.
  • $HOME_NET defaults to the firewall VPC CIDR. For a central firewall, set it to all spoke CIDRs in the rule variables.
  • dotprefix + endswith on .github.com matches subdomains but not evilgithub.com.
  • flow:to_server matches client-to-server packets only.
  • Protocol keywords (tls, http) wait for the app layer; tcp rules can match the first SYN.
Task 4.3 · TLS inspection

Decrypting traffic in Network Firewall

Without decryption, the firewall sees only SNI and certificates. With it, rules can match URLs, headers and payloads.

Inbound inspection

  • Protects servers in your VPC.
  • Import the server certificate and key into ACM.
  • Firewall decrypts, inspects with stateful rules, re-encrypts to the target.
  • Scope by destination IP/port of your servers.

Outbound inspection

  • Protects clients going to the internet.
  • Provide a CA certificate in ACM (imported, or issued by AWS Private CA).
  • The firewall generates certificates on the fly, signed by that CA.
  • Every client must trust the CA; pinned certificates break.

Costs and caveats

  • Billed as advanced inspection per GB, on top of endpoint hours.
  • Create a TLS inspection configuration → add to a new policy → associate the policy.
  • Exclude sensitive destinations (banking, health) by scope.
  • TLS logs record decryption errors and certificate revocation results.
Task 4.1 · Distributed model

A firewall in every VPC

Each VPC has its own firewall endpoints; ingress routing on the IGW sends inbound traffic through them.

InternetIGWedge route tableVPC 10.1.0.0/16 · AZ a (AZ c mirrors it)Firewall subnetPublic subnet (ALB / NAT)Private subnetFirewall endpointvpce-fw-aALB node10.1.1.0/24App10.1.2.0/24IGW edge RT:10.1.1.0/24 → vpce-fw-aFirewall subnet RT:0.0.0.0/0 → igwPublic subnet RT:0.0.0.0/0 → vpce-fw-aPrivate RT:0.0.0.0/0 → NAT

Pros: no single choke point, blast radius of one VPC, no Transit Gateway data processing. Cons: many firewalls to pay for and manage (use Firewall Manager), and east-west traffic between VPCs is not inspected unless it also crosses a firewall.

Task 4.1 · Centralized model

One inspection VPC behind Transit Gateway

Spokes send everything to the inspection VPC; the firewall returns it to TGW, which routes it to the destination.

Spoke VPC A 10.1.0.0/16App ASpoke VPC B 10.2.0.0/16App BTransit Gatewaytwo route tablesInspection / egress VPCTGW subnetsFirewall subnetsPublic subnetsTGW ENIappliance mode ONFirewallendpoint per AZNAT GWIGWSpoke RT: 0.0.0.0/0 → inspection att.Firewall RT: spoke CIDRs propagatedTGW subnet RT: 0.0.0.0/0 → vpce-fw · Firewall RT: 10.0.0.0/8 → tgw, 0/0 → NAT

Two TGW route tables: spokes are associated with the "spoke" table (default route to the inspection attachment); the inspection attachment is associated with the "firewall" table, where spoke routes are propagated.

Task 4.1 · Appliance mode

Why centralized inspection needs appliance mode

Transit Gateway keeps traffic in the source AZ by default. Two spokes in different AZs then cross different firewall endpoints.

Without appliance modeApp AAZ aApp BAZ cTGWFW endpoint aFW endpoint crequestreplyRequest uses endpoint a, reply uses endpoint c.Endpoint c never saw the SYN → drops the reply.With appliance mode on the inspection attachmentApp AAZ aApp BAZ cTGWFW endpoint aFW endpoint cRequest and replyboth use endpoint aTGW hashes the flow and picks one AZfor its whole lifetime → symmetric.
aws ec2 modify-transit-gateway-vpc-attachment --transit-gateway-attachment-id tgw-attach-0abc \
  --options ApplianceModeSupport=enable
Task 4.1 · Native TGW attachment

Network Firewall attached directly to Transit Gateway

Available in all Regions with both services since July 2025. The firewall becomes a TGW attachment type.

What changes

  • No inspection VPC, firewall subnets or VPC route tables to build.
  • You create the firewall with a Transit Gateway instead of a VPC.
  • Multi-AZ redundancy is automatic.
  • Inspects VPC, VPN and Direct Connect traffic that crosses the TGW.

What stays the same

  • Same firewall policies, rule groups, TLS inspection and logging.
  • You still steer traffic with TGW route tables (spoke table → firewall attachment).
  • Egress to the internet still needs a NAT path somewhere (an egress VPC).
  • Firewall policy and rule changes work exactly as before.

When to pick which

  • Native attachment: new builds, least operational effort, AWS-native rules only.
  • Inspection VPC: you need custom subnets, a NAT/IGW in the same VPC, or GWLB appliances alongside.
  • Distributed: strict per-VPC isolation or very high per-VPC throughput.
Task 4.1 · Gateway Load Balancer

Third-party appliances with GWLB

GWLB is a layer 3 load balancer: it keeps flows sticky to one appliance and wraps packets in GENEVE.

InternetIGWedge RTApplication VPCGWLBe subnetApp subnetGWLB endpointvpce-gwlbApp / ALBSecurity (appliance) VPCGWLBone per AZ ENIFirewall 1Firewall 2Firewall 3PrivateLink (GWLBe → GWLB)GENEVE UDP 6081 to appliancesIGW edge RT: app CIDR → vpce-gwlb

Facts

  • Appliances must support GENEVE (UDP 6081) and health checks.
  • Flow stickiness: 5-tuple by default, or 3-tuple / 2-tuple.
  • GWLB endpoint supports 8,500-byte MTU.

Use when

  • A vendor NGFW is mandated, or features beyond Network Firewall are needed.
  • Endpoints in many VPCs share one central appliance fleet.
Task 4.1 · Egress filtering

Options for controlling outbound internet access

From cheapest to most capable. Many designs combine DNS Firewall with one of the inline options.

OptionHow it filtersStrengthsLimits
NAT gateway onlyNo filtering; hides private IPsSimple, managedAny destination allowed
Resolver DNS FirewallBlocks DNS answers for listed domainsCheap, catches DNS tunnelingBypassed by hard-coded IPs or other resolvers
Network Firewall (transparent)Domain lists, SNI, IP, Suricata, TLS inspectionManaged, no client changesEndpoint hours + per-GB processing
Network Firewall proxy modeExplicit proxy (HTTP CONNECT), no source preservation; uses the attached NAT gatewaySame policy for proxy and transparent filteringPreview (re-introduced Aug 2026, us-east-2 only)
Self-managed proxy (Squid) behind NLBURL allow lists, authenticationFull control, URL-level rulesYou patch and scale it; clients need proxy settings
Third-party firewall via GWLBVendor URL filtering, IPS, user-IDFeature parity with on-premisesLicenses and appliance operations
Task 4.2 · Firewall logging

Network Firewall logs and metrics

Logging is configured per firewall and per log type; each type can go to a different destination.

Log types

  • Alert: packets that matched a rule with drop, reject or alert.
  • Flow: standard network flow records for traffic the stateful engine forwards.
  • TLS: TLS inspection events such as certificate errors and revocation checks.
  • Records are Suricata EVE JSON with firewall metadata.

Destinations

  • Amazon S3: cheapest for long retention; query with Athena.
  • CloudWatch Logs: metric filters and alarms, Logs Insights.
  • Data Firehose: stream to OpenSearch, Splunk or a SIEM.
  • Security Lake can ingest the findings for organization-wide analysis.

CloudWatch metrics

  • Namespace AWS/NetworkFirewall, per AZ endpoint.
  • ReceivedPackets, PassedPackets, DroppedPackets, RejectedPackets, OtherDroppedPackets (includes throttling).
  • TLS inspection: TLSErrors, TLSDroppedPackets, revocation-status counters.
  • Flow logs on the firewall subnet ENIs show what reached the endpoint.
Exam traps

Network Firewall and inspection traps

Wrong answers that look right.

Forgot appliance mode

  • Centralized inspection across AZs without appliance mode breaks stateful flows.

Firewall in the app subnet

  • An endpoint cannot inspect its own subnet. Use dedicated firewall subnets.

$HOME_NET

  • A central firewall with default $HOME_NET only treats the inspection VPC as home. Add the spoke CIDRs.

Asymmetric ingress

  • IGW edge route sends inbound through the firewall, but the subnet route sends replies straight to the IGW → dropped.

Action order surprise

  • In action order, a broad pass rule wins over a later, specific drop. Use strict order.

GWLB vs NLB

  • NLB does not preserve flows to appliances transparently or speak GENEVE. Inline appliances need GWLB.
Exam-style scenarios

Inspection scenarios

Scenario → answer → why.

ScenarioAnswerWhy
Inspect all traffic between 50 VPCs and to on-premises with least operational effortNetwork Firewall with native Transit Gateway attachmentNo inspection VPC to build; inspects VPC, VPN and DX traffic
Security team must use its existing Palo Alto policy in AWSGWLB with the vendor appliances; GWLB endpoints in spokes or inspection VPCGWLB is built for third-party inline appliances
Some flows through the central firewall drop randomly; others workEnable appliance mode on the inspection VPC attachmentCross-AZ flows were asymmetric
Block outbound traffic to any domain not on an approved list, without decryptingStrict order, Drop established, domain list allow rules on SNI and HostSNI is visible in the TLS Client Hello
Need to block a specific malicious URL path over HTTPSNetwork Firewall TLS inspection (outbound) + http.uri ruleThe path is encrypted; only decryption exposes it
Chapter 7 · Part 7.3

Governance, audit and zero trust

Firewall Manager across the organization, configuration audits, AWS Verified Access, and guardrails as code.

Task 4.2 · Firewall Manager

AWS Firewall Manager: one policy, every account

Central management of network security policies across an AWS Organization, including accounts created later.

Prerequisites

  • AWS Organizations with all features enabled.
  • A Firewall Manager administrator account (management account designates it).
  • AWS Config enabled in every account and Region in scope.
  • Shield Advanced subscription for Shield policies only.

Policy types

PolicyEnforces
AWS WAF (and WAF Classic)First and last rule groups in every web ACL; accounts add rules in between
Shield AdvancedProtections on chosen resource types
Security groupCommon baseline SGs, content audit, usage audit
Network ACLBaseline first and last NACL rules
Network FirewallFirewalls and policies in VPCs (distributed or centralized)
Resolver DNS FirewallRule group associations on VPCs
Third-party firewallPalo Alto Cloud NGFW, Fortigate CNF

Scope by account, OU, resource type or tag. Choose "identify only" first to see non-compliant resources, then turn on auto-remediation.

Task 4.2 · Security group policies

Three kinds of Firewall Manager security group policy

The most frequently tested Firewall Manager feature.

Common security group

  • Attaches a primary SG (for example "allow from corporate monitoring") to every in-scope ENI, instance or ALB.
  • Can share the SG into accounts and keep it attached.
  • Use for org-wide baseline access.

Content audit

  • Checks SG rules against an audit SG or managed rule list.
  • Example: flag or remove rules that allow 0.0.0.0/0 on 22 or 3389.
  • Allow-list or deny-list style.
  • Auto-remediation can delete offending rules.

Usage audit

  • Finds unused security groups (not attached for a set time).
  • Finds redundant SGs with identical rules.
  • Optional automatic clean-up.
  • Keeps SG sprawl and quota usage down.
Task 4.2 · Audit the configuration

AWS Config, Trusted Advisor and analyzers

Firewall Manager enforces; these tools continuously report on what exists.

ToolWhat it checksExample
AWS Config managed rulesResource configuration against a rule, with historyrestricted-ssh, restricted-common-ports, vpc-sg-open-only-to-authorized-ports, vpc-default-security-group-closed, vpc-flow-logs-enabled
Config conformance packsA bundle of rules deployed org-wideOperational best practices for networking; remediation with SSM Automation
AWS Trusted AdvisorAccount-level best-practice checks"Security Groups – Specific Ports Unrestricted", "Security Groups – Unrestricted Access"
Network Access AnalyzerPaths that violate an intent (e.g. internet → database)Finds unintended reachability across the account
Security Hub CSPMAggregated findings with controls such as EC2 security group checksOne dashboard across accounts
CloudTrailWho changed the network and whenAuthorizeSecurityGroupIngress, CreateRoute, ModifyVpcEndpoint
Task 4.1 · Zero trust access

AWS Verified Access

Per-request access decisions based on identity and device posture, without a VPN.

Components

  • Instance: evaluates requests.
  • Trust providers: user identity (IAM Identity Center or any OIDC IdP) and device (Jamf, CrowdStrike, JumpCloud).
  • Groups: endpoints that share a policy.
  • Policies: written in Cedar, using claims such as group membership and device risk score.
  • Endpoints: what users reach.

Endpoint types

  • HTTP(S): load balancer or network interface.
  • Non-HTTP(S) (GA Feb 2025): TCP, SSH, RDP to an ENI, load balancer, RDS instance / proxy / cluster, or a CIDR range.
  • Non-HTTP access uses the Verified Access client on the device.
  • Access logs to CloudWatch Logs, S3 or Firehose.

Verified Access vs Client VPN

  • Client VPN: network-level access to a VPC; authorization rules by CIDR and group.
  • Verified Access: application-level, every request evaluated, no network-wide access.
  • Pick Verified Access for "zero trust", "no VPN", "device posture".
  • Billing: per application hour + per GB processed.
Task 4.1 · Guardrails as code

SCPs and endpoint policies as network controls

Preventive controls stop teams from building around the network design.

{
  "Sid": "OnlyNetworkAccountBuildsEdges",
  "Effect": "Deny",
  "Action": [
    "ec2:CreateInternetGateway",
    "ec2:AttachInternetGateway",
    "ec2:CreateEgressOnlyInternetGateway",
    "ec2:CreateVpcPeeringConnection",
    "ec2:AcceptVpcPeeringConnection"
  ],
  "Resource": "*",
  "Condition": {
    "ArnNotLike": { "aws:PrincipalArn":
      "arn:aws:iam::*:role/NetworkAdmin" }
  }
}
{
  "Statement": [{
    "Sid": "OnlyOurOrgBuckets",
    "Effect": "Allow",
    "Principal": "*",
    "Action": ["s3:GetObject", "s3:PutObject"],
    "Resource": "*",
    "Condition": {
      "StringEquals": {
        "aws:ResourceOrgID": "o-abc123",
        "aws:PrincipalOrgID": "o-abc123"
      }
    }
  }]
}

Left: SCP statement in workload OUs. Right: S3 gateway endpoint policy that limits both who uses the endpoint and which buckets it reaches. More data perimeter patterns are in the AWS Security & Governance deck.

Exam traps and scenarios

Governance and zero trust

Scenario → answer → why.

ScenarioAnswerWhy
Remove any SG rule that opens SSH to the world in 200 accounts, automaticallyFirewall Manager content audit SG policy with auto-remediationCentral, covers new accounts, remediates
Attach the security team’s scanner SG to every EC2 instanceFirewall Manager common SG policyBaseline SG on every in-scope resource
Firewall Manager policy shows no resourcesEnable AWS Config in those accounts and RegionsFirewall Manager depends on Config
Contractors need SSH to a few hosts, no VPN, with device checksVerified Access non-HTTP endpoint + device trust providerPer-connection, posture-aware access
Workload accounts keep creating VPC peerings that bypass inspectionSCP denying ec2:CreateVpcPeeringConnection except for network adminsPreventive, organization-wide
Report which accounts have SGs open to 0.0.0.0/0 on 3389AWS Config rule restricted-common-ports with an aggregatorContinuous evaluation and history
Chapter 7 · Part 7.4

Encryption in transit

Who encrypts what, Nitro encryption, VPC Encryption Controls, IPsec and MACsec for hybrid links, and TLS everywhere.

Task 4.3 · Shared responsibility

Which traffic AWS already encrypts

AWS encrypts several layers automatically. Your data between your endpoints is still yours to protect.

LayerEncrypted byConditions
Between AWS Regions and between AZsAWS, at the physical layerAll traffic on the AWS global network leaving secured facilities
Inter-Region VPC peering and TGW peeringAWS, bulk encryptionAutomatic when traffic exits a Region
Between EC2 instancesNitro hardware (AEAD, 256-bit), no performance impactSupported instance types on both ends, same Region, same or peered VPC, no load balancer or TGW in the path
VPC Encryption Controls enforce modeNitro + application layer, verified by AWSAll resources in the VPC must be compatible or excluded
Site-to-Site VPN, Client VPNIPsec / TLS by AWS and your deviceOver the internet or over Direct Connect
Direct ConnectNot encrypted by defaultAdd MACsec (L2) or IPsec VPN over DX (L3)
Application dataYou: TLS, mTLSLoad balancers, databases, S3, APIs
aws ec2 describe-instance-types --filters Name=network-info.encryption-in-transit-supported,Values=true \
  --query "InstanceTypes[*].InstanceType" --output text
Task 4.3 · VPC Encryption Controls

Monitor, then enforce encryption in transit

Launched Nov 2025; paid per non-empty VPC per hour since 1 Mar 2026. Uses Nitro encryption plus application-layer TLS.

Modes

  • Monitor: shows encryption status of flows; lists resources that block enforcement (get-vpc-resources-blocking-encryption-enforcement).
  • Enforce: blocks creating or attaching resources that allow cleartext.
  • Existing VPCs must start in monitor; new VPCs can be created in enforce.
  • Also set at account level or with an Organizations declarative policy.

Exclusions (the only 8)

  • Internet gateway, NAT gateway, egress-only IGW.
  • VPC peering to a non-enforced VPC.
  • Virtual private gateway.
  • Lambda in VPC, VPC Lattice, EFS.
  • You own encryption for excluded paths.

Gotchas

  • GWLB and Network Firewall are not supported in enforce mode.
  • Old non-Nitro instances must be replaced; ALB, NLB, Fargate migrate automatically.
  • NLB TLS listeners cannot offload TLS in an enforced VPC.
  • TGW needs encryption support turned on explicitly.
  • Audit with flow log field encryption-status (0–3).
Task 4.3 · Hybrid encryption

IPsec over Direct Connect and MACsec

Direct Connect is private but not encrypted. Pick the layer that matches the requirement.

MACsec (layer 2)

  • Point-to-point between your router and the AWS DX device at the location.
  • Dedicated 10, 100 and 400 Gbps connections at selected locations; also LAGs. Not hosted connections.
  • Cipher: GCM-AES-256 or GCM-AES-XPN-256 (XPN required at 100/400 Gbps).
  • Static CAK mode only: you supply CKN/CAK (64 hex chars) with associate-mac-sec-key.
  • Modes should_encrypt (default) and must_encrypt. No extra charge.

IPsec VPN over DX (layer 3)

  • Private IP VPN: Site-to-Site VPN with private outer IPs over a transit VIF to a Transit Gateway.
  • Or a public VIF that carries a normal public VPN endpoint.
  • End-to-end from your router to TGW, not just the first hop.
  • Throughput limited per tunnel; ECMP across tunnels on TGW.

How to choose

  • "Encrypt at line rate on 100 Gbps" → MACsec.
  • "Encryption all the way into AWS / to the VPC" → IPsec VPN over DX.
  • "Must not traverse the internet and must be encrypted" → private IP VPN over DX.
  • Compliance requiring both → MACsec + application TLS.
Task 4.3 · TLS on load balancers and edge

Certificates and TLS termination choices

Where TLS ends decides who can see plaintext and which certificates you manage.

Certificate sources

  • ACM public: free, auto-renewed, for ELB, CloudFront, API Gateway.
  • CloudFront certificates must be in us-east-1; ELB certificates in the LB Region.
  • AWS Private CA: private PKI for internal names, mTLS clients, service mesh, IoT.
  • Imported certificates are not auto-renewed: alarm on DaysToExpiry.

Termination patterns

  • ALB: terminate TLS, optionally re-encrypt to targets (HTTPS target group).
  • NLB TLS listener: terminate at L4, or TCP listener for passthrough to targets.
  • Passthrough keeps end-to-end encryption; the LB cannot see headers.
  • Security policies set minimum TLS version and ciphers (TLS 1.3 policies exist).

Mutual TLS

  • ALB mTLS modes: verify (ALB validates client certs against a trust store) or passthrough (sends the chain to targets in headers).
  • Trust store holds your CA bundle in S3; optional revocation lists.
  • API Gateway custom domains also support mTLS with a trust store.
  • Use for B2B APIs and device fleets.
Task 4.3 · Enforce TLS to services

Make cleartext impossible, not just discouraged

Deny non-TLS requests at the resource so a misconfigured client fails loudly.

{
  "Sid": "DenyInsecureTransport",
  "Effect": "Deny",
  "Principal": "*",
  "Action": "s3:*",
  "Resource": [
    "arn:aws:s3:::payroll-data",
    "arn:aws:s3:::payroll-data/*"
  ],
  "Condition": {
    "Bool": { "aws:SecureTransport": "false" }
  }
}

Other services

  • S3 also supports s3:TlsVersion to require a minimum TLS version.
  • RDS for PostgreSQL: parameter rds.force_ssl = 1.
  • RDS for MySQL / MariaDB: require_secure_transport = ON.
  • ElastiCache: in-transit encryption on the cluster.
  • SQS / SNS: aws:SecureTransport deny in the queue or topic policy.
  • Clients should verify the server certificate (RDS CA bundle).
Exam traps and scenarios

Encryption in transit

Scenario → answer → why.

ScenarioAnswerWhy
Encrypt a 100 Gbps dedicated DX at line rateMACsec with GCM-AES-XPN-256IPsec tunnels cannot reach that throughput
Encrypt DX traffic end to end into the VPCs without using the internetPrivate IP VPN over a transit VIF to Transit GatewayIPsec from your router to TGW with private outer IPs
Prove all east-west traffic in a VPC is encrypted for an auditorVPC Encryption Controls in monitor, then enforce; flow logs with encryption-statusAuthoritative status per flow
Enforce mode is blocked by the central firewall in the VPCKeep that VPC in monitor mode (GWLB / Network Firewall unsupported in enforce)Documented limitation
Instances on supported types talk via an NLB; is Nitro encryption automatic?No; add TLSNitro encryption excludes paths through load balancers or TGW
Hosted 1 Gbps DX must be encryptedIPsec VPN over DXMACsec is not available on hosted connections
Self-check

Network security: can you answer these?

Explain the answer and name the exact control.

Perimeter and inspection

  • Which resources can AWS WAF attach to, and what protects an NLB?
  • Why must firewall endpoints sit in dedicated subnets?
  • What does appliance mode change in Transit Gateway?
  • Action order vs strict order: which wins, a broad pass or a specific drop?
  • When do you choose GWLB over Network Firewall?

Governance

  • Name the three Firewall Manager security group policy types.
  • What must be enabled before Firewall Manager can evaluate an account?
  • How does an SCP keep workload accounts from bypassing a perimeter VPC?
  • Verified Access vs Client VPN: when is each the right answer?
  • Which Config rule finds RDP open to the world?

Encryption

  • Which EC2 traffic is encrypted automatically, and when is it not?
  • Name four of the eight VPC Encryption Controls exclusions.
  • MACsec vs IPsec over DX: layer, speed and scope?
  • Where must a CloudFront ACM certificate live?
  • How do you force TLS for every request to an S3 bucket?
Chapter 8

Monitoring and troubleshooting

Flow logs, mirroring, metrics and analyzers — then a repeatable playbook for the failures you will actually see.

Chapter 8 · Part 8.1

Visibility tools

VPC Flow Logs in depth, Athena queries, Traffic Mirroring, access logs, CloudWatch metrics and network monitors.

Task 1.4 · Tool map

Which tool answers which question

Pick the tool from the question. Most investigations use two or three of them together.

QuestionToolWhat you get
Was this flow accepted or rejected, and by which ENI?VPC Flow Logs5-tuple, bytes, ACCEPT / REJECT per ENI
What is inside the packets?VPC Traffic MirroringFull packet copies (VXLAN) to an IDS or tcpdump host
Should this path work, per the configuration?Reachability AnalyzerHop-by-hop path or the blocking component
Which paths exist that should not?Network Access AnalyzerFindings for paths matching a scope (e.g. internet → DB)
Do TGW route tables send this prefix where I expect?Network Manager Route AnalyzerRoute path through TGW route tables only
Who changed the route table or SG?CloudTrailAPI call, principal, time, parameters
Is a link, tunnel or gateway healthy or saturated?CloudWatch metrics and alarmsTunnel state, DX light levels, NAT port errors, drops
Is latency or loss from AWS or from my app?Network Flow Monitor, Internet Monitor, Network Synthetic MonitorRTT, retransmissions, health indicators
Task 4.2 · Flow Logs basics

VPC Flow Logs: scope, timing and what is not captured

Metadata about IP flows per ENI. Not packet contents, not real time.

Scope and timing

  • Create on a VPC, subnet, ENI, or Transit Gateway / TGW attachment.
  • Aggregation interval: 10 minutes (default) or 1 minute; Nitro instances always 1 minute or less.
  • Delivery is best effort: typically ~5 min to CloudWatch Logs, ~10 min to S3.
  • A flow log cannot be edited: delete and recreate to change role or format.
  • Up to 250 subscriptions per resource.

Not captured

  • Queries to the Amazon DNS server (queries to your own DNS server are logged).
  • Instance metadata 169.254.169.254 and Time Sync 169.254.169.123.
  • DHCP, ARP, Windows license activation.
  • Traffic to the reserved VPC router address.
  • Mirrored source traffic (only the mirror target sees it).
  • Traffic between an endpoint ENI and an NLB ENI.

Gotchas

  • Secondary IPs: dstaddr shows the primary IP; add pkt-dstaddr for the real one.
  • Through NAT or EKS pods: use pkt-srcaddr / pkt-dstaddr.
  • log-status SKIPDATA means records were dropped during that interval.
  • You cannot enable flow logs on a peer VPC in another account.
  • Use Resolver query logs for DNS, not flow logs.
Task 4.2 · Default format

The version 2 default record, field by field

The default format is fixed: the 14 version-2 fields in this order. Anything else needs a custom format.

# version account-id interface-id srcaddr dstaddr srcport dstport
#   protocol packets bytes start end action log-status
2 111122223333 eni-0a1b2c3d4e5f60718 203.0.113.45 10.0.1.25
  51522 22 6 4 240 1760947200 1760947260 REJECT OK
2 111122223333 eni-0a1b2c3d4e5f60718 10.0.1.25 10.0.10.40
  41030 5432 6 18 3120 1760947200 1760947260 ACCEPT OK
2 111122223333 eni-0a1b2c3d4e5f60718 - - - - - - -
  1760947260 1760947320 - NODATA

Reading the first record

FieldValue → meaning
srcaddr → dstaddrInternet host → instance private IP
srcport → dstportEphemeral 51522 → SSH 22
protocol6 = TCP (17 UDP, 1 ICMP, 58 ICMPv6)
packets / bytes4 packets, 240 bytes: SYN retries
start / endUnix seconds of the capture window
actionREJECT by SG or NACL
log-statusOK, NODATA, SKIPDATA

A REJECT does not say which control blocked it. Inbound REJECT with no matching outbound record: SG or inbound NACL. Inbound ACCEPT followed by outbound REJECT: the outbound NACL (stateless) is missing ephemeral ports.

Task 4.2 · Extended fields

Custom-format fields worth adding

The record version becomes the highest version of the fields you choose.

FieldVer.Why add it
vpc-id, subnet-id, instance-id3Group by subnet or instance without joins
tcp-flags3FIN 1, SYN 2, RST 4, SYN-ACK 18; OR-ed per interval (19 = SYN-ACK + FIN)
pkt-srcaddr, pkt-dstaddr3Original IPs behind NAT gateways, EKS pods, secondary IPs
region, az-id4Find cross-AZ traffic (cost, latency)
pkt-src-aws-service, pkt-dst-aws-service5Label AWS ranges: S3, DYNAMODB, EC2, ROUTE53_RESOLVER, …
flow-direction, traffic-path5ingress / egress, and which gateway egress used
ecs-* fields7Cluster, service, task ARN for awsvpc tasks
reject-reason8BPA (Block Public Access) or EC (Encryption Controls)
encryption-status100 none, 1 Nitro, 2 application, 3 both
next-hop-*, interface-type, tag fields11Next-hop ENI/subnet/VPC; NAT, NLB, TGW, endpoint ENIs; instance/ASG tags
Task 4.2 · traffic-path and NAT

Decode egress paths and NAT translations

Two custom-format examples: which gateway the traffic used, and who is really behind a NAT gateway.

traffic-path values (egress only)

ValueEgress went through
1Another resource in the same VPC (incl. AWS-managed ENIs)
2Internet gateway or gateway VPC endpoint
3Virtual private gateway
4Intra-Region VPC peering
5Inter-Region VPC peering
6Local Zone or Wavelength Zone
7Gateway VPC endpoint
8Internet gateway
# format: interface-id srcaddr pkt-srcaddr dstaddr
#         pkt-dstaddr dstport flow-direction traffic-path
# 1) On the instance ENI: leaving towards the NAT GW
eni-app01 10.0.10.25 10.0.10.25 52.94.8.10
  52.94.8.10 443 egress 1
# 2) On the NAT GW ENI: arriving from the instance
eni-nat01 10.0.10.25 10.0.10.25 10.0.0.9
  52.94.8.10 443 ingress -
# 3) On the NAT GW ENI: leaving to the internet
eni-nat01 10.0.0.9 10.0.10.25 52.94.8.10
  52.94.8.10 443 egress 8

Record 3 is the one that answers "which instance is generating NAT data processing charges": srcaddr is the NAT gateway, pkt-srcaddr is the real source.

Task 4.2 · Destinations

Where to send flow logs

Choose by how you will query them and how long you keep them.

CloudWatch Logs

  • Needs an IAM role that the flow logs service assumes.
  • Logs Insights for ad-hoc queries; metric filters for alarms.
  • Highest ingestion cost per GB.
  • Good for short retention and near-real-time alerting.

Amazon S3

  • Bucket policy for delivery.logs.amazonaws.com; no role.
  • Text or Apache Parquet; optional Hive-compatible prefixes and hourly partitions.
  • Cheapest storage; query with Athena.
  • Central bucket in a Log Archive account works cross-account.

Data Firehose

  • Stream to OpenSearch, Splunk, Datadog, or S3 with transforms.
  • Cross-account delivery to a central stream.
  • Use when a SIEM needs flows within minutes.
fields @timestamp, srcAddr, dstAddr, dstPort, action
| filter action = "REJECT" and dstPort in [22, 3389]
| stats count(*) as attempts by srcAddr, dstPort
| sort attempts desc | limit 20                      # CloudWatch Logs Insights
Task 4.2 · Athena table

Query flow logs in S3 with Athena

Partition projection avoids running MSCK REPAIR or crawlers as new days arrive.

CREATE EXTERNAL TABLE vpc_flow_logs (
  version int, account_id string, interface_id string,
  srcaddr string, dstaddr string, srcport int, dstport int,
  protocol bigint, packets bigint, bytes bigint,
  start bigint, `end` bigint, action string, log_status string)
PARTITIONED BY (`date` date)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ' '
LOCATION
  's3://flowlogs/AWSLogs/111122223333/vpcflowlogs/ap-northeast-1/'
TBLPROPERTIES (
  "skip.header.line.count"="1",
  "projection.enabled"="true",
  "projection.date.type"="date",
  "projection.date.format"="yyyy/MM/dd",
  "projection.date.range"="2026/01/01,NOW",
  "projection.date.interval"="1",
  "projection.date.interval.unit"="DAYS",
  "storage.location.template"=
    "s3://flowlogs/AWSLogs/111122223333/vpcflowlogs/ap-northeast-1/${date}");

Notes

  • Column order must match the flow log format; the file header line is skipped.
  • Default S3 prefix: AWSLogs/<account>/vpcflowlogs/<region>/yyyy/MM/dd/.
  • Always filter on date to scan only the days you need.
  • Parquet output cuts scan cost and query time substantially.
  • The VPC console can generate a CloudFormation template that builds this integration for you.
Task 4.2 · Athena queries

Questions answered with SQL

Run against the table on the previous slide.

-- 1) Who is knocking on SSH/RDP from outside?
SELECT srcaddr, dstport, count(*) AS hits
FROM vpc_flow_logs
WHERE date = DATE '2026-10-20' AND action = 'REJECT'
  AND dstport IN (22, 3389)
  AND NOT (srcaddr LIKE '10.%')
GROUP BY 1, 2 ORDER BY hits DESC LIMIT 20;

-- 2) Top talkers by bytes in the last 7 days
SELECT srcaddr, dstaddr, sum(bytes)/1e9 AS gb
FROM vpc_flow_logs
WHERE date >= current_date - INTERVAL '7' DAY
GROUP BY 1, 2 ORDER BY gb DESC LIMIT 20;
-- 3) Which hosts use the NAT gateway most?
--    (custom-format table with pkt_srcaddr, flow_direction)
SELECT pkt_srcaddr, sum(bytes)/1e9 AS gb
FROM vpc_flow_logs_v5
WHERE date >= current_date - INTERVAL '1' DAY
  AND interface_id = 'eni-nat01'
  AND flow_direction = 'egress'
GROUP BY 1 ORDER BY gb DESC LIMIT 10;

-- 4) Was a specific flow ever accepted?
SELECT from_unixtime(start) AS t, action, packets
FROM vpc_flow_logs
WHERE date = DATE '2026-10-20'
  AND srcaddr = '10.0.10.25' AND dstaddr = '10.0.11.40'
  AND dstport = 5432
ORDER BY t;
Task 4.2 · Traffic Mirroring

VPC Traffic Mirroring: packet copies for deep analysis

Copies packets from a source ENI, wraps them in VXLAN, and sends them to a target for IDS or capture.

Source VPCSource ENINitro instanceMirror filterrules: accept / rejectMirror sessionnumber, VNI, packet lengthTarget (same VPC or connected via peering / TGW)NLB (UDP 4789)GWLB endpointENI of a capture hostIDS / Zeek / Suricataor tcpdumpVXLANUDP 4789
aws ec2 create-traffic-mirror-target --network-load-balancer-arn $NLB_ARN
aws ec2 create-traffic-mirror-filter --description "web only"
aws ec2 create-traffic-mirror-filter-rule --traffic-mirror-filter-id tmf-0abc \
  --traffic-direction ingress --rule-number 100 --rule-action accept \
  --protocol 6 --destination-port-range FromPort=443,ToPort=443 \
  --source-cidr-block 0.0.0.0/0 --destination-cidr-block 0.0.0.0/0
aws ec2 create-traffic-mirror-session --network-interface-id eni-src01 \
  --traffic-mirror-target-id tmt-0abc --traffic-mirror-filter-id tmf-0abc \
  --session-number 1 --packet-length 128      # first 128 bytes only

Limits and rules

  • Max 3 sessions per source ENI (not adjustable); 10 rules per filter.
  • Target SG/NACL must allow UDP 4789 from the source.
  • Mirrored traffic counts against the source instance bandwidth.
  • Use --packet-length to copy headers only.
Task 4.2 · Access logs and audit

Load balancer, CloudFront and CloudTrail records

Flow logs stop at L4. These logs carry the L7 and change history you need to correlate.

SourceContentsDeliveryNetwork use
ALB access logsClient IP:port, target, status codes, latencies, TLS cipher, trace IDS3 (every 5 min)Find 5xx by target, slow targets, client IPs
ALB connection logsTLS handshake details, client certificate info for mTLSS3Debug mTLS and TLS version issues
NLB access logsTLS listener connections: TLS version, cipher, handshake timeS3TLS listeners only
CloudFront standard logsEdge location, client IP, cache result, time takenS3, CloudWatch Logs, FirehoseCache hit ratio, geographic issues
CloudFront real-time logsConfigurable fields within secondsKinesis Data StreamsLive dashboards and alerting
CloudTrailCreateRoute, AuthorizeSecurityGroupIngress, ModifyTransitGatewayVpcAttachmentS3, CloudWatch Logs, EventBridge"What changed right before the outage?"
VPC Resolver query logsQuery name, type, response code, source ENIS3, CloudWatch Logs, FirehoseDNS failures, malicious lookups
Task 3.2 · CloudWatch metrics

The network metrics to alarm on

Default metrics most teams forget, by service.

ServiceMetricAlarm when
NAT gatewayErrorPortAllocation, PacketsDropCount, IdleTimeoutCount> 0: port exhaustion (55,000 connections per unique destination), drops
Transit GatewayPacketDropCountBlackhole, PacketDropCountNoRoute, BytesIn/OutDrops > 0 after a routing change
Site-to-Site VPNTunnelState, TunnelDataIn, TunnelDataOutTunnelState < 1 (per tunnel)
Direct ConnectConnectionState, ConnectionLightLevelRx/Tx, ConnectionErrorCount, ConnectionBpsEgressState 0, light level out of range, CRC errors
ALBHTTPCode_ELB_5XX_Count, TargetResponseTime, RejectedConnectionCountELB 5xx vs target 5xx tells LB vs app
NLBTCP_Target_Reset_Count, TCP_ELB_Reset_Count, UnHealthyHostCount, PortAllocationErrorCountResets or port allocation errors > 0
EC2 (ENA, via ethtool -S)bw_in/out_allowance_exceeded, pps_allowance_exceeded, conntrack_allowance_exceeded, linklocal_allowance_exceededAny increase: instance network limits hit
Task 1.4 · Network monitors and baselines

CloudWatch Network Monitoring and performance baselines

Three monitors with different vantage points. Capture a baseline before you need it.

Internet Monitor

  • Internet-side availability and performance for your users, by city, ASN and ISP.
  • No agent: uses AWS’s own internet measurements for traffic to your VPCs, CloudFront, WorkSpaces.
  • Health events when an ISP problem affects you.
  • Answers: is it the internet or us?

Network Flow Monitor

  • Launched Dec 2024. eBPF agent on EC2 and EKS nodes; reads TCP stats, never payload.
  • Metrics: data transferred, retransmissions, retransmission timeouts, RTT.
  • Network health indicator: shows if AWS infrastructure caused the degradation.
  • Answers: is it the AWS network or our app?

Network Synthetic Monitor

  • Active probes from your subnets to on-premises IPs (ICMP or TCP).
  • Measures latency and packet loss over Direct Connect and VPN.
  • No agent on the destination.
  • Answers: is the hybrid path degrading?

Baseline (Task 1.4): record normal RTT, loss, tunnel throughput, NAT connections and TGW bytes per hour for a few weeks; use CloudWatch anomaly detection alarms instead of fixed thresholds.

Chapter 8 · Part 8.2

Analysis tools

Reachability Analyzer, Network Access Analyzer, and Network Manager with Route Analyzer.

Task 3.2 · Reachability Analyzer

Prove whether a path works without sending a packet

A configuration analysis of routes, SGs, NACLs, gateways and load balancers between a source and a destination.

# 1) Define the path: IGW -> instance on TCP 22
aws ec2 create-network-insights-path \
  --source igw-0797cccdc9d73b0e5 \
  --destination i-0495d385ad28331c7 --protocol TCP \
  --filter-at-source '{"DestinationPortRange":
      {"FromPort":22,"ToPort":22}}'

# 2) Run the analysis (takes a few minutes)
aws ec2 start-network-insights-analysis \
  --network-insights-path-id nip-0b26f224f1d131fa8

# 3) Read the verdict
aws ec2 describe-network-insights-analyses \
  --network-insights-analysis-ids nia-0abc123def456789 \
  --query 'NetworkInsightsAnalyses[0].[NetworkPathFound,
      Explanations[0].ExplanationCode]'
# [false, "ENI_SG_RULES_MISMATCH"]

What to know

  • Output: NetworkPathFound, ForwardPathComponents, ReturnPathComponents, Explanations.
  • Explanation codes name the blocker, for example ENI_SG_RULES_MISMATCH.
  • Destination can be a resource or just an IP address.
  • Works across accounts in an organization with a delegated administrator.
  • Charged per analysis; analyses are deleted after 120 days.
  • Automate: re-run saved paths after every change (CI or EventBridge) to verify intent.
Task 4.2 · Network Access Analyzer

Find paths that should not exist

Reachability Analyzer checks one path. Network Access Analyzer searches for every path matching a pattern.

{
  "MatchPaths": [{
    "Source": { "ResourceStatement": {
      "ResourceTypes": ["AWS::EC2::InternetGateway"] } },
    "Destination": {
      "ResourceStatement": {
        "ResourceTypes": ["AWS::EC2::NetworkInterface"] },
      "PacketHeaderStatement": {
        "DestinationPorts": ["3306", "5432"] } }
  }],
  "ExcludePaths": [{
    "ThroughResources": [{ "ResourceStatement": {
      "ResourceTypes": ["AWS::NetworkFirewall::Firewall"] } }]
  }]
}

How it works

  • Define a Network Access Scope: match paths, exclude paths.
  • Static analysis with automated reasoning; no traffic sent.
  • Findings list each path with every hop.
  • Example intent: "no internet path to DB ports unless it passes the firewall".
  • IPv4, TCP/UDP only; one account and Region per analysis.
  • Does not evaluate Network Firewall rules, and does not cover Direct Connect or Global Accelerator.
Task 3.2 · Network Manager

Topology, events and Route Analyzer

AWS Network Manager is the console for global networks built on Transit Gateway and Cloud WAN.

Global network

  • Register Transit Gateways from many Regions and accounts.
  • Add on-premises sites, devices and links (or SD-WAN partner integrations).
  • Topology tree and geographic map views.
  • Dashboards for TGW, VPN and Connect metrics in one place.

Route Analyzer

  • Analyzes TGW route tables only: not VPC route tables, SGs, NACLs or your routers.
  • Source: TGW + attachment + IP; destination: IP + TGW + attachment.
  • Can model a middlebox (inspection VPC) as extra hops.
  • Optional return-path analysis.
  • TGWs must be registered in the global network; intra-Region peering not supported.

Events

  • Topology changes, routing changes, status updates (tunnel down, BGP).
  • Sent to CloudWatch Logs and EventBridge (Network Manager events are delivered in us-west-2).
  • Trigger SNS / Lambda on "VPN tunnel down" or "route removed".
Exam traps and scenarios

Visibility and analysis

Scenario → answer → why.

ScenarioAnswerWhy
Need the payload of suspicious traffic for an IDSVPC Traffic Mirroring to an NLB in front of IDS sensorsFlow logs have no payload
Flow logs show the NAT gateway IP, not the instanceCustom format with pkt-srcaddrsrcaddr is the intermediate hop
Validate before go-live that the app subnet can reach RDS on 5432Reachability Analyzer pathConfiguration proof without traffic
Continuously prove no database is reachable from the internetNetwork Access Analyzer scope, scheduledSearches all paths, not one
A TGW route change dropped traffic between two VPCs; find which tableNetwork Manager Route AnalyzerIt walks TGW route tables
DNS lookups are failing, but flow logs show nothingResolver query logsFlow logs do not capture Amazon DNS traffic
Alert within minutes when a VPN tunnel goes downCloudWatch alarm on TunnelState (or Network Manager event in EventBridge)Native metric per tunnel
Chapter 8 · Part 8.3

Troubleshooting playbook

One decision tree and six real failure patterns: symptom → checks → root cause → fix.

Task 3.2 · Decision tree

Work from the name down to the application

Each step has a tool. Stop at the first step that fails.

1 · DNSdig, query logs2 · Route outsubnet RT, TGW RT3 · Route backreturn path, symmetry4 · SG and NACLboth ENIs, both ways5 · FirewallNFW, DNS FW, policy6 · Policiesendpoint, resource, IAM7 · MTUlarge packets only8 · OS and applistening? iptables?Fixed → verifyReachability Analyzer + flow logs ACCEPT both waysWrong IP or NXDOMAIN → Chapter 5No route / blackhole → Reachability, Route AnalyzerFlow logs: REJECT → SG/NACL; nothing → routeAccessDenied → policy,not networkHandshake OK,transfer hangsConnection refused = reached host

Timeouts usually mean a route or a filter silently dropped the packet. "Connection refused" means the packet reached the host and nothing listens. Learn to tell them apart first.

Case study 1

A private subnet cannot reach the internet

Symptom: yum update and curl https://… time out from instances in a private subnet.

Checks

  • Private subnet RT: 0.0.0.0/0 → nat-…?
  • NAT gateway in a public subnet whose RT has 0.0.0.0/0 → igw-…?
  • NAT state available, public NAT has an EIP?
  • SG egress, NACL outbound 443 and inbound 1024–65535.
  • NAT metrics: ErrorPortAllocation, PacketsDropCount.
→

Common root causes

  • NAT gateway placed in the private subnet.
  • Custom NACL without inbound ephemeral ports.
  • Route points to a NAT in another AZ that was deleted (blackhole).
  • IPv6-only workload: needs an egress-only IGW, or DNS64 + NAT64.
→

Fix and verify

  • Move or recreate the NAT in a public subnet; one per AZ.
  • Add NACL rules for return traffic.
  • Reachability Analyzer: instance → IGW on 443.
  • Flow logs on the NAT ENI show egress 8.
Case study 2

On-premises cannot resolve a private hosted zone

Symptom: app.internal.example.com resolves in the VPC but returns NXDOMAIN from the data center.

Checks

  • Is there a Resolver inbound endpoint in the VPC the zone is associated with?
  • On-prem DNS conditional forwarder for internal.example.com → inbound endpoint IPs.
  • Inbound endpoint SG allows UDP and TCP 53 from on-prem resolvers.
  • Route from on-prem to the endpoint subnets over VPN/DX.
  • Query logs on the VPC: does the query even arrive?
→

Common root causes

  • Forwarder points at VPC+2 (10.0.0.2): not reachable from outside the VPC.
  • Private hosted zone not associated with the endpoint’s VPC.
  • TCP 53 blocked: large responses fail.
  • Endpoint subnet CIDR not advertised to on-premises.
→

Fix and verify

  • Create inbound endpoint (2+ AZs) in a shared-services VPC; associate the zone.
  • Point on-prem forwarders at the endpoint IPs.
  • dig @<endpoint-ip> app.internal.example.com from on-prem.
  • Watch InboundQueryVolume on the endpoint.
Case study 3

Asymmetric routing through an inspection VPC

Symptom: after adding a central firewall, some VPC-to-VPC connections hang, others work; failures vary by instance.

Checks

  • Do failing pairs sit in different AZs?
  • Firewall alert / flow logs: replies dropped as out-of-state, or missing SYN?
  • Inspection VPC attachment: ApplianceModeSupport?
  • Firewall subnet route tables send traffic back to the TGW, not elsewhere.
  • Each AZ of the inspection VPC has a firewall endpoint.
→

Root cause

  • TGW keeps traffic in the sender’s AZ.
  • Request crosses the endpoint in AZ a; reply crosses the endpoint in AZ c.
  • The stateful engine in AZ c never saw the handshake and drops the reply.
  • Same-AZ pairs work, which hides the problem in testing.
→

Fix and verify

  • Enable appliance mode on the inspection VPC attachment only.
  • Or use the native Network Firewall TGW attachment.
  • Retest cross-AZ pairs; firewall flow logs show both directions on one endpoint.
  • Add a cross-AZ test to the change checklist.
Case study 4

MTU black hole over a VPN

Symptom: SSH works and small API calls succeed over Site-to-Site VPN, but file transfers and large responses hang.

Checks

  • Test with DF set: ping -M do -s 1472 <host> (Linux) then lower the size.
  • Is ICMP "fragmentation needed" (type 3 code 4) allowed in SGs and NACLs?
  • Instance MTU 9001 (ip link) sending into a 1,446-byte tunnel?
  • Customer gateway TCP MSS clamping configured?
  • Flow logs show ACCEPT: it is not a filter problem.
→

Root cause

  • Inside a Site-to-Site VPN tunnel the maximum packet is 1,446 bytes / MSS 1,406 (AES-GCM, no NAT-T), less with AES-CBC or NAT-T; no jumbo frames, no PMTUD.
  • Jumbo-frame instances send 9001-byte packets inside the VPC.
  • Path MTU discovery fails because ICMP is blocked somewhere.
  • The TCP handshake is small, so the connection "works" until data flows.
→

Fix and verify

  • Clamp TCP MSS on the customer gateway to ≤1,406 (1,382 with AES-CBC + SHA-256 + NAT-T).
  • Allow ICMP type 3 code 4 (and ICMPv6 packet too big).
  • Or set a per-route MTU for on-prem prefixes (ip route … mtu 1422 for CBC + NAT-T).
  • tracepath <host> shows the discovered path MTU.
Case study 5

BGP session down on Direct Connect

Symptom: on-prem routes disappear from TGW route tables; traffic fails over to VPN or stops.

Checks

  • ConnectionState up? Light levels ConnectionLightLevelRx/Tx in range (physical layer)?
  • VIF state and BGP status in the console (down vs idle).
  • VLAN, peer IPs and ASN match the VIF on your router.
  • BGP MD5 key matches; BFD settings.
  • Number of prefixes advertised on the VIF (100 per private / transit VIF).
→

Common root causes

  • Optic or cross-connect failure: light level out of range, CRC errors.
  • Router change replaced the MD5 key or VLAN.
  • More than 100 prefixes advertised: AWS puts the session into idle.
  • Maintenance on the DX device (notification in AWS Health).
→

Fix and verify

  • Physical: work with the colocation provider; check LOA-CFA port.
  • Logical: correct VLAN / ASN / key; summarize routes below the limit.
  • Design: second connection at a second location; VPN as backup.
  • Alarm on ConnectionState and VIF BGP status; test with the Resiliency Toolkit failover test.
Case study 6

IP address exhaustion in an EKS subnet

Symptom: new pods stay Pending / ContainerCreating with "failed to assign an IP address"; nodes do not scale.

Checks

  • Subnet AvailableIpAddressCount in the console or CLI.
  • VPC CNI settings: WARM_ENI_TARGET, WARM_IP_TARGET, MINIMUM_IP_TARGET.
  • Prefix delegation on (ENABLE_PREFIX_DELEGATION)?
  • Are many IPs held by warm pools on idle nodes?
  • IPAM utilization for the VPC CIDR.
→

Root cause

  • Each pod takes a VPC IP; warm pools pre-allocate more.
  • Small /24 subnets per AZ fill quickly at scale.
  • Prefix delegation needs contiguous /28 blocks; fragmented subnets cannot supply them.
  • No room left in the primary VPC CIDR.
→

Fix and verify

  • Add a secondary CIDR (e.g. 100.64.0.0/16) and new subnets for pods (custom networking).
  • Tune warm targets; enable prefix delegation on new subnets.
  • Consider IPv6 for pods.
  • Alarm on free IPs per subnet with a scheduled Lambda or IPAM metrics.
Self-check

Monitoring and troubleshooting: can you answer these?

Explain the answer and name the exact tool.

Flow logs

  • Name five traffic types flow logs do not capture.
  • Which field reveals the real source behind a NAT gateway?
  • What does traffic-path 8 mean, and what does 7 mean?
  • Why can a flow log not be edited after creation?
  • How do you decide between CloudWatch Logs and S3 as a destination?

Tools

  • Reachability Analyzer vs Network Access Analyzer: one sentence each.
  • What does Route Analyzer not look at?
  • Which port and encapsulation does Traffic Mirroring use?
  • How many mirror sessions can one ENI be a source for?
  • Which monitor tells you an ISP is the problem?

Playbook

  • Timeout vs connection refused: what does each prove?
  • Why do same-AZ flows work but cross-AZ flows fail through a central firewall?
  • Why does SSH work but SCP hang over a VPN?
  • Which NAT gateway metric shows port exhaustion?
  • Two ways to free IP space for EKS pods?
Chapter 9

Automation and IaC

Tasks 2.4, 4.1 — repeatable network configuration, no hardcoded values, change safety, and event-driven operations.

Chapter 9 · Part 9.1

Network as code

Build VPCs, Transit Gateways and DNS from code, allocate addresses from IPAM, and prove every change before it ships.

Task 2.4 · Why code

Networks drift fastest when they are built by hand

The exam asks for "repeatable", "consistent across accounts" and "least operational overhead": that is infrastructure as code.

What goes wrong by hand

  • Two VPCs get the same CIDR because nobody checked a spreadsheet.
  • A route is added in prod but not in DR.
  • A security group is opened "temporarily" and never closed.
  • Nobody can say who changed a route table, or why.

What code gives you

  • Review before change: a plan or change set shows the diff.
  • The same module stamps out dev, staging and prod.
  • Git history is the audit trail for intent; CloudTrail is the audit trail for API calls.
  • Drift detection finds console edits.

Network-specific risks

  • Replacing a VPC, subnet or TGW attachment is an outage, not an update.
  • Ordering matters: routes need attachments, attachments need subnets.
  • Cross-account pieces (RAM shares, attachment acceptance) need two providers or two stacks.
  • Deleting a subnet fails while any ENI is still in it.
Task 2.4 · Tools

CloudFormation, CDK, Terraform, CLI and SDK

All are in scope. Pick by team skills and by how many accounts and Regions you deploy to.

ToolModelStrengths for networkingWatch out for
AWS CloudFormationDeclarative JSON/YAML; state kept by AWSStackSets to every account/Region in an OU; change sets; drift detectionReplacement on some property changes; verbose templates
AWS CDKTypeScript/Python code that synthesizes CloudFormationHigh-level ec2.Vpc construct, loops, unit testsDefaults hide resources (NAT per AZ); review cdk diff
TerraformDeclarative HCL; state in S3 (lock file or DynamoDB)One tool for AWS, on-prem routers and SD-WAN vendors; plan shows replacementsState is yours to secure; provider version drift
AWS CLI / SDKImperative callsGlue, one-off fixes, Lambda automation, testsNo state, no plan; easy to make snowflakes
Ansible / vendor APIsImperative or declarative device configCustomer gateway and firewall config from AWS-generated VPN filesKeep it in the same pipeline as the AWS side

Hybrid automation: aws ec2 get-vpn-connection-device-sample-configuration returns a vendor-specific CGW config that a pipeline can template and push to the on-premises router.

Task 2.4 · Layout

A repository layout for a network account

Separate what changes rarely (core network) from what changes often (workload attachments), and keep state per account and Region.

network-iac/
├── modules/
│   ├── vpc/            # subnets per AZ, route tables, endpoints, flow logs
│   ├── tgw/            # TGW, route tables, RAM share
│   ├── tgw-attachment/ # spoke attachment + association/propagation
│   ├── dns/            # PHZs, Resolver endpoints, rules, RAM share
│   └── inspection/     # Network Firewall, appliance mode, routes
├── live/
│   ├── network-prod/ap-northeast-1/   # core: TGW, inspection, DNS
│   ├── workload-a-prod/ap-northeast-1/
│   └── workload-a-dev/ap-northeast-1/
└── policies/           # OPA/Checkov rules: no 0.0.0.0/0 SSH, flow logs on

Rules that keep it safe

  • One state file per account per Region per layer: a bad apply cannot touch everything.
  • Core layer outputs (TGW ID, PHZ IDs, prefix list IDs) are read by workload layers from SSM Parameter Store or remote state.
  • Pin module and provider versions.
  • Run plans in CI with a read-only role; apply with a separate deploy role.
  • Tag everything with owner, environment and cost center at the provider level.
Task 2.4 · No hardcoding

Values that must never be hardcoded

Hardcoded values work in one account and break in the next. The exam calls this out explicitly in Task 2.4.

Hardcoded valueWhy it breaksUse instead
CIDR blocksOverlaps appear as soon as a second team copies the templateIPAM pool allocation (ipv4_ipam_pool_id + netmask length)
AZ names (ap-northeast-1a)AZ names map to different physical zones per account; some accounts lack a zoneAZ IDs (apne1-az1) or data.aws_availability_zones / Fn::GetAZs
Subnet mathOff-by-one errors and overlapscidrsubnet() in Terraform, Fn::Cidr in CloudFormation
Resource IDs (TGW, PHZ, prefix list)IDs differ per account and RegionSSM parameters, remote state, data lookups, CloudFormation exports
Account IDs and ARNsBreak when promoted between accountsdata.aws_caller_identity, AWS::AccountId, AWS::Partition
Partner or SaaS IP listsThey change without noticeCustomer-managed prefix lists updated by automation
AMI IDs for appliancesRegion-specific, deprecated over timeSSM public parameters (/aws/service/...)
Task 2.4 · IPAM in Terraform

Allocate VPC CIDRs from IPAM instead of a spreadsheet

The network account owns the pools; workload accounts ask for "a /20" and IPAM guarantees no overlap.

# network account: IPAM, top-level pool, regional pool
resource "aws_vpc_ipam" "main" {
  operating_regions { region_name = "ap-northeast-1" }
}
resource "aws_vpc_ipam_pool" "apne1" {
  address_family = "ipv4"
  ipam_scope_id  = aws_vpc_ipam.main.private_default_scope_id
  locale         = "ap-northeast-1"
  allocation_default_netmask_length = 20
  allocation_min_netmask_length     = 16
  allocation_max_netmask_length     = 24
}
resource "aws_vpc_ipam_pool_cidr" "apne1" {
  ipam_pool_id = aws_vpc_ipam_pool.apne1.id
  cidr         = "10.64.0.0/12"
}
# share the pool with the organization via RAM (not shown)

# workload account: no CIDR literal anywhere
resource "aws_vpc" "app" {
  ipv4_ipam_pool_id   = var.ipam_pool_id   # from SSM / remote state
  ipv4_netmask_length = 20
}
locals {
  private = [for i in range(3) : cidrsubnet(aws_vpc.app.cidr_block, 4, i)]
}

What to notice

  • locale ties the pool to a Region; a VPC can only draw from a pool in its own Region.
  • Allocation rules (min/max/default netmask, required tags) stop a team from taking a /8.
  • IPAM tracks utilization and flags overlapping or unmanaged CIDRs across the org.
  • Subnets are derived with cidrsubnet(), so the VPC CIDR is the only input.
  • Use AZ IDs for subnets in multi-account designs so "az1" means the same zone everywhere.
Task 2.4 · CloudFormation and CDK

The same idea in CloudFormation and CDK

Both read the pool from a parameter and derive subnets, so the template is identical in every account.

# CloudFormation (YAML)
Parameters:
  IpamPoolId:
    Type: AWS::SSM::Parameter::Value<String>
    Default: /network/ipam/apne1-pool-id
Resources:
  Vpc:
    Type: AWS::EC2::VPC
    Properties:
      Ipv4IpamPoolId: !Ref IpamPoolId
      Ipv4NetmaskLength: 20
  SubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref Vpc
      AvailabilityZone: !Select [0, !GetAZs ""]
      CidrBlock: !Select [0, !Cidr [!GetAtt Vpc.CidrBlock, 6, 8]]
// CDK (TypeScript)
const poolId = ssm.StringParameter.valueForStringParameter(
  this, '/network/ipam/apne1-pool-id');

const vpc = new ec2.Vpc(this, 'App', {
  ipAddresses: ec2.IpAddresses.awsIpamAllocation({
    ipv4IpamPoolId: poolId,
    ipv4NetmaskLength: 20,
    defaultSubnetIpv4NetmaskLength: 24,
  }),
  maxAzs: 3,
  natGateways: 1,          // default is one per AZ: cost!
  subnetConfiguration: [
    { name: 'public',  subnetType: ec2.SubnetType.PUBLIC },
    { name: 'private', subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS },
  ],
});

!Cidr [ipBlock, count, cidrBits] uses host bits: 8 bits = /24 subnets. In CDK, check cdk diff before deploy: the Vpc construct creates NAT gateways, route tables and an IGW for you.

Task 2.4 · Multi-account

Deploying network pieces across accounts

Most real designs span a network account and many workload accounts. Automate both sides of every handshake.

Share once, consume everywhere

  • RAM share the TGW, subnets, Resolver rules, prefix lists and IPAM pools to the organization or an OU.
  • With sharing to the organization enabled, no invitation acceptance is needed.
  • Turn on TGW auto-accept for shared attachments only if route table association is also automated.

StackSets and pipelines

  • CloudFormation StackSets with service-managed permissions deploy a baseline (flow logs, endpoints, Resolver rule associations) to every new account automatically.
  • Terraform: one pipeline per account with an assumed deploy role; the core layer runs first.
  • Account Factory (Control Tower / AFT) can call these on account creation.

Handshakes to automate

  • TGW attachment → associate + propagate in the right TGW route table (network account).
  • PHZ association across accounts: create-vpc-association-authorization then associate-vpc-with-hosted-zone.
  • PrivateLink: allowed principals on the service, acceptance of endpoint connections.
  • VPN: generate the CGW config and push it to the device.
Task 2.4 · Change safety

Prove the change before and after it ships

A network change is safe when you know what it replaces and you can prove reachability did not regress.

1 · Plan

  • terraform plan / change set.
  • Block on "must be replaced" for VPCs, subnets, TGW attachments.
  • Policy-as-code: Checkov, OPA, cfn-guard.
→

2 · Review

  • Two-person review for core layers.
  • Diff of route tables and SG rules in the PR.
  • Change window for hybrid routing.
→

3 · Apply

  • Deploy role, not a human.
  • Small blast radius: one layer, one account.
  • Stagger Regions.
→

4 · Verify

  • Reachability Analyzer paths as tests.
  • Network Access Analyzer scopes ("no internet path to DB").
  • Synthetic checks and alarms.
→

5 · Detect drift

  • CloudFormation drift detection.
  • terraform plan -refresh-only on a schedule.
  • Config rules on SGs and flow logs.
Task 3.2 · Intent as a test

Reachability Analyzer as a CI check

Encode "app can reach DB on 5432" and "DB cannot reach the internet" as resources; fail the pipeline when the answer changes.

resource "aws_ec2_network_insights_path" "app_to_db" {
  source           = aws_instance.app.primary_network_interface_id
  destination      = var.db_eni_id  # the DB's ENI ID
  protocol         = "tcp"
  destination_port = 5432
}

resource "aws_ec2_network_insights_analysis" "app_to_db" {
  network_insights_path_id = aws_ec2_network_insights_path.app_to_db.id
  wait_for_completion      = true

  lifecycle {
    replace_triggered_by = [aws_security_group.db]
    postcondition {
      condition     = self.path_found
      error_message = "app -> db:5432 is no longer reachable"
    }
  }
}

How to use it

  • replace_triggered_by re-runs the analysis when the SG changes.
  • For "must NOT be reachable" intents, assert !self.path_found.
  • Reachability Analyzer checks configuration only: it sends no packets, so it cannot see OS firewalls or a down app.
  • Each analysis is billed; keep the list of intents short and meaningful.
  • From the CLI: create-network-insights-path → start-network-insights-analysis → read NetworkPathFound and Explanations.
Chapter 9 · Part 9.2

Event-driven network automation

React to network events with EventBridge, Lambda, Config and Systems Manager, and report incidents automatically.

Task 2.4 · Event sources

Where network events come from

EventBridge is the bus. Know which service emits which event and in which Region.

SourceExample eventsNotes
CloudTrail (API calls)AuthorizeSecurityGroupIngress, CreateRoute, DeleteFlowLogsManagement events; arrive as "AWS API Call via CloudTrail"
Network ManagerTopology change, routing update (route added/removed in TGW), VPN/Connect peer statusDelivered in us-west-2 (Network Manager home Region); TGW must be registered
CloudWatch alarmsTunnelState = 0 (AWS/VPN), ConnectionState (AWS/DX), NAT ErrorPortAllocationAlarm state change events, or alarm actions to SNS
AWS HealthDirect Connect maintenance, VPN tunnel endpoint replacementPlan failover before scheduled maintenance
AWS ConfigCompliance change for restricted-ssh, vpc-flow-logs-enabledPair with SSM Automation remediation
GuardDuty / Security HubPort probing, DNS exfiltration, unusual traffic findingsSecurity incident reporting (Task 4.1)
SNS AmazonIpSpaceChangedAWS published ip-ranges.json changedTrigger prefix list or firewall updates
Task 4.1 · Auto-remediation

Close a world-open SSH rule within seconds

Detect the API call, then revoke the rule and notify. Prevention (SCPs, Firewall Manager) is better; this is the safety net.

{
  "source": ["aws.ec2"],
  "detail-type": ["AWS API Call via CloudTrail"],
  "detail": {
    "eventName": ["AuthorizeSecurityGroupIngress"],
    "requestParameters": {
      "ipPermissions": { "items": {
        "ipRanges": { "items": {
          "cidrIp": ["0.0.0.0/0"] } } } }
    }
  }
}
# Lambda (Python) target of the rule
import boto3
ec2 = boto3.client("ec2")

def handler(event, _):
    p = event["detail"]["requestParameters"]
    gid = p["groupId"]
    for perm in p["ipPermissions"]["items"]:
        if perm.get("fromPort") in (22, 3389):
            ec2.revoke_security_group_ingress(
                GroupId=gid,
                IpPermissions=[{
                    "IpProtocol": perm["ipProtocol"],
                    "FromPort": perm["fromPort"],
                    "ToPort": perm["toPort"],
                    "IpRanges": [{"CidrIp": "0.0.0.0/0"}]}])
    # then publish to SNS with the caller identity
    #   event["detail"]["userIdentity"]["arn"]

Managed alternative with less code: Config rule restricted-ssh + automatic remediation with the SSM document AWS-DisablePublicAccessForSecurityGroup. Org-wide: a Firewall Manager security group content audit policy.

Task 4.2 · Hybrid alarms

Alarm on the hybrid links before users notice

VPN and Direct Connect publish state metrics; alarm on them and route the alarm to people and to automation.

# VPN: alarm when any tunnel of the connection is down
aws cloudwatch put-metric-alarm \
  --alarm-name vpn-onprem-tunnel-down \
  --namespace AWS/VPN --metric-name TunnelState \
  --dimensions Name=VpnId,Value=vpn-0abc1234 \
  --statistic Minimum --period 60 --evaluation-periods 3 \
  --threshold 1 --comparison-operator LessThanThreshold \
  --treat-missing-data breaching \
  --alarm-actions arn:aws:sns:ap-northeast-1:111122223333:netops

# Direct Connect: connection down
aws cloudwatch put-metric-alarm \
  --alarm-name dx-primary-down \
  --namespace AWS/DX --metric-name ConnectionState \
  --dimensions Name=ConnectionId,Value=dxcon-abc123 \
  --statistic Minimum --period 60 --evaluation-periods 2 \
  --threshold 1 --comparison-operator LessThanThreshold \
  --alarm-actions arn:aws:sns:ap-northeast-1:111122223333:netops

Useful metrics

  • TunnelState (1 up, 0 down), TunnelDataIn/Out per tunnel.
  • ConnectionState, ConnectionBpsEgress/Ingress, ConnectionLightLevelRx (optics) for DX.
  • VirtualInterfaceBpsEgress per VIF to spot a failover.
  • NAT gateway ErrorPortAllocation, PacketsDropCount.
  • TGW PacketDropCountNoRoute, PacketDropCountBlackhole.
  • Use Minimum for state metrics so a single down sample counts.
Task 4.1 · Incident reporting

Automated security incident reporting and alerting

Turn findings into tickets and chat messages with no human copy-paste.

Detect

  • GuardDuty: port probing, DNS exfiltration, traffic to known-bad IPs.
  • Network Firewall alert logs.
  • Flow Logs metric filter on REJECT spikes.
→

Route

  • EventBridge rule on aws.guardduty with severity ≥ 7.
  • Security Hub aggregates across accounts.
  • Delegated admin account receives all findings.
→

Act

  • SNS → email / pager.
  • Amazon Q Developer in chat applications (formerly AWS Chatbot) → Slack or Teams.
  • Lambda / SSM Automation: isolate the ENI with a quarantine SG, add the IP to a Network Firewall block list.
→

Record

  • Ticket in the ITSM tool.
  • Evidence: flow logs, firewall logs, CloudTrail.
  • Post-incident: add a preventive control.

GuardDuty network findings use VPC Flow Logs, DNS logs and CloudTrail as foundational sources: you do not need to enable flow logs for GuardDuty to analyze them.

Task 2.4 · Keep lists current

Prefix lists and firewall rules that update themselves

IP lists for SaaS partners, CloudFront origins or on-premises ranges change. Automate the update, not the rule.

# subscribe a Lambda to AWS IP range changes (topic is in us-east-1)
aws sns subscribe --region us-east-1 --protocol lambda \
  --topic-arn \
    arn:aws:sns:us-east-1:806199016981:AmazonIpSpaceChanged \
  --notification-endpoint \
    arn:aws:lambda:us-east-1:111122223333:function:sync-ranges

# the Lambda rewrites a customer-managed prefix list
aws ec2 get-managed-prefix-list-entries \
  --prefix-list-id pl-0abc --query 'Entries'
aws ec2 modify-managed-prefix-list --prefix-list-id pl-0abc \
  --current-version 7 \
  --add-entries Cidr=203.0.113.0/24,Description=partner-a \
  --remove-entries Cidr=198.51.100.0/24

Why prefix lists

  • Security groups and route tables reference the list, so one update changes every rule.
  • --current-version gives optimistic locking: concurrent writers fail instead of overwriting.
  • Share the list to the organization with RAM.
  • The list size is its max entries, and that whole size counts against SG rule and route quotas.
  • For AWS services, prefer AWS-managed lists such as com.amazonaws.global.cloudfront.origin-facing.
Task 2.4 · Cost through automation

Eliminate risk and cost with automation

Task 2.4 asks for efficiency "while maintaining the lowest possible cost". Automation is how you get there at scale.

Shut down what is idle

  • Scheduled teardown of non-prod NAT gateways and interface endpoints (EventBridge Scheduler + Lambda or pipeline).
  • Delete unattached Elastic IPs: every public IPv4 address is billed hourly.
  • Find idle load balancers and endpoints with Trusted Advisor checks.

Right-size by design

  • Modules default to one NAT per AZ only in prod; dev uses one.
  • Add S3 and DynamoDB gateway endpoints to every VPC by default: they are free.
  • Centralize interface endpoints in a shared-services VPC when many VPCs need the same services.

Guardrails in code

  • Policy-as-code blocks unencrypted, untagged or world-open resources before apply.
  • Required cost-allocation tags at the provider level.
  • Budgets and anomaly detection alerts per account.
Exam traps

Automation and IaC traps

Wrong answers that look right.

Hardcoded AZ names

  • us-east-1a in one account is not the same physical AZ as in another.
  • Use AZ IDs when accounts must line up (shared subnets, endpoint services).

Replace = outage

  • Changing a VPC CIDR or a subnet AZ in IaC forces replacement.
  • Add a secondary CIDR instead of changing the primary.

Custom code when native exists

  • Writing Lambda to enforce SGs org-wide when Firewall Manager does it.
  • Writing a CIDR tracker when IPAM exists.

Wrong Region for events

  • Network Manager events arrive in us-west-2.
  • Global Accelerator APIs are called in us-west-2; CloudFront and Route 53 metrics and alarms live in us-east-1.

Reachability ≠ connectivity test

  • Reachability Analyzer reads configuration; it cannot detect a stopped service or an OS firewall.

Drift

  • A console fix that IaC does not know about is undone by the next apply.
  • Detect drift on a schedule; import or codify the fix.
Exam-style scenarios

Automation scenarios

Scenario → answer → why.

ScenarioAnswerWhy
200 accounts each create VPCs; CIDRs keep overlappingIPAM pools shared with RAM; IaC allocates by netmask lengthIPAM guarantees unique allocations and tracks utilization
Every new account must get flow logs and S3/DynamoDB gateway endpointsCloudFormation StackSet with service-managed permissions and auto-deployment on the OUDeploys automatically to new accounts with no custom code
Ops must know within a minute when a VPN tunnel dropsCloudWatch alarm on TunnelState → SNSNative metric; no polling script
Pipeline must prove the DB is never reachable from the internetNetwork Access Analyzer scope (or Reachability Analyzer path) in CIEncodes intent; fails the build on regression
Partner IP list changes monthly; 30 SGs reference itCustomer-managed prefix list referenced by the SGs, updated by automationOne update, all rules follow
Template works in account A but fails in B with "subnet AZ not supported"Replace hardcoded AZ names with AZ IDs or GetAZsAZ names map per account
Self-check

Automation and IaC: can you answer these?

Explain the answer and name the exact feature.

Network as code

  • Name five values you should never hardcode and what replaces each.
  • How does an IPAM allocation rule stop a team taking too much space?
  • Which IaC changes force replacement of a VPC or subnet?
  • How do you split Terraform state for a network account?

Change safety

  • What can Reachability Analyzer prove, and what can it not?
  • How do you detect a console change to a route table?
  • What does --current-version protect on a prefix list update?
  • How do you automate cross-account PHZ association?

Events

  • Which Region receives Network Manager events?
  • Which metric and statistic alarm on a VPN tunnel going down?
  • Native vs custom remediation for a world-open SSH rule?
  • How do findings reach Slack without custom code?
Chapter 10

Performance, quotas and cost

Tasks 3.1, 3.3 — get the throughput you pay for, stay inside the quotas, and stop paying for bytes you do not need to move.

Chapter 10 · Part 10.1

Performance tuning

Instance bandwidth, per-flow limits, interface types, placement, MTU per path, and multicast.

Task 3.3 · Bandwidth model

What limits the throughput of one instance

The instance type sets the ceiling; the number of flows, the destination and the credit bucket decide what you actually get.

Per instance

  • "Up to 10 Gbps" sizes (usually 4xlarge and smaller) have a baseline plus burst from network I/O credits (typically 5–60 minutes).
  • Fixed sizes ("25 Gigabit") sustain that rate.
  • Limits apply separately inbound and outbound.
  • Read the baseline: describe-instance-types --query ...NetworkCards[0].BaselineBandwidthInGbps.

Per flow (5-tuple)

  • 5 Gbps outside a cluster placement group.
  • Up to 10 Gbps inside the same cluster placement group.
  • Up to 25 Gbps with ENA Express (same Region), capped by the instance limit.
  • For GRE/IPsec a flow is the 3-tuple (src, dst, protocol): one VPN tunnel = one flow.

Per destination

  • To another instance in the Region: full instance bandwidth.
  • Through an internet gateway: 5 Gbps for types with < 32 vCPUs, 50% of instance bandwidth above that.
  • Exceeding allowances drops packets: watch ENA bw_out_allowance_exceeded, pps_allowance_exceeded, conntrack_allowance_exceeded.

Throughput fix order: more flows (parallel streams, MPTCP) → cluster placement group or ENA Express → bigger instance → jumbo frames on paths that support them.

Task 3.3 · Interface types

ENI, ENA, ENA Express, EFA: pick the right one

Task 3.3 skill: "selecting the right network interface for the best performance".

InterfaceWhat it isUse whenLimits
ENIThe virtual NIC object: IPs, SGs, MAC, source/dest checkAlways; extra ENIs for dual-homed appliances or management networksNumber per instance depends on type
ENA (enhanced networking)SR-IOV driver; required on current-generation typesDefault for high throughput and low latencyPer-flow 5 Gbps outside a cluster PG
ENA ExpressENA + AWS SRD transport: multipath, congestion-awareSingle large flows (storage, replication, media) between supported instancesBoth sides enabled and supported types; no middleboxes in the path; slight latency increase when idle
EFAENA + OS-bypass interface (libfabric) for MPI and NCCLTightly coupled HPC and ML trainingSame subnet; SG must allow all traffic to and from itself; OS-bypass traffic is not routable
aws ec2 modify-network-interface-attribute \
  --network-interface-id eni-0abc1234 \
  --ena-srd-specification \
  'EnaSrdEnabled=true,EnaSrdUdpSpecification={EnaSrdUdpEnabled=true}'
ethtool -S ens5 | grep -E 'ena_srd|allowance'  # SRD counters, drops

Placement groups

  • Cluster: one AZ, low latency, 10 Gbps per flow.
  • Partition: separate racks per partition (HDFS, Kafka).
  • Spread: each instance on distinct hardware, max 7 per AZ.
Task 3.3 · MTU per path

Jumbo frames work only where every hop supports them

The smallest MTU on the path wins. Know these values cold.

PathMax MTUPMTUD
Inside a VPC, intra-Region peering9001Yes (allow ICMP)
Transit Gateway (VPC, DX, Connect, peering attachments)8500VPC and Connect attachments only
Inter-Region VPC peering8500—
VPC endpoints (PrivateLink), NAT gateway8500Endpoints: no; NAT: yes
Direct Connect private VIF9001 (opt-in)—
Direct Connect transit VIF8500 (opt-in)—
Internet gateway, public VIF, between Regions without peering or TGW1500—
Site-to-Site VPN tunnelMax 1446 / MSS 1406 (AES-GCM, no NAT-T); 1422 / 1382 with AES-CBC + SHA-256 + NAT-TNo

Configure and verify

  • Linux: ip link set dev ens5 mtu 9001 (persist in netplan or the ifcfg file).
  • Per-route MTU keeps internet traffic at 1500: ip route add 0.0.0.0/0 via 10.0.0.1 mtu 1500.
  • Test: tracepath <ip> or ping -M do -s 8972 <ip> (8972 + 28 = 9000).
  • Jumbo on a VIF: set MTU on the VIF; it briefly drops BGP while it changes.
  • MSS clamping happens on TGW, NAT gateway and endpoints, which saves most TCP flows; UDP and ICMP still break.
Task 3.2 · MTU black holes

How a packet size mismatch looks, and how to fix it

Task 3.2 skill: "troubleshooting packet size mismatches in a VPC to restore network connectivity".

Symptoms

  • TCP handshake works, small requests work, large responses hang.
  • SSH connects, scp stalls; TLS handshake hangs at certificate exchange.
  • Works over VPC peering, breaks after moving the same flow to TGW or VPN.
  • Flow Logs show ACCEPT both ways: nothing is "blocked".

Root causes

  • DF bit set and the ICMP "fragmentation needed" message is dropped by a NACL or appliance.
  • A hop that does not do PMTUD at all (VPN, PrivateLink endpoints).
  • Mixed MTU during a peering → TGW migration (9001 vs 8500): asymmetric drops.
  • Tunnel overhead (GRE, IPsec, GENEVE) not subtracted.

Fixes

  • Allow ICMP type 3 code 4 (IPv4) and ICMPv6 type 2 in NACLs and firewalls.
  • Lower the instance or route MTU to the smallest hop.
  • Clamp TCP MSS on the customer gateway (for example 1382 for AES-CBC + SHA2-256 with NAT-T).
  • Fragment before encryption on the CGW.
  • Migrate both VPCs of a peering to TGW in the same change.
Task 3.3 · Service throughput

Throughput of the managed network services

When one box is not enough, add more of them and spread the flows. These are the numbers that decide that.

ServiceThroughputScale-out pattern
NAT gateway5 Gbps scaling to 100 Gbps; 1 M scaling to 10 M pps; 55,000 connections per unique destination per IPAdd secondary IPs (up to 8 IPs per NAT gateway: 1 primary + 7 secondary), or one NAT per subnet
Interface / GWLB endpoint10 Gbps per AZ scaling to 100 GbpsSpread across AZs
Transit Gateway VPC attachmentUp to 100 Gbps per direction per AZ; up to 7.5 M ppsMultiple AZs, multiple attachments
Site-to-Site VPN tunnelUp to 1.25 Gbps / 140,000 pps (standard); up to 5 Gbps / 400,000 pps (large)ECMP across tunnels on TGW (dynamic routing only)
TGW Connect peer (GRE)Up to 5 Gbps, 4 peers per attachment (20 Gbps)ECMP across peers and attachments
Direct ConnectPort speed: 1, 10, 100 or 400 Gbps dedicated; hosted from 50 MbpsLAG (up to 4 links below 100G, 2 at 100G), more connections

A VPN over TGW with ECMP across 4 tunnels of 1.25 Gbps can reach ~5 Gbps aggregate, but any single flow still rides one tunnel.

Task 3.3 · Multicast

Multicast with Transit Gateway

The only way to do multicast in AWS. Task 3.3 skill: "implementing a multicast capability within a VPC and on-premises".

How it works

  • Create a new TGW with multicast enabled; create a multicast domain.
  • Associate subnets with the domain (a subnet joins one domain).
  • Membership: IGMPv2 (dynamic) or static via API; only static supports IPv6.
  • TGW sends IGMP queries from 0.0.0.0/32 to 224.0.0.1 every two minutes: SG and NACL must allow protocol 2.

Limits

  • Not over Direct Connect, VPN, peering or Connect attachments.
  • For on-premises: tunnel multicast inside GRE to an appliance, or re-source it in AWS.
  • 1 Gbps per flow, 20 Gbps aggregate per AZ.
  • No fragmentation of multicast packets.
  • Non-Nitro instances cannot be senders.

Unicast vs multicast vs CDN

  • Same data to many receivers inside AWS (market data, video in a facility): TGW multicast.
  • Same content to many internet users: CloudFront caching cuts origin bandwidth.
  • Few receivers: plain unicast is simpler.
Task 3.3 · Global performance

Shorten the internet part of the path

Users far from the Region suffer on the public internet. Put them on the AWS backbone as early as possible.

CloudFront

  • HTTP(S) and WebSocket; caches content at the edge.
  • Even uncached dynamic requests benefit: TLS ends near the user, persistent connections to the origin.
  • Origin Shield adds one regional cache layer to cut origin load.

Global Accelerator

  • TCP and UDP, no caching; two static anycast IPs.
  • Traffic enters the AWS backbone at the nearest edge.
  • Fast regional failover with health checks and traffic dials; no DNS TTL wait.
  • Endpoints: ALB, NLB, EC2, Elastic IP.

Accelerated VPN

  • Site-to-Site VPN on TGW through Global Accelerator edges.
  • Helps branch offices far from the Region.
  • Not available with large-bandwidth (5 Gbps) tunnels.

Exam keyword: "static IPs", "UDP", "non-HTTP", "fast failover without DNS" → Global Accelerator. "Cache", "static content", "HTTP" → CloudFront.

Exam traps

Performance traps

Wrong answers that look right.

Bigger instance for one flow

  • A single TCP stream stays at 5 Gbps across placement groups whatever the instance size.
  • Use parallel flows, a cluster PG or ENA Express.

Jumbo frames to the internet

  • The IGW forwards 1500 bytes; set per-route MTU instead of 9001 everywhere.

VPN ECMP needs BGP

  • Static-routed VPNs cannot use ECMP on TGW.
  • VGW never does ECMP across connections.

Blocking all ICMP

  • Breaks PMTUD and causes black holes on TGW and NAT paths.

EFA across subnets

  • EFA OS-bypass traffic is not routable; keep the cluster in one subnet.

Multicast over hybrid

  • TGW multicast does not cross DX or VPN attachments.
Chapter 10 · Part 10.2

Quotas that bite

Default quotas that shape designs, which ones can be raised, and how to watch them.

Task 3.1 · VPC quotas

VPC quotas that shape a design

Defaults per Region per account unless noted. "No" means you must design around it.

QuotaDefaultAdjustable
VPCs per Region5Yes (hundreds)
IPv4 CIDR blocks per VPC5Yes, up to 50
Routes per route table (non-propagated)500Yes, up to 1,000
Propagated routes per route table100No — advertise a summary or default route
Inbound or outbound rules per security group60 (IPv4 and IPv6 counted separately)Yes; rules × SGs per ENI ≤ 1,000
Security groups per ENI5Yes, up to 16
Rules per network ACL (each direction)20Yes, up to 40
Active peering connections per VPC50Yes, up to 125
Interface + GWLB endpoints per VPC50Yes
Network Address Usage units per VPC64,000Yes, up to 256,000
Task 3.1 · Hybrid quotas

Route limits on hybrid connections

Exceeding a BGP prefix limit does not drop a few routes: on Direct Connect the session goes down.

QuotaValueAdjustable / what happens
Prefixes from on-premises per private or transit VIF BGP session100 each for IPv4 and IPv6Up to 1,000 with prefix controls; over the limit the session goes idle (DOWN)
Prefixes per public VIF BGP session1,000No
Prefixes per TGW from AWS to on-premises on a transit VIF200 (IPv4 + IPv6 combined)Contact your SA/TAM — summarize VPC CIDRs in allowed prefixes
Dynamic routes from a CGW to a VPN on a VGW100No
Dynamic routes from a CGW to a VPN on a TGW1,000No
Total routes across all route tables of one TGW10,000Contact your SA/TAM
VIFs per dedicated connection (private/public + transit)51 total, of which up to 4 transitNo
VGWs / TGWs per Direct Connect gateway20 / 6No

Design reflex: summarize. Allocate each Region or environment a contiguous block (IPAM) so it can be advertised as one prefix.

Task 3.1 · DNS and NAT quotas

Quotas that cause intermittent failures

These do not fail a deployment; they fail some requests under load, which is harder to find.

Link-local services

  • 1,024 packets per second per ENI to the VPC Resolver (VPC+2, 169.254.169.253), IMDS and Time Sync.
  • Cannot be raised.
  • Symptom: random DNS timeouts on busy hosts.
  • Fix: local DNS cache (systemd-resolved, dnsmasq, NodeLocal DNSCache on EKS).

Resolver endpoints

  • Up to 10,000 UDP queries per second per endpoint IP; can be as low as 1,500 when connections are tracked (restrictive SG rules, NLB in front).
  • 6 IPs per endpoint and 4 endpoints per Region by default (adjustable).
  • Add IPs when an ENI exceeds ~50% of capacity.

NAT gateway ports

  • 55,000 simultaneous connections per unique destination (IP, port, protocol) per NAT IP.
  • Symptom: ErrorPortAllocation > 0, failed connections to one popular endpoint.
  • Fix: add secondary IPs (up to 8 IPs per NAT gateway: 1 primary + 7 secondary), split subnets across NAT gateways, use VPC endpoints for AWS APIs.
Task 3.1 · Watch quotas

Monitor quotas before they fail a change

Service Quotas exposes current values and usage; CloudWatch alarms on usage turn quotas into normal monitoring.

# current value of a quota (SG rules per SG)
aws service-quotas get-service-quota \
  --service-code vpc --quota-code L-0EA8095F

# list all VPC quotas with their values
aws service-quotas list-service-quotas --service-code vpc \
  --query 'Quotas[].[QuotaName,Value,Adjustable]' --output table

# request an increase
aws service-quotas request-service-quota-increase \
  --service-code vpc --quota-code L-0EA8095F --desired-value 100

# Network Address Usage of a VPC (must be enabled per VPC)
aws ec2 modify-vpc-attribute --vpc-id vpc-0abc \
  --enable-network-address-usage-metrics

Habits

  • Check quotas in every new account and Region before a migration wave.
  • Use Service Quotas templates in Organizations so new accounts start with raised values.
  • Trusted Advisor service-limit checks flag usage above 80%.
  • A prefix list counts at its max entries size against SG and route quotas.
  • Some network quotas are "contact your SA/TAM", not Service Quotas: plan weeks ahead.
Chapter 10 · Part 10.3

Data transfer cost

Where bytes cost money in AWS networking, and the design patterns that remove those charges.

Task 3.3 · Cost map

Where a byte leaving an instance gets billed

Relative, not absolute: check the pricing pages for your Region. Inbound from the internet is free.

RegionEC2, same AZfree via private IPEC2 in AZ asourceEC2 in AZ bbilled both directionsNAT gatewayhourly + per GBS3 gateway endpointfreeInterface endpointhourly per AZ + per GBTransit Gatewayper attachment-hour + per GBDX gateway / VIFport-hours + DTO rateInternetdata transfer out: highest per-GB rateAmazon S3, same Regionno data transfer chargeAWS service APIsprivately, without NAT processingOther VPC / other Regioninter-AZ or inter-Region per GBOn-premiseslower per-GB rate than internet

Charges stack: a byte from a private subnet to the internet pays NAT processing and internet data transfer out; through a centralized egress VPC it also pays TGW processing.

Task 3.3 · Cost by service

How each network service is billed

Know the billing dimensions; the exam asks for "most cost-effective", not exact prices.

ServiceBilled onCost lever
Data transferPer GB: internet out, inter-Region, inter-AZ (each direction); same AZ via private IP freeKeep chatty traffic in one AZ; cache at the edge
Public IPv4 addressesPer hour for every public IPv4 address, attached or not (since Feb 2024)IPv6, fewer public IPs, release idle EIPs
NAT gatewayPer hour per gateway + per GB processedGateway endpoints for S3/DynamoDB; interface endpoints for heavy AWS API use
VPC peeringNo hourly or processing charge; normal inter-AZ / inter-Region transferCheapest for a few high-volume VPC pairs
Transit GatewayPer attachment-hour + per GB processed (sender side by default; flexible allocation since Nov 2025)Peer the heaviest pairs directly
PrivateLinkPer endpoint per AZ per hour + per GB processedShare endpoints from a central VPC
Direct ConnectPort-hours + data transfer out at DX rates; inbound freeSteady high volume; hosted connection for smaller needs
Site-to-Site VPNPer connection-hour + normal data transfer outCheap for low volume and backup paths
Task 3.3 · Patterns

Design patterns that cut network cost

Most savings come from removing a hop or keeping traffic in one AZ.

Remove NAT processing

  • S3 and DynamoDB gateway endpoints in every VPC (free).
  • Interface endpoints for high-volume AWS APIs (ECR, CloudWatch Logs) when GB volume × NAT rate > endpoint hours.
  • Pull container images through an ECR pull-through cache in-Region.

Stay in the AZ

  • Route each AZ to its own NAT gateway (also an availability requirement).
  • Topology-aware routing in EKS; ALB cross-zone off where balance allows.
  • Use interface endpoint zonal DNS names for AZ-local calls.
  • Replicate data, not every request, across AZs.

Choose the right hub

  • Peering for a few heavy VPC pairs; TGW for many VPCs and transitive routing.
  • DX for steady volume to on-premises; VPN for burst and backup.
  • CloudFront in front of S3 and ALB: transfer from origin to CloudFront is free and the edge serves cached bytes.

Decide with data, not intuition: compare GB processed (CloudWatch BytesOutToDestination on NAT, BytesIn on TGW attachments) with each option's hourly cost.

Task 3.3 · Find the bytes

Find who is generating the bill

A network bill is a list of usage types. Map them back to flows.

Cost tools

  • Cost Explorer grouped by usage type (Region-prefixed): APN1-NatGateway-Bytes, APN1-DataTransfer-Regional-Bytes (inter-AZ), APN1-DataTransfer-Out-Bytes, APN1-TransitGateway-Bytes.
  • Cost allocation tags on NAT gateways, TGW attachments and endpoints.
  • Cost and Usage Report (CUR 2.0 / Data Exports) + Athena for per-resource detail.
  • Cost Explorer API calls are billed per request.

Flow tools

  • VPC Flow Logs with pkt-srcaddr, pkt-dstaddr, az-id, flow-direction and bytes → Athena: top talkers across AZs.
  • CloudWatch Network Flow Monitor: TCP flow performance and volume between workloads.
  • NAT gateway CloudWatch metrics per gateway.

Typical findings

  • A single NAT in one AZ serving all AZs (inter-AZ + NAT processing).
  • ECR image pulls and S3 traffic going through NAT.
  • Cross-AZ replication or chatty microservices.
  • Idle Elastic IPs and forgotten interface endpoints.
Exam traps

Quota and cost traps

Wrong answers that look right.

"Peering is always cheaper"

  • Per GB, yes; but a full mesh of 50 VPCs is 1,225 connections. Operability wins at scale.

Gateway endpoint for everything

  • Only S3 and DynamoDB have gateway endpoints; others need interface endpoints (billed).

Same AZ, public IP

  • Traffic between instances in the same AZ over public or Elastic IPs is billed; use private IPs.

Raising a hard quota

  • Propagated routes per route table (100), VGW VPN routes (100) and link-local 1,024 pps cannot be raised: redesign.

DX prefix overflow

  • More than the allowed prefixes on a private VIF takes the BGP session down, not just the extra routes.

Inbound is not free everywhere

  • Inbound from the internet is free, but NAT, TGW and PrivateLink still bill per GB processed in either direction.
Exam-style scenarios

Performance, quota and cost scenarios

Scenario → answer → why.

ScenarioAnswerWhy
Replication job between two instances is stuck at 5 Gbps on 25 Gbps instancesENA Express on both (or cluster PG + parallel streams)Single-flow limit, not instance limit
Large NAT gateway bill; workloads mostly read S3 in the same RegionS3 gateway endpoint on the private route tablesRemoves NAT processing; gateway endpoints are free
Random DNS timeouts on a busy proxy fleet; Resolver itself is healthyLocal DNS cache on each instance1,024 pps link-local limit per ENI
BGP over a private VIF keeps going down after a new site was addedSummarize prefixes or raise the limit with prefix controlsOver 100 prefixes takes the session idle
Need 4 Gbps to on-premises over VPNTGW VPN with BGP and ECMP across tunnels, or 5 Gbps large tunnelsStandard tunnel is 1.25 Gbps; ECMP needs dynamic routing
Market data feed must reach 40 receivers in 3 VPCsTGW multicast domainOnly native multicast option
Self-check

Performance, quotas and cost: can you answer these?

Explain the answer and name the exact number or feature.

Performance

  • What are the per-flow limits outside a PG, inside a cluster PG, and with ENA Express?
  • Give the MTU for IGW, TGW, inter-Region peering, private VIF and VPN.
  • Why does blocking ICMP break large transfers?
  • ENA Express vs EFA: when each?

Quotas

  • Which network quotas cannot be raised?
  • What happens when you advertise 101 prefixes on a private VIF?
  • How do prefix lists count against SG rules?
  • What does ErrorPortAllocation tell you?

Cost

  • Which is billed: same-AZ private IP, same-AZ public IP, inter-AZ?
  • NAT gateway vs gateway endpoint vs interface endpoint: who pays what?
  • Peering vs TGW: when does each win?
  • Which usage types point to inter-AZ traffic?
Reference architecture 1

Single account, three tiers, two AZs

The baseline every other design builds on. Each tier has its own subnets and route tables.

InternetusersVPC 10.0.0.0/16 · one accountIGWAZ aAZ bALB nodepublic subnetNAT gatewaypublic subnetALB nodepublic subnetNAT gatewaypublic subnetApp tier (ASG / ECS / EKS)private subnet → NAT in same AZApp tier (ASG / ECS / EKS)private subnet → NAT in same AZRDS primaryisolated subnetS3 gateway endpointroute via prefix listRDS standbyisolated subnetInterface endpointsECR, Logs, STS

Rules: ALB in public subnets only; app in private subnets with a NAT gateway per AZ; DB in isolated subnets (no route to 0.0.0.0/0); SG chain ALB-SG → App-SG → DB-SG by reference; S3/DynamoDB gateway endpoints everywhere.

Reference architecture 2

Multi-account hub-and-spoke with shared services

The network account owns the TGW and its route tables; workload accounts only attach.

Network accountTransit Gatewayshared to the org via RAMTGW RT: prodprod + shared routesTGW RT: non-proddev + shared routesTGW RT: sharedall spoke routesDX gateway / VPNon-premisesWorkload and shared-services accountsProd VPCassociated: RT prodDev VPCassociated: RT non-prodShared services VPCassociated: RT sharedResolver endpoints, ADinterface endpoints, toolingEach spoke: 10.0.0.0/8 → TGW in its route tables; CIDR from IPAM

Isolation comes from TGW route tables: prod and dev never propagate into each other's table, both reach shared services. Shared services hosts Resolver endpoints and centralized interface endpoints; PHZs and Resolver rules are shared with RAM or Route 53 Profiles.

Reference architecture 3

Centralized egress and inspection

One inspection VPC filters spoke-to-internet and spoke-to-spoke traffic for the whole organization.

Spoke VPCsSpoke A0.0.0.0/0 → TGWSpoke B0.0.0.0/0 → TGWTransit GatewayRT inspection: spoke CIDRsTGW RT spoke0.0.0.0/0 → inspection att.Inspection VPC · appliance mode ONTGW subnet0/0 → FW endpointNetwork Firewallendpoint per AZNAT gatewaypublic subnetInternet gatewayFirewall subnet RT0/0 → NAT · 10/8 → TGWPublic subnet RT0/0 → IGW · 10/8 → FW endpoint

Appliance mode keeps both directions of a flow on the same firewall AZ. Since Jun 2025 Network Firewall can also attach natively to the TGW, which removes the inspection VPC plumbing; the routing logic stays the same.

Reference architecture 4

Hybrid: Direct Connect with VPN backup

High resiliency: two DX locations, BGP chooses the active path, VPN takes over only when both DX paths fail.

On-premisesRouter 1ASN 65010Router 2ASN 65010Firewall (VPN backup)internetDX location 1dedicated 10GDX location 2dedicated 10GDX gatewaytransit VIFs, allowed prefixesTransit GatewayECMP for VPNVPCssummarized 10.64.0.0/127224:73007224:7100local preference communitiesSite-to-Site VPN backup (same prefixes, never more specific)

For identical prefixes, AWS already prefers routes learned over DX to VPN routes; a more specific prefix over the VPN would win, so never advertise one. On-premises, use local preference so the return path also prefers DX.

Reference architecture 5

Multi-Region with AWS Cloud WAN

One JSON policy defines Regions, segments, attachment rules and inter-segment routing; Cloud WAN builds the global mesh.

Cloud WAN core network (policy-defined)CNE ap-northeast-1core network edgeCNE us-east-1core network edgeSegments: prod, dev, sharedrouting per segment, all RegionsProd VPC (Tokyo)tag segment=prodDev VPC (Tokyo)tag segment=devProd VPC (Virginia)tag segment=prodShared services VPCshared with prod and devOn-premisesConnect (SD-WAN) / VPN / DX gatewayInspectionsend-via a firewall segment

Attachments land in a segment by tag (attachment policies). Segments are isolated by default; share them or send traffic through an inspection segment with segment actions. Alternative at smaller scale: one TGW per Region with inter-Region TGW peering and static routes.

Exam strategy

Reading an ANS-C01 question

Most questions have two technically valid answers. The constraint in the stem picks the winner.

1 · Find the constraint

  • "Least operational overhead" → managed service, native feature, no appliances.
  • "Most cost-effective" → peering over TGW for few VPCs, gateway endpoints, VPN over DX for low volume.
  • "Highest availability" → two DX locations, two tunnels, multi-AZ endpoints.
  • "Overlapping CIDRs" → PrivateLink or private NAT, never peering.

2 · Trace the packet

  • Draw source → each hop → destination, then the return path.
  • At each hop: route? allowed? MTU? name resolution?
  • Count transitive hops: peering and VGW are non-transitive.
  • Check the Region of each piece (CloudFront certs, GA, Network Manager).

3 · Eliminate

  • Options that change the application when the stem forbids it.
  • Features that do not exist (ECMP on a VGW, SGs on a NAT gateway, BGP over static VPN).
  • Answers that move the problem: opening 0.0.0.0/0 to "fix" reachability.
  • Multi-response: every chosen option must be right; pick exactly the number asked.

Matching questions: every pair must be right for credit. Start with the pairs you are sure of and eliminate.

Exam strategy

Keyword → answer cheat sheet (1/2)

Common stems and the answer they almost always point to.

If the stem says…Think first of
Connect thousands of VPCs and on-premises, transitive routing, segmentationTransit Gateway with multiple route tables
Global network across Regions defined by policy, segmentsAWS Cloud WAN
Expose a service to other VPCs or accounts with overlapping CIDRsPrivateLink (NLB endpoint service)
Consistent private connection, predictable latency, high volumeDirect Connect (+ VPN or second DX for resiliency)
Encrypt Direct ConnectMACsec (layer 2, dedicated) or IPsec VPN over DX (private IP VPN on TGW)
On-premises must resolve private hosted zone namesResolver inbound endpoint + conditional forwarder
VPC must resolve on-premises domainResolver outbound endpoint + forwarding rule (share with RAM)
Exam strategy

Keyword → answer cheat sheet (2/2)

Common stems and the answer they almost always point to.

If the stem says…Think first of
Static anycast IPs, TCP/UDP, fast regional failoverAWS Global Accelerator
Third-party firewall appliances inline, transparent, scalableGateway Load Balancer (GENEVE) + GWLB endpoints
Stateful inspection across AZs breaks through TGWEnable appliance mode on the inspection VPC attachment
Preserve client IP at the targetNLB (client IP preservation) or ALB X-Forwarded-For; proxy protocol v2 for TLS passthrough
Find a configuration reason two ENIs cannot talkVPC Reachability Analyzer
Inspect packet payloads for an intermittent problemVPC Traffic Mirroring
Large transfers hang but small requests workMTU mismatch: allow ICMP, lower MTU, clamp MSS
Lab setup

Before you start

One sandbox account (plus a second for Lab 2), a budget alarm, and a simulated data center.

Accounts and tools

  • Sandbox account lab-a; second account lab-b (CLI profile) for Lab 2.
  • AWS CLI v2, Terraform ≥ 1.6, jq, dig, iperf3.
  • export AWS_REGION=ap-northeast-1 (AZ IDs apne1-az1… differ per Region).
  • Tag everything Project=netlab and find leftovers with the Resource Groups Tagging API.
  • Shell access through EC2 Instance Connect Endpoint: no public IPs, no SSH keys, no SSM endpoints.

Simulated on-premises

  • A separate VPC onprem 10.200.0.0/16 stands in for the data center.
  • One Ubuntu 24.04 instance cgw (10.200.0.10) with an Elastic IP and source/destination check disabled.
  • It runs strongSwan (IPsec), FRRouting (BGP) and BIND (DNS).
  • onprem route table: 10.0.0.0/8 → the cgw ENI.
  • Direct Connect cannot be simulated; VPN + BGP teaches the same routing.
# budget alarm first
aws budgets create-budget \
 --account-id $ACCT --budget '{
  "BudgetName":"netlab","TimeUnit":"MONTHLY",
  "BudgetType":"COST","BudgetLimit":
  {"Amount":"30","Unit":"USD"}}' \
 --notifications-with-subscribers '[{
  "Notification":{
   "NotificationType":"ACTUAL",
   "ComparisonOperator":"GREATER_THAN",
   "Threshold":80},
  "Subscribers":[{"SubscriptionType":
   "EMAIL","Address":"you@example.com"}]}]'

# shell into a private instance
aws ec2 create-instance-connect-endpoint \
  --subnet-id $PRV \
  --security-group-ids $SG_EICE
aws ec2-instance-connect ssh \
  --instance-id $I --os-user ec2-user \
  --connection-type eice

Each lab = Build, Verify, Break it, Clean up. Save every lab's commands as labN.sh: Lab 8 rebuilds Lab 1 from its script.

Lab 1 · Build

A dual-stack VPC from scratch with the CLI

Goal: subnets, IGW, egress-only IGW, NAT and flow logs by hand, so every later abstraction is familiar.

VPC=$(aws ec2 create-vpc --cidr-block 10.10.0.0/16 \
  --amazon-provided-ipv6-cidr-block --query Vpc.VpcId --output text)
V6=$(aws ec2 describe-vpcs --vpc-ids $VPC --output text --query \
  'Vpcs[0].Ipv6CidrBlockAssociationSet[0].Ipv6CidrBlock')
echo $V6                  # e.g. 2406:da14:abc:de00::/56
P6=${V6%00::/56}          # prefix for the /64 subnets
sub() { aws ec2 create-subnet --vpc-id $VPC --cidr-block $1 \
  --availability-zone-id apne1-az1 --ipv6-cidr-block $2 \
  --query Subnet.SubnetId --output text; }
PUB=$(sub 10.10.0.0/24 ${P6}00::/64)
PRV=$(sub 10.10.10.0/24 ${P6}01::/64)
IGW=$(aws ec2 create-internet-gateway --output text \
  --query InternetGateway.InternetGatewayId)
aws ec2 attach-internet-gateway --internet-gateway-id $IGW \
  --vpc-id $VPC
EIGW=$(aws ec2 create-egress-only-internet-gateway --vpc-id $VPC \
  --output text \
  --query EgressOnlyInternetGateway.EgressOnlyInternetGatewayId)
EIP=$(aws ec2 allocate-address --domain vpc \
  --query AllocationId --output text)
NAT=$(aws ec2 create-nat-gateway --subnet-id $PUB \
  --allocation-id $EIP --query NatGateway.NatGatewayId --output text)
aws ec2 wait nat-gateway-available --nat-gateway-ids $NAT
rtb() { aws ec2 create-route-table --vpc-id $VPC \
  --query RouteTable.RouteTableId --output text; }
RTPUB=$(rtb); RTPRV=$(rtb)
aws ec2 create-route --route-table-id $RTPUB \
  --destination-cidr-block 0.0.0.0/0 --gateway-id $IGW
aws ec2 create-route --route-table-id $RTPRV \
  --destination-cidr-block 0.0.0.0/0 --nat-gateway-id $NAT
aws ec2 create-route --route-table-id $RTPRV \
  --destination-ipv6-cidr-block ::/0 \
  --egress-only-internet-gateway-id $EIGW
aws ec2 associate-route-table --route-table-id $RTPUB --subnet-id $PUB
aws ec2 associate-route-table --route-table-id $RTPRV --subnet-id $PRV

aws logs create-log-group --log-group-name /netlab/flow
ROLE=arn:aws:iam::$ACCT:role/netlab-flowlogs
FL=$(aws ec2 create-flow-logs --resource-type VPC --resource-ids $VPC \
  --log-destination-type cloud-watch-logs --log-group-name /netlab/flow \
  --traffic-type ALL --deliver-logs-permission-arn $ROLE \
  --query 'FlowLogIds[0]' --output text)
AMI=/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64
I=$(aws ec2 run-instances --image-id resolve:ssm:$AMI \
  --instance-type t3.micro --subnet-id $PRV --ipv6-address-count 1 \
  --security-group-ids $SG_APP \
  --query 'Instances[0].InstanceId' --output text)

Prerequisites: IAM role netlab-flowlogs trusted by vpc-flow-logs.amazonaws.com; SG_APP allows TCP 22 from the EICE security group; an EICE in $PRV.

Lab 1 · Verify, break it, clean up

Prove each path, then break it on purpose

Every break has one symptom and one tool that explains it.

Verify

  • curl -4 -s https://checkip.amazonaws.com prints the NAT EIP, not the instance IP.
  • ping -6 -c3 2001:4860:4860::8888 works through the egress-only IGW.
  • Nothing on the internet can reach the instance: no public IPv4, and the EIGW blocks inbound IPv6.
  • Logs Insights on /netlab/flow: fields srcaddr, dstaddr, dstport, action | sort @timestamp desc | limit 20.

Break it

  • Route: delete the 0.0.0.0/0 route in $RTPRV → IPv4 curl times out, IPv6 still works. Reachability Analyzer from the ENI to the IGW names the missing route.
  • NACL: move the private subnet to a NACL that allows all outbound but inbound only TCP 22 from 10.10.0.0/16 → curl hangs; Flow Logs show inbound REJECT to high ports. Fix: allow inbound TCP 1024–65535.
  • Why does SSH through the EICE keep working?
aws ec2 terminate-instances \
  --instance-ids $I
aws ec2 delete-instance-connect-endpoint \
  --instance-connect-endpoint-id $EICE
aws ec2 delete-nat-gateway \
  --nat-gateway-id $NAT
aws ec2 wait nat-gateway-deleted \
  --nat-gateway-ids $NAT
aws ec2 release-address --allocation-id $EIP
aws ec2 delete-flow-logs --flow-log-ids $FL
aws logs delete-log-group \
  --log-group-name /netlab/flow
aws ec2 detach-internet-gateway \
  --internet-gateway-id $IGW --vpc-id $VPC
aws ec2 delete-internet-gateway \
  --internet-gateway-id $IGW
aws ec2 delete-egress-only-internet-gateway \
  --egress-only-internet-gateway-id $EIGW
for S in $PUB $PRV; do
  aws ec2 delete-subnet --subnet-id $S; done
for R in $RTPUB $RTPRV; do
  aws ec2 delete-route-table \
    --route-table-id $R; done
aws ec2 delete-vpc --vpc-id $VPC
Lab 2 · Build

Endpoints, endpoint policies and your own PrivateLink service

Goal: reach S3 and STS without the internet, restrict them, and publish a service to another account that uses the same CIDR.

# start from the Lab 1 VPC; bucket netlab-$ACCT exists
cat > s3-ep.json <<EOF
{"Statement":[{"Effect":"Allow","Principal":"*",
  "Action":["s3:GetObject","s3:PutObject","s3:ListBucket"],
  "Resource":["arn:aws:s3:::netlab-$ACCT",
              "arn:aws:s3:::netlab-$ACCT/*"]}]}
EOF
S3EP=$(aws ec2 create-vpc-endpoint --vpc-id $VPC \
  --service-name com.amazonaws.$AWS_REGION.s3 \
  --route-table-ids $RTPRV --policy-document file://s3-ep.json \
  --query VpcEndpoint.VpcEndpointId --output text)

aws ec2 modify-vpc-attribute --vpc-id $VPC \
  --enable-dns-hostnames '{"Value":true}'
STSEP=$(aws ec2 create-vpc-endpoint --vpc-id $VPC \
  --vpc-endpoint-type Interface \
  --service-name com.amazonaws.$AWS_REGION.sts \
  --subnet-ids $PRV --security-group-ids $SG_443 \
  --private-dns-enabled \
  --query VpcEndpoint.VpcEndpointId --output text)
# provider (lab-a): nginx instance $I behind an internal NLB
NLB=$(aws elbv2 create-load-balancer --name netlab-nlb \
  --type network --scheme internal --subnets $PRV \
  --query 'LoadBalancers[0].LoadBalancerArn' --output text)
TG=$(aws elbv2 create-target-group --name netlab-tg --protocol TCP \
  --port 80 --vpc-id $VPC --target-type instance \
  --query 'TargetGroups[0].TargetGroupArn' --output text)
aws elbv2 register-targets --target-group-arn $TG --targets Id=$I
aws elbv2 create-listener --load-balancer-arn $NLB --protocol TCP \
  --port 80 --default-actions Type=forward,TargetGroupArn=$TG
SVC=$(aws ec2 create-vpc-endpoint-service-configuration \
  --network-load-balancer-arns $NLB --acceptance-required \
  --query ServiceConfiguration.ServiceId --output text)
aws ec2 modify-vpc-endpoint-service-permissions --service-id $SVC \
  --add-allowed-principals arn:aws:iam::$ACCT_B:root
# consumer (lab-b): its VPC is also 10.10.0.0/16, on purpose
EP_B=$(aws ec2 create-vpc-endpoint --profile lab-b \
  --vpc-id $VPC_B --vpc-endpoint-type Interface \
  --service-name com.amazonaws.vpce.$AWS_REGION.$SVC \
  --subnet-ids $SUB_B --security-group-ids $SG_B \
  --query VpcEndpoint.VpcEndpointId --output text)
aws ec2 accept-vpc-endpoint-connections --service-id $SVC \
  --vpc-endpoint-ids $EP_B

SG_443 allows TCP 443 from 10.10.0.0/16. The nginx SG allows TCP 80 from 10.10.0.0/16: PrivateLink traffic arrives from NLB private IPs, not from the consumer.

Lab 2 · Verify, break it, clean up

Private paths to AWS and to your own service

Prove the path in DNS, in the route table, and in the policy.

Verify

  • describe-route-tables --route-table-ids $RTPRV shows a pl-… route to the gateway endpoint.
  • aws s3 ls s3://netlab-$ACCT works; another bucket returns AccessDenied (endpoint policy).
  • dig +short sts.$AWS_REGION.amazonaws.com returns 10.10.x.x.
  • From lab-b: curl http://<endpoint DNS name> returns nginx although both VPCs are 10.10.0.0/16.
  • The nginx log shows NLB private IPs as the client.

Break it

  • SG: remove TCP 443 from SG_443 → aws sts get-caller-identity hangs while dig still returns private IPs: it resolves, but is not allowed.
  • Private DNS: modify-vpc-endpoint --vpc-endpoint-id $STSEP --no-private-dns-enabled → dig returns public IPs; calls go through NAT again.
  • Acceptance: reject-vpc-endpoint-connections --service-id $SVC --vpc-endpoint-ids $EP_B → consumer state rejected, curl fails.
aws ec2 delete-vpc-endpoints \
  --profile lab-b --vpc-endpoint-ids $EP_B
aws ec2 \
  delete-vpc-endpoint-service-configurations \
  --service-ids $SVC
aws elbv2 delete-load-balancer \
  --load-balancer-arn $NLB
aws elbv2 delete-target-group \
  --target-group-arn $TG
aws ec2 delete-vpc-endpoints \
  --vpc-endpoint-ids $STSEP $S3EP
# lab-b: terminate the test instance,
#        delete its VPC
# lab-a: keep the Lab 1 VPC for Lab 10,
#        or run the Lab 1 cleanup
Lab 3 · Build

Transit Gateway with isolated prod and dev and shared services

Goal: prod 10.11.0.0/16, dev 10.12.0.0/16, shared 10.13.0.0/16; two TGW route tables; Route Analyzer to explain paths.

TGW=$(aws ec2 create-transit-gateway --description netlab \
  --options AmazonSideAsn=64512,DefaultRouteTableAssociation=disable,\
DefaultRouteTablePropagation=disable \
  --query TransitGateway.TransitGatewayId --output text)
until [ "$(aws ec2 describe-transit-gateways \
  --transit-gateway-ids $TGW --output text \
  --query 'TransitGateways[0].State')" = available ]; do sleep 15; done

att() { aws ec2 create-transit-gateway-vpc-attachment \
  --transit-gateway-id $TGW --vpc-id $1 --subnet-ids $2 \
  --output text --query \
  TransitGatewayVpcAttachment.TransitGatewayAttachmentId; }
A_PROD=$(att $VPC_PROD $SUB_PROD)
A_DEV=$(att $VPC_DEV $SUB_DEV)
A_SH=$(att $VPC_SH $SUB_SH)
until [ -z "$(aws ec2 describe-transit-gateway-attachments --output text \
  --transit-gateway-attachment-ids $A_PROD $A_DEV $A_SH \
  --query 'TransitGatewayAttachments[?State!=`available`].State')" ]
do sleep 15; done

rt() { aws ec2 create-transit-gateway-route-table --output text \
  --transit-gateway-id $TGW \
  --query TransitGatewayRouteTable.TransitGatewayRouteTableId; }
RT_SPOKE=$(rt); RT_SH=$(rt)
assoc() { aws ec2 associate-transit-gateway-route-table \
  --transit-gateway-route-table-id $1 \
  --transit-gateway-attachment-id $2; }
prop() { aws ec2 enable-transit-gateway-route-table-propagation \
  --transit-gateway-route-table-id $1 \
  --transit-gateway-attachment-id $2; }
assoc $RT_SPOKE $A_PROD; assoc $RT_SPOKE $A_DEV; assoc $RT_SH $A_SH
prop $RT_SPOKE $A_SH                    # spokes see shared only
prop $RT_SH $A_PROD; prop $RT_SH $A_DEV # shared sees both

for R in $RT_PROD $RT_DEV $RT_SHVPC; do # the VPC route tables
  aws ec2 create-route --route-table-id $R \
    --destination-cidr-block 10.0.0.0/8 --transit-gateway-id $TGW
done

# Network Manager (home Region us-west-2) for Route Analyzer
GN=$(aws networkmanager create-global-network --region us-west-2 \
  --query GlobalNetwork.GlobalNetworkId --output text)
TGW_ARN=arn:aws:ec2:$AWS_REGION:$ACCT:transit-gateway/$TGW
aws networkmanager register-transit-gateway --region us-west-2 \
  --global-network-id $GN --transit-gateway-arn $TGW_ARN

Each VPC: one attachment subnet (same AZ ID in all three) and a test instance whose SG allows ICMP from 10.0.0.0/8. Create an EICE in prod and in shared.

Lab 3 · Verify, break it, clean up

Segmentation by route table, and the return path

Isolation is just "no route". Prove it with the route tables, then with Route Analyzer.

Verify

  • search-transit-gateway-routes --transit-gateway-route-table-id $RT_SPOKE --filters Name=state,Values=active lists only 10.13.0.0/16.
  • prod → shared ping works; prod → dev fails, by design.
  • aws networkmanager start-route-analysis --region us-west-2 --global-network-id $GN --source TransitGatewayAttachmentArn=<prod att ARN>,IpAddress=10.11.0.10 --destination TransitGatewayAttachmentArn=<dev att ARN>,IpAddress=10.12.0.10 --include-return-path, then get-route-analysis.

Break it

  • Missing propagation: disable-transit-gateway-route-table-propagation of $A_SH from $RT_SPOKE → prod → shared fails; TGW metric PacketDropCountNoRoute rises; Route Analyzer shows no route. Re-enable to fix.
  • Half a route: add a static route 10.12.0.0/16 → $A_DEV in $RT_SPOKE. Does prod reach dev? Explain with the table dev is associated with.
for A in $A_PROD $A_DEV $A_SH; do
  aws ec2 \
    delete-transit-gateway-vpc-attachment \
    --transit-gateway-attachment-id $A
done
# wait until the attachments are deleted
for R in $RT_SPOKE $RT_SH; do
  aws ec2 \
    delete-transit-gateway-route-table \
    --transit-gateway-route-table-id $R
done
aws networkmanager \
  deregister-transit-gateway \
  --region us-west-2 \
  --global-network-id $GN \
  --transit-gateway-arn $TGW_ARN
aws networkmanager delete-global-network \
  --region us-west-2 --global-network-id $GN
aws ec2 delete-transit-gateway \
  --transit-gateway-id $TGW
# then instances, EICEs and the three VPCs
# (keep TGW + prod VPC for Lab 4)
Lab 4 · Build

Site-to-Site VPN with BGP from Transit Gateway to "on-premises"

Goal: two IPsec tunnels from the Lab 3 TGW to cgw (strongSwan + FRR), BGP over both. AWS side is .1 of each /30, cgw is .2.

CGW=$(aws ec2 create-customer-gateway --type ipsec.1 \
  --bgp-asn 65010 --ip-address $CGW_EIP --output text \
  --query CustomerGateway.CustomerGatewayId)
VPN=$(aws ec2 create-vpn-connection --type ipsec.1 \
  --customer-gateway-id $CGW --transit-gateway-id $TGW \
  --options '{"TunnelOptions":[
   {"TunnelInsideCidr":"169.254.10.0/30","PreSharedKey":"netlabTunnelOne"},
   {"TunnelInsideCidr":"169.254.11.0/30","PreSharedKey":"netlabTunnelTwo"}]}' \
  --query VpnConnection.VpnConnectionId --output text)
aws ec2 describe-vpn-connections --vpn-connection-ids $VPN --query \
  'VpnConnections[0].Options.TunnelOptions[].OutsideIpAddress'
aws ec2 wait vpn-connection-available --vpn-connection-ids $VPN
A_VPN=$(aws ec2 describe-transit-gateway-attachments --output text \
  --filters Name=resource-id,Values=$VPN Name=state,Values=available \
  --query 'TransitGatewayAttachments[0].TransitGatewayAttachmentId')
assoc $RT_SH $A_VPN; prop $RT_SPOKE $A_VPN  # helpers from Lab 3

# on cgw, as root
apt-get install -y strongswan-swanctl charon-systemd frr
sysctl -w net.ipv4.ip_forward=1 net.ipv4.conf.all.rp_filter=0
for n in 1 2; do
  ip link add xfrm$n type xfrm dev ens5 if_id $n
  ip addr add 169.254.1$((n-1)).2/30 dev xfrm$n
  ip link set xfrm$n up mtu 1422
done
# FRR: set bgpd=yes in /etc/frr/daemons, restart frr, then:
vtysh <<'END'
conf t
router bgp 65010
 bgp router-id 10.200.0.10
 no bgp ebgp-requires-policy
 neighbor 169.254.10.1 remote-as 64512
 neighbor 169.254.11.1 remote-as 64512
 address-family ipv4 unicast
  network 10.200.0.0/24
  maximum-paths 2
 exit-address-family
end
write memory
END

# AWS side .1 = BGP neighbor (ASN 64512, the TGW Amazon-side ASN)
# cgw side .2 = local address on xfrm1 / xfrm2
# "network 10.200.0.0/24" works because the subnet is connected
# on ens5; maximum-paths 2 installs both tunnels for cgw → AWS
# (outbound ECMP); AWS → cgw ECMP needs VPN ECMP on the TGW

Next slide: the strongSwan config that brings up the two tunnels. BGP comes up only after both IPsec tunnels are established.

Lab 4 · Build (IPsec)

strongSwan route-based tunnels with XFRM interfaces

One connection per tunnel; if_id binds each child SA to its xfrm interface, so BGP decides what enters which tunnel.

# /etc/swanctl/conf.d/aws.conf — tunnel 1
# (copy as t2: tunnel 2 outside IP and PSK, if_id 2)
connections {
  t1 {
    version = 2
    local_addrs = 10.200.0.10
    remote_addrs = <T1_OUTSIDE_IP>
    proposals = aes256-sha256-modp2048
    dpd_delay = 10s
    local {
      auth = psk
      id = <CGW_EIP>
    }
    remote {
      auth = psk
      id = <T1_OUTSIDE_IP>
    }
    children {
      t1 {
        local_ts = 0.0.0.0/0
        remote_ts = 0.0.0.0/0
        esp_proposals = aes256-sha256-modp2048
        if_id_in = 1
        if_id_out = 1
        start_action = start
        dpd_action = restart
      }
    }
  }
}
secrets {
  ike-t1 {
    id-1 = <CGW_EIP>
    id-2 = <T1_OUTSIDE_IP>
    secret = "netlabTunnelOne"
  }
}

# load and check
swanctl --load-all
swanctl --list-sas
#  IKE_SA ESTABLISHED,
#  CHILD_SA INSTALLED
ip -s link show xfrm1
#  counters move with traffic

Why it is built this way

  • local_addrs is the private IP; the IKE id is the Elastic IP, which AWS knows as the customer gateway.
  • Behind the EIP, NAT-T moves IKE and ESP to UDP 4500.
  • 0.0.0.0/0 selectors + if_id = route-based VPN: BGP, not the IPsec policy, picks the tunnel.
  • AES-256, SHA-256 and DH group 14 are in the default AWS tunnel proposals.
  • One key = value per line: the format requires it.
Lab 4 · Verify, break it, clean up

Two tunnels, BGP, failover, and an MTU black hole

Prove BGP state on both sides, then steer and break the path. cgw SG: UDP 500/4500 from both tunnel IPs.

Verify

  • swanctl --list-sas: two ESTABLISHED IKE SAs.
  • vtysh -c "show ip bgp summary": two neighbors Established; show ip bgp lists 10.11.0.0/16.
  • AWS: describe-vpn-connections --query "VpnConnections[0].VgwTelemetry[].[OutsideIpAddress,Status,AcceptedRouteCount]" → UP, 1.
  • search-transit-gateway-routes on $RT_SPOKE shows 10.200.0.0/24 via the VPN attachment.
  • prod instance: ping 10.200.0.10 works.

Break it

  • Steer: FRR route-map PREPEND permit 10 / set as-path prepend 65010 65010 / neighbor 169.254.10.1 route-map PREPEND out → AWS returns traffic over tunnel 2 (tcpdump -ni xfrm2, TunnelDataOut).
  • Fail over: swanctl --terminate --ike t2 → traffic moves to tunnel 1 after the BGP hold time.
  • MTU: ip link set xfrm1 mtu 1500 → ESP packets exceed 1500 bytes on the internet path; large transfers stall or fragment while ping works (watch sizes with tcpdump -ni ens5 udp port 4500). Fix: MTU 1422 and iptables -t mangle -A FORWARD -o xfrm+ -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1382 (AWS table: AES-CBC + SHA2-256 with NAT-T; max 1446/1406 only with AES-GCM and no NAT-T).
aws ec2 delete-vpn-connection \
  --vpn-connection-id $VPN
aws ec2 delete-customer-gateway \
  --customer-gateway-id $CGW
# a VPN bills per connection-hour:
# delete it the same day.

# keep onprem VPC + cgw for Lab 5,
# otherwise terminate cgw, release its
# EIP and delete the onprem VPC.
# Then run the Lab 3 cleanup.
Lab 5 · Build

Hybrid DNS: private zone, Resolver endpoints and BIND

Goal: the VPC resolves corp.example from BIND on cgw, on-premises resolves aws.netlab. Needs Lab 4 and a 2nd prod subnet.

HZ=$(aws route53 create-hosted-zone --name aws.netlab \
  --caller-reference $(date +%s) --hosted-zone-config PrivateZone=true \
  --vpc VPCRegion=$AWS_REGION,VPCId=$VPC_PROD \
  --query HostedZone.Id --output text)
aws route53 change-resource-record-sets --hosted-zone-id $HZ \
  --change-batch '{"Changes":[{"Action":"UPSERT",
   "ResourceRecordSet":{"Name":"db.aws.netlab","Type":"A","TTL":60,
    "ResourceRecords":[{"Value":"10.11.0.10"}]}}]}'

ep() { aws route53resolver create-resolver-endpoint --name nl-$1 \
  --direction $2 --creator-request-id $1-$(date +%s) \
  --security-group-ids $SG_DNS --ip-addresses $3 $4 \
  --query ResolverEndpoint.Id --output text; }
IN=$(ep in INBOUND SubnetId=$S1,Ip=10.11.0.20 SubnetId=$S2,Ip=10.11.1.20)
OUT=$(ep out OUTBOUND SubnetId=$S1 SubnetId=$S2)
RULE=$(aws route53resolver create-resolver-rule --name corp \
  --rule-type FORWARD --domain-name corp.example \
  --resolver-endpoint-id $OUT --target-ips Ip=10.200.0.10,Port=53 \
  --creator-request-id r-$(date +%s) \
  --query ResolverRule.Id --output text)
aws route53resolver associate-resolver-rule \
  --resolver-rule-id $RULE --vpc-id $VPC_PROD
# BIND on cgw (apt-get install -y bind9): named.conf.local
zone "corp.example" { type master; file "/etc/bind/db.corp"; };
zone "aws.netlab" { type forward; forward only;
                    forwarders { 10.11.0.20; 10.11.1.20; }; };
# named.conf.options: allow-query { 10.0.0.0/8; }; recursion yes;
#   validate-except { "aws.netlab"; };
# /etc/bind/db.corp
$TTL 60
@   IN SOA ns.corp.example. admin.corp.example. 1 60 60 600 60
@   IN NS  ns.corp.example.
ns  IN A   10.200.0.10
app IN A   10.200.0.10
aws logs create-log-group --log-group-name /netlab/dns
QL=$(aws route53resolver create-resolver-query-log-config \
  --name netlab --creator-request-id q-$(date +%s) \
  --destination-arn \
    arn:aws:logs:$AWS_REGION:$ACCT:log-group:/netlab/dns \
  --query ResolverQueryLogConfig.Id --output text)
aws route53resolver associate-resolver-query-log-config \
  --resolver-query-log-config-id $QL --resource-id $VPC_PROD
# DNS Firewall: create-firewall-domain-list, update-firewall-domains
#   (ADD example.org), create-firewall-rule-group, create-firewall-rule
#   (--action BLOCK --block-response NXDOMAIN),
#   associate-firewall-rule-group (--vpc-id $VPC_PROD --priority 101)

SG_DNS allows UDP and TCP 53 from 10.0.0.0/8. Endpoints need IPs in at least two subnets. Each create-* call needs a unique --creator-request-id.

Lab 5 · Verify, break it, clean up

Both directions of hybrid resolution

Check the answer, the server that gave it, and the log line that proves the path.

Verify

  • prod instance: dig +short app.corp.example → 10.200.0.10 (outbound endpoint → BIND).
  • cgw: dig @10.200.0.10 +short db.aws.netlab → 10.11.0.10 (BIND → inbound endpoint → PHZ).
  • prod instance: dig example.org → NXDOMAIN (DNS Firewall).
  • Logs Insights on /netlab/dns: fields query_name, rcode, srcaddr | sort @timestamp desc.

Break it

  • Rule: disassociate-resolver-rule → app.corp.example returns NXDOMAIN: the VPC asks public DNS.
  • SG: remove port 53 from SG_DNS → on-premises queries time out (SERVFAIL from BIND).
  • DNSSEC: remove validate-except (Ubuntu ships dnssec-validation auto;) → db.aws.netlab returns SERVFAIL: netlab does not exist in the signed public tree.
R53R="aws route53resolver"
$R53R disassociate-resolver-rule \
  --resolver-rule-id $RULE \
  --vpc-id $VPC_PROD
$R53R delete-resolver-rule \
  --resolver-rule-id $RULE
for E in $IN $OUT; do
  $R53R delete-resolver-endpoint \
    --resolver-endpoint-id $E; done
$R53R \
  disassociate-resolver-query-log-config \
  --resolver-query-log-config-id $QL \
  --resource-id $VPC_PROD
$R53R delete-resolver-query-log-config \
  --resolver-query-log-config-id $QL
# DNS Firewall: disassociate the group,
# delete rule, group and domain list.
# Delete the A record (DELETE change):
aws route53 delete-hosted-zone --id $HZ
Lab 6 · Build

ALB, NLB, CloudFront with OAC, and Global Accelerator

Goal: the same nginx targets behind each front door; compare what the target sees as the client and the time to first byte.

# VPC with public subnets in 2 AZs ($PUB1 $PUB2), nginx targets
# nginx: log_format x '$remote_addr xff="$http_x_forwarded_for"';
ALB=$(aws elbv2 create-load-balancer --name netlab-alb \
  --type application --subnets $PUB1 $PUB2 \
  --security-groups $SG_ALB \
  --query 'LoadBalancers[0].LoadBalancerArn' --output text)
NLB=$(aws elbv2 create-load-balancer --name netlab-nlb6 \
  --type network --subnets $PUB1 $PUB2 \
  --query 'LoadBalancers[0].LoadBalancerArn' --output text)
# target groups: HTTP:80 (ALB) and TCP:80 (NLB), target-type
# instance; register-targets; create-listener on port 80 for each

# CloudFront: S3 origin with OAC + ALB origin for /api/*
aws cloudfront create-origin-access-control \
  --origin-access-control-config '{"Name":"netlab-oac",
    "SigningProtocol":"sigv4","SigningBehavior":"always",
    "OriginAccessControlOriginType":"s3"}'
aws cloudfront create-distribution \
  --distribution-config file://dist.json
# bucket policy: allow cloudfront.amazonaws.com s3:GetObject when
#   AWS:SourceArn = the distribution ARN
# Global Accelerator: the API lives in us-west-2
GA=$(aws globalaccelerator create-accelerator --region us-west-2 \
  --name netlab --query Accelerator.AcceleratorArn --output text)
L=$(aws globalaccelerator create-listener --region us-west-2 \
  --accelerator-arn $GA --protocol TCP \
  --port-ranges FromPort=80,ToPort=80 \
  --query Listener.ListenerArn --output text)
aws globalaccelerator create-endpoint-group --region us-west-2 \
  --listener-arn $L --endpoint-group-region $AWS_REGION \
  --endpoint-configurations \
    EndpointId=$ALB,Weight=100,ClientIPPreservationEnabled=true

# compare from your laptop (run each 5 times)
for U in http://$NLB_DNS http://$ALB_DNS \
         https://$CF_DNS/api/ http://$GA_DNS; do
  curl -o /dev/null -s \
    -w "$U connect=%{time_connect} ttfb=%{time_starttransfer}\n" $U
done

dist.json: two origins (S3 with OriginAccessControlId and an empty OriginAccessIdentity; the ALB as an HTTP-only custom origin); default behavior uses managed CachingOptimized, /api/* uses CachingDisabled + AllViewer.

Lab 6 · Verify, break it, clean up

Who is the client, and who is faster?

Read the nginx log on the target for each front door.

Verify

  • NLB: $remote_addr is your public IP (client IP preservation on instance targets).
  • ALB: $remote_addr is an ALB private IP; X-Forwarded-For holds your IP.
  • CloudFront → ALB: X-Forwarded-For lists your IP, then a CloudFront IP.
  • GA → ALB with preservation: your IP appears in X-Forwarded-For.
  • Second request for a static file: x-cache: Hit from cloudfront.
  • Far from the Region, GA and CloudFront usually win on ttfb.

Break it, lock it down

  • OAC: delete the bucket policy → CloudFront returns 403; restore it.
  • Lock the origin: let SG_ALB allow only the prefix list com.amazonaws.global.cloudfront.origin-facing → direct ALB access times out, CloudFront works, GA stops: plan for that.
  • Targets: stop nginx → 502/503 at every front door; check describe-target-health.
GAX="aws globalaccelerator --region us-west-2"
$GAX update-accelerator --no-enabled \
  --accelerator-arn $GA
# when Status=DEPLOYED: delete endpoint
# group, listener, then the accelerator
# (fixed hourly fee: delete it today)
aws cloudfront get-distribution-config \
  --id $CF > cf.json       # note the ETag
# set Enabled=false; update-distribution
# --if-match <ETag>; wait until deployed
aws cloudfront delete-distribution \
  --id $CF --if-match <new ETag>
aws cloudfront \
  delete-origin-access-control \
  --id $OAC --if-match <ETag>
for LB in $ALB $NLB; do
  aws elbv2 delete-load-balancer \
    --load-balancer-arn $LB; done
# target groups, instances, bucket, VPC
Lab 7 · Build

Centralized inspection with Network Firewall and appliance mode

Goal: spoke A (10.21.0.0/16, AZ 1 only) and B (10.22.0.0/16, AZ 2 only) reach the internet and each other via an inspection VPC.

NF="aws network-firewall"
RG=$($NF create-rule-group --type STATEFUL --capacity 100 \
  --rule-group-name netlab-deny-domains --rule-group '{
   "RulesSource":{"RulesSourceList":{"Targets":[".example.net"],
   "TargetTypes":["TLS_SNI","HTTP_HOST"],
   "GeneratedRulesType":"DENYLIST"}}}' \
  --query RuleGroupResponse.RuleGroupArn --output text)
POL=$($NF create-firewall-policy --firewall-policy-name netlab-pol \
  --firewall-policy '{
   "StatelessDefaultActions":["aws:forward_to_sfe"],
   "StatelessFragmentDefaultActions":["aws:forward_to_sfe"],
   "StatefulRuleGroupReferences":[{"ResourceArn":"'$RG'"}]}' \
  --query FirewallPolicyResponse.FirewallPolicyArn --output text)
$NF create-firewall --firewall-name netlab-fw \
  --firewall-policy-arn $POL --vpc-id $VPC_INSP \
  --subnet-mappings SubnetId=$FW_SUB1 SubnetId=$FW_SUB2
$NF describe-firewall --firewall-name netlab-fw --query \
  'FirewallStatus.SyncStates.*.Attachment.[SubnetId,EndpointId]'
$NF update-logging-configuration --firewall-name netlab-fw \
  --logging-configuration '{"LogDestinationConfigs":[{
   "LogType":"ALERT","LogDestinationType":"CloudWatchLogs",
   "LogDestination":{"logGroup":"/netlab/nfw-alert"}}]}'

Routes to create

Route tableDestination → target
Spoke A / B subnets0.0.0.0/0 → TGW
TGW RT spoke (A, B associated)0.0.0.0/0 → inspection attachment
TGW RT inspectionpropagate A and B
Inspection TGW subnet, per AZ0.0.0.0/0 → FW endpoint, same AZ
Firewall subnet, per AZ0/0 → NAT same AZ; 10.0.0.0/8 → TGW
Public subnet, per AZ0/0 → IGW; 10.0.0.0/8 → FW endpoint
Inspection attachmentApplianceModeSupport=enable

Route to a firewall endpoint: create-route --destination-cidr-block 0.0.0.0/0 --vpc-endpoint-id $FWE1. Appliance mode: modify-transit-gateway-vpc-attachment --options ApplianceModeSupport=enable.

Lab 7 · Verify, break it, clean up

Inspect north-south and east-west, then break symmetry

Prove that every flow crosses the firewall, then see what appliance mode is for.

Verify

  • Spoke A: curl -sI https://aws.amazon.com works; curl -sI https://www.example.net is reset or times out.
  • The alert log /netlab/nfw-alert shows the blocked SNI.
  • A → B curl http://<B IP> works (add FLOW logging to see it cross the firewall).
  • Reachability Analyzer A ENI → B ENI lists the firewall endpoint as a hop.

Break it

  • Symmetry: set ApplianceModeSupport=disable on the inspection attachment and repeat A → B. TGW keeps each direction in its source AZ: request via the AZ 1 endpoint, reply via AZ 2.
  • Expected: in FLOW logs each endpoint sees one direction only; stateful rules that need both directions stop matching or drop the flow. Re-enable and compare.
  • Return path: delete the public subnet route 10.0.0.0/8 → FW endpoint → NAT replies have no route back to the spokes: egress breaks, east-west still works.
# firewall endpoints bill per hour
NF="aws network-firewall"
$NF delete-firewall \
  --firewall-name netlab-fw
# wait until the firewall is deleted
$NF delete-firewall-policy \
  --firewall-policy-name netlab-pol
$NF delete-rule-group --type STATEFUL \
  --rule-group-name netlab-deny-domains
aws logs delete-log-group \
  --log-group-name /netlab/nfw-alert
# NAT gateways + EIPs, TGW attachments,
# route tables, TGW, instances, 3 VPCs
Lab 8 · Build

Troubleshooting drill: five faults in one environment

Goal: fix each injected fault with evidence from a tool. Rebuild Lab 1 (lab1.sh), add nginx $WEB and a client $I.

# fault 1: default route gone
aws ec2 delete-route --route-table-id $RTPRV \
  --destination-cidr-block 0.0.0.0/0

# fault 2: NACL drops return traffic (as in Lab 1)

# fault 3: the web SG no longer allows the client
aws ec2 revoke-security-group-ingress --group-id $SG_WEB \
  --protocol tcp --port 80 --source-group $SG_APP

# fault 4: DHCP options point at a DNS server that does not exist
DOPT=$(aws ec2 create-dhcp-options --dhcp-configurations \
  Key=domain-name-servers,Values=10.10.0.99 \
  --query DhcpOptions.DhcpOptionsId --output text)
aws ec2 associate-dhcp-options --dhcp-options-id $DOPT \
  --vpc-id $VPC          # reboot instances to renew the lease

# fault 5: the client SG has no outbound rules
aws ec2 revoke-security-group-egress --group-id $SG_APP \
  --ip-permissions 'IpProtocol=-1,IpRanges=[{CidrIp=0.0.0.0/0}]'

# Reachability Analyzer path: client ENI -> web ENI :80
aws ec2 create-network-insights-path --source $ENI_APP \
  --destination $ENI_WEB --protocol tcp --destination-port 80
# Traffic Mirroring: watch packets reach $WEB (Nitro types only)
TGT=$(aws ec2 create-traffic-mirror-target \
  --network-interface-id $ENI_SNIFFER --output text \
  --query TrafficMirrorTarget.TrafficMirrorTargetId)
F=$(aws ec2 create-traffic-mirror-filter --output text \
  --query TrafficMirrorFilter.TrafficMirrorFilterId)
aws ec2 create-traffic-mirror-filter-rule \
  --traffic-mirror-filter-id $F --traffic-direction ingress \
  --rule-number 100 --rule-action accept --protocol 6 \
  --destination-port-range FromPort=80,ToPort=80 \
  --source-cidr-block 0.0.0.0/0 --destination-cidr-block 0.0.0.0/0
SES=$(aws ec2 create-traffic-mirror-session \
  --network-interface-id $ENI_WEB --traffic-mirror-target-id $TGT \
  --traffic-mirror-filter-id $F --session-number 1 --output text \
  --query TrafficMirrorSession.TrafficMirrorSessionId)

# on the sniffer: mirrored packets arrive as VXLAN on UDP 4789
sudo tcpdump -ni ens5 -vv udp port 4789 | grep 'Flags \[S'

The sniffer SG allows UDP 4789 from the web ENI. Make it realistic: a partner injects the faults in random order, and you do not read the injection script.

Lab 8 · Verify, break it, clean up

Symptom → tool → evidence → fix

Write one line of evidence for each fault before you change anything.

FaultSymptomTool and evidenceFix
1 · No default routeOutbound IPv4 times out; no REJECT in Flow LogsReachability Analyzer ENI → IGW: no route at the route tableRe-add 0.0.0.0/0 → NAT
2 · NACL ephemeralOutbound connects hangFlow Logs: inbound REJECT to ports 1024–65535Allow inbound TCP 1024–65535
3 · Web SGClient → web times outReachability Analyzer names $SG_WEB; mirroring shows SYNs with no replyRe-add the SG reference rule
4 · DHCP DNS"Could not resolve host"; IPs workresolvectl status shows 10.10.0.99; dig @169.254.169.253 worksRe-associate the default options set; renew leases
5 · Egress SGEvery outbound connection failsFlow Logs: outbound REJECT from the client ENIRestore the egress rule

Clean up: delete the mirror session, filter and target (sessions bill per ENI-hour), the insights analyses and path; re-associate the default DHCP options and delete $DOPT; run the Lab 1 cleanup.

Lab 9 · Build

Network as code: IPAM, VPC and TGW in Terraform with an intent check

Goal: rebuild Lab 3 in Terraform, CIDRs from an IPAM pool (Chapter 9), and fail apply when a path regresses.

variable "ipam_pool_id" {}
data "aws_availability_zones" "az" { state = "available" }

resource "aws_vpc" "spoke" {
  for_each            = toset(["prod", "dev", "shared"])
  ipv4_ipam_pool_id   = var.ipam_pool_id
  ipv4_netmask_length = 20
  tags                = { Name = each.key, Project = "netlab" }
}
resource "aws_subnet" "tgw" {
  for_each             = aws_vpc.spoke
  vpc_id               = each.value.id
  availability_zone_id = data.aws_availability_zones.az.zone_ids[0]
  cidr_block           = cidrsubnet(each.value.cidr_block, 8, 0)
}
resource "aws_ec2_transit_gateway" "hub" {
  default_route_table_association = "disable"
  default_route_table_propagation = "disable"
}
resource "aws_ec2_transit_gateway_vpc_attachment" "a" {
  for_each           = aws_vpc.spoke
  transit_gateway_id = aws_ec2_transit_gateway.hub.id
  vpc_id             = each.value.id
  subnet_ids         = [aws_subnet.tgw[each.key].id]
}
# + route tables, associations and propagations as in Lab 3
#   (propagations: for_each resource named "p")
resource "aws_ec2_network_insights_path" "prod_shared" {
  source      = aws_instance.t["prod"].primary_network_interface_id
  destination = aws_instance.t["shared"].primary_network_interface_id
  protocol         = "tcp"
  destination_port = 22
}
resource "terraform_data" "routing" {  # changes when a propagation goes
  input = [for p in aws_ec2_transit_gateway_route_table_propagation.p : p.id]
}
resource "aws_ec2_network_insights_analysis" "prod_shared" {
  network_insights_path_id = aws_ec2_network_insights_path.prod_shared.id
  wait_for_completion      = true
  lifecycle {
    replace_triggered_by = [terraform_data.routing]
    postcondition {
      condition     = self.path_found
      error_message = "prod can no longer reach shared services"
    }
  }
}

# CI: plan on pull request, apply on merge
#   terraform fmt -check && terraform validate
#   terraform plan -out tfplan && terraform apply tfplan
#   terraform apply -replace=aws_ec2_network_insights_analysis.prod_shared
#   (re-checks intent after the routing change is live)
Lab 9 · Verify, break it, clean up

Let the pipeline catch your mistake

The point of the lab: a bad change fails in apply, not in production.

Verify

  • terraform apply completes; the analysis reports the expected result. Add a prod → dev path asserting !self.path_found to encode isolation.
  • aws ec2 describe-vpcs --filters Name=tag:Project,Values=netlab --query "Vpcs[].CidrBlock": three non-overlapping /20s from the pool.
  • The IPAM console shows the allocations with your VPC IDs.
  • terraform plan right after apply shows no changes.

Break it

  • Regression: remove the propagation of shared into the spoke route table and apply. replace_triggered_by re-runs the analysis, but Terraform may run it before the propagation is deleted, so run the CI re-check terraform apply -replace=aws_ec2_network_insights_analysis.prod_shared → the postcondition fails with "prod can no longer reach shared services".
  • Replacement: change ipv4_netmask_length to 21 → plan says the VPCs must be replaced (terraform show -json tfplan | jq on resource_changes lists the deletes). Do not apply.
  • Drift: add a route in the console, then terraform plan -refresh-only reports it.

Clean up

  • terraform destroy (analyses and paths go first automatically).
  • Check for leftover allocations: aws ec2 get-ipam-pool-allocations --ipam-pool-id $POOL.
  • Delete the IPAM if you created it for this lab; check the IPAM tier pricing first.
  • Remove the state bucket or keep it for later labs.
Lab 10 · Build

Measure cost drivers and throughput

Goal: watch NAT processing vanish behind a gateway endpoint, find it in Cost Explorer, and measure the single-flow limit.

# Part A: Lab 1 VPC without the S3 endpoint
aws ce update-cost-allocation-tags-status \
  --cost-allocation-tags-status TagKey=Project,Status=Active
# on the private instance, three times:
aws s3 cp s3://netlab-$ACCT/1g.bin /dev/null
# NAT bytes over the last hour (GNU date: Linux or CloudShell)
aws cloudwatch get-metric-statistics --namespace AWS/NATGateway \
  --metric-name BytesInFromDestination --statistics Sum \
  --dimensions Name=NatGatewayId,Value=$NAT --period 300 \
  --start-time $(date -u -d '-1 hour' +%FT%TZ) \
  --end-time $(date -u +%FT%TZ)
# add the S3 gateway endpoint (Lab 2), repeat: NAT bytes stay flat

# next day; every Cost Explorer API call is billed
aws ce get-cost-and-usage --granularity DAILY \
  --time-period Start=2026-10-01,End=2026-10-31 \
  --metrics UnblendedCost UsageQuantity \
  --filter '{"Tags":{"Key":"Project","Values":["netlab"]}}' \
  --group-by Type=DIMENSION,Key=USAGE_TYPE
# Part B: two c6in.8xlarge in the same AZ, no placement group
# (expensive: finish within 30 minutes)
# on B:  iperf3 -s
# on A:
iperf3 -c $B_IP -t 20 -P 1     # one flow: about 5 Gbps
iperf3 -c $B_IP -t 20 -P 8     # eight flows: near the instance limit

# enable ENA Express on BOTH primary ENIs
for E in $ENI_A $ENI_B; do
  aws ec2 modify-network-interface-attribute \
    --network-interface-id $E \
    --ena-srd-specification 'EnaSrdEnabled=true'
done

iperf3 -c $B_IP -t 20 -P 1     # one flow again: higher
ethtool -S ens5 | grep ena_srd # SRD counters increase
Lab 10 · Verify, break it, clean up

Read the numbers back

Write each measured value next to the documented limit.

Verify

  • NAT BytesInFromDestination grows by about 3 GB without the endpoint and stays flat with it.
  • Cost Explorer shows Region-prefixed usage types such as APN1-NatGateway-Bytes and APN1-NatGateway-Hours for the tag.
  • One iperf3 flow stays near 5 Gbps; eight flows approach the instance bandwidth.
  • With ENA Express, one flow goes beyond 5 Gbps and the ena_srd counters rise.

Break it

  • Enable ENA Express on one side only → the single flow drops back to about 5 Gbps: both ends need it.
  • Put a middlebox in the path → SRD is not used.
  • Point the gateway endpoint policy at another bucket → aws s3 cp fails with AccessDenied while NAT stays idle: the endpoint route still wins.

Clean up

  • Terminate both c6in instances first (largest cost).
  • Delete the 1 GB object and the bucket.
  • Run the Lab 2 and Lab 1 cleanups.
  • Leave the cost allocation tag active.
Lab safety

Cost and cleanup

Network resources bill per hour whether or not traffic flows. A forgotten lab can cost more than the exam.

Billed per hour (delete the same day)

  • NAT gateways and every public IPv4 address (attached or idle).
  • Transit Gateway attachments, Site-to-Site VPN connections.
  • Interface endpoints and Resolver endpoints (per ENI / per IP).
  • Network Firewall endpoints, Traffic Mirroring sessions.
  • Global Accelerator, Cloud WAN core network edges.
  • VPC Encryption Controls and Route Server if you try them.

Never in a lab

  • Shield Advanced (monthly fee with a 1-year commitment).
  • Ordering a Direct Connect connection (port-hours start, plus a cross connect at the colocation).
  • Large instances left running after Lab 10.

Clean up

  • Run each lab's cleanup in reverse order of creation.
  • Check leftovers by tag: aws resourcegroupstaggingapi get-resources --tag-filters Key=Project,Values=netlab.
  • Check for orphans the tag API misses: describe-addresses, describe-nat-gateways, describe-vpc-endpoints, describe-transit-gateway-attachments.
  • Keep the budget alarm on.
Glossary

Routing and connectivity terms

Short anchors; explain each in your own words.

TermShort meaning
Longest prefix matchThe most specific matching route wins
PropagationRoutes learned automatically from an attachment or VGW
AssociationWhich route table a subnet or TGW attachment uses
Appliance modeTGW keeps both directions of a flow in one AZ
ECMPEqual-cost paths used in parallel (TGW VPN, Connect)
ASNAutonomous system number used by BGP
AS_PATH prependMake a path look longer so peers prefer another
TermShort meaning
Local preferenceBGP attribute: higher wins, set by communities on DX
MEDBGP hint to a neighbor about the preferred entry point
VIFDirect Connect virtual interface: private, public or transit
LOA-CFALetter authorizing the cross connect at a DX location
LAGBundle of DX connections acting as one
MACsecLayer 2 encryption on dedicated DX connections
Transitive routingTraffic passing through a middle network; peering and VGW do not allow it
Glossary

Services, DNS and operations terms

Short anchors; explain each in your own words.

TermShort meaning
ENIVirtual network interface: IPs, SGs, MAC
PrivateLinkPrivate, one-way access to a service through interface endpoints
GENEVEEncapsulation GWLB uses to send traffic to appliances (UDP 6081)
Prefix listNamed set of CIDRs referenced by SGs and routes
IPAM poolAddress space from which VPC CIDRs are allocated
NAUNetwork Address Usage: VPC size metric and quota
MSS clampingRewriting TCP MSS so segments fit a smaller MTU
TermShort meaning
PHZPrivate hosted zone, answered only inside associated VPCs
Inbound endpointResolver IPs that on-premises DNS forwards to
Outbound endpointResolver IPs that forward VPC queries to other DNS servers
Alias recordRoute 53 record pointing to an AWS resource, allowed at the apex
DNSSECSigned DNS answers that resolvers can validate
Reachability AnalyzerConfiguration-based path analysis between two endpoints
Traffic MirroringCopy of ENI packets sent to a target over VXLAN
Further reading

Primary references

The exam tests current AWS behavior; when a slide and the docs disagree, the docs win.

Exam and design

  • ANS-C01 exam guide — docs.aws.amazon.com/aws-certification/latest/advanced-networking-specialty-01/
  • AWS networking best practices and decision map — aws.github.io/aws-networking-best-practices/
  • Building a scalable and secure multi-VPC network infrastructure (whitepaper) — docs.aws.amazon.com/whitepapers/
  • Hybrid connectivity (whitepaper) — docs.aws.amazon.com/whitepapers/latest/hybrid-connectivity/
  • AWS Networking & Content Delivery Blog — aws.amazon.com/blogs/networking-and-content-delivery/

Service guides and quotas

  • Amazon VPC quotas — docs.aws.amazon.com/vpc/latest/userguide/amazon-vpc-limits.html
  • Transit Gateway quotas and MTU — docs.aws.amazon.com/vpc/latest/tgw/transit-gateway-quotas.html
  • Direct Connect quotas and VIFs — docs.aws.amazon.com/directconnect/latest/UserGuide/
  • Site-to-Site VPN quotas and CGW best practices — docs.aws.amazon.com/vpn/latest/s2svpn/
  • Network MTU for EC2 — docs.aws.amazon.com/AWSEC2/latest/UserGuide/network_mtu.html
  • Route 53 and VPC Resolver quotas — docs.aws.amazon.com/Route53/latest/DeveloperGuide/DNSLimitations.html
1 / 1