Follow a packet from one ENI to many VPCs, accounts, Regions and on-premises sites: routing, BGP, DNS, load balancing, inspection, troubleshooting and cost — at production depth and exam depth.
Use ← → / Space to navigate · I contents · O overview · T theme · F fullscreen
Click any line to jump there · press I to return to this page
The mental model of the AWS network, the building blocks, and how the ANS-C01 exam is built.
Every packet leaves an ENI and is handled by the Nitro card and the VPC mapping service before it touches a wire.
A VPC spans every AZ in a Region; a subnet lives in exactly one AZ; an ENI lives in exactly one subnet.
Everything that has an IP in a VPC has an ENI: EC2, Lambda in a VPC, RDS, ALB nodes, NAT gateways, interface endpoints, Resolver endpoints, Transit Gateway attachments.
Ask them in this order. Most "network issues" are answered by the first two.
dig, Resolver query logs.ACCEPT / REJECT).Then check the size: MTU mismatches pass the TCP handshake and fail on the first large packet.
The rest of the deck goes service by service; this is the map.
| Need | Services | Chapter |
|---|---|---|
| Network for workloads | VPC, subnets, route tables, IGW, NAT, SG, NACL, IPAM | 1 |
| Reach AWS services privately | Gateway / interface endpoints, PrivateLink, VPC Lattice | 2 |
| Connect many VPCs and accounts | Peering, Transit Gateway, Cloud WAN, RAM, VPC sharing | 3 |
| Connect on-premises | Site-to-Site VPN, Direct Connect, DX gateway, Client VPN | 4 |
| Name things | Route 53 public / private zones, VPC Resolver, DNS Firewall | 5 |
| Spread and accelerate traffic | ALB, NLB, GWLB, CloudFront, Global Accelerator, API Gateway | 6 |
| Filter and encrypt | WAF, Shield, Network Firewall, Firewall Manager, Verified Access, ACM | 7 |
| See and fix | Flow Logs, Traffic Mirroring, Reachability Analyzer, Network Manager | 8 |
Recommended: 5+ years of networking, 2+ years of cloud and hybrid networking.
| Domain | Weight | Main chapters |
|---|---|---|
| 1 Network Design | 30% | 2, 3, 4, 5, 6, 8 |
| 2 Network Implementation | 26% | 1, 3, 4, 5, 6, 7, 8, 9 |
| 3 Network Management and Operation | 20% | 1, 2, 3, 4, 5, 6, 8, 10 |
| 4 Network Security, Compliance, and Governance | 24% | 1, 5, 7, 8, 9 |
Task numbers (for example "Task 1.5") on each slide point to the official exam guide task statements.
The exam guide predates many of these; questions still use the older names and patterns.
Read a part, do its lab, then answer the self-check out loud without notes.
Navigation: ← → or Space to move, I for Contents, O for overview, T for theme, F for fullscreen. Every Contents line is a link.
Tasks 1.6, 2.2, 3.3, 4.1 — addressing, routing, filtering, interfaces and DNS inside one VPC.
CIDR rules, reserved addresses, IPv6, IPAM, and sizing subnets so you never run out.
The primary IPv4 CIDR is fixed for the life of the VPC. Plan it before the first subnet.
local route to every route table.0.0.0.0/8, 127.0.0.0/8, 169.254.0.0/16, 224.0.0.0/4.172.17.0.0/16: Docker bridge default, used by Cloud9 and SageMaker AI.100.64.0.0/10 (CGNAT) is the usual secondary for EKS pods.198.19.0.0/16 with RFC 1918 primaries.Usable IPv4 addresses = 2^(32 − prefix) − 5. A /28 has 11, a /24 has 251.
| Address | Reserved for |
|---|---|
| 10.0.0.0 | Network address |
| 10.0.0.1 | VPC router (implicit router, default gateway) |
| 10.0.0.2 | Amazon DNS (base of the primary VPC CIDR + 2); reserved in every subnet |
| 10.0.0.3 | Reserved for future use |
| 10.0.0.255 | Broadcast (not supported, so reserved) |
100.68.0.18/18 becomes 100.68.0.0/18.Carve by tier, keep a spare /17, and give the busiest tier the biggest subnets.
Public subnets stay small (only load balancers and NAT live there). App subnets get the space because Auto Scaling and EKS pods consume IPs fast.
Every IPv6 address in a VPC is globally unique (GUA). "Private" is achieved with routing, not with address ranges.
::/0 → eigw-… in private subnets.64:ff9b::/96 AAAA records.64:ff9b::/96 → NAT gateway, which performs NAT64.DNS64 rewrites the answer; NAT64 rewrites the packet. Both are needed.
IPAM plans, allocates and audits IP space across Regions and accounts. VPCs request a CIDR from a pool.
--ipv4-ipam-pool-id and a netmask: no overlap by design.Most often hit by EKS with the VPC CNI, where every pod takes a VPC IP.
ContainerCreating: "failed to assign an IP address".InsufficientFreeAddressesInSubnet.AvailableIpAddressCount falling (subnet metric via IPAM / describe-subnets).100.64.0.0/16) and new subnets.WARM_IP_TARGET, MINIMUM_IP_TARGET).Every command maps to a concept on the previous slides.
# VPC from IPAM (no hand-picked CIDR) + Amazon IPv6 /56
aws ec2 create-vpc --ipv4-ipam-pool-id ipam-pool-0a1b \
--ipv4-netmask-length 16 --amazon-provided-ipv6-cidr-block \
--tag-specifications 'ResourceType=vpc,Tags=[{Key=Name,Value=lab}]'
# Dual-stack subnet in one AZ
aws ec2 create-subnet --vpc-id vpc-0abc --availability-zone-id apne1-az1 \
--cidr-block 10.20.32.0/19 --ipv6-cidr-block 2406:da14:aa:ab00::/64
aws ec2 modify-subnet-attribute --subnet-id subnet-0123 \
--assign-ipv6-address-on-creation
# Out of space? add a secondary CIDR
aws ec2 associate-vpc-cidr-block --vpc-id vpc-0abc \
--cidr-block 100.64.0.0/16
# How many IPs are left per subnet
aws ec2 describe-subnets --filters Name=vpc-id,Values=vpc-0abc \
--query 'Subnets[].[SubnetId,CidrBlock,AvailableIpAddressCount]' \
--output tableapne1-az1), not names: ap-northeast-1a maps to different physical AZs in different accounts.--assign-ipv6-address-on-creation gives new ENIs an IPv6 address automatically.create-subnet --ipv6-native with no --cidr-block.Wrong answers that look right.
us-east-1a in two accounts can be different AZs. Match on AZ ID.Scenario → answer → why.
| Scenario | Answer | Why |
|---|---|---|
| EKS pods exhaust a 10.1.0.0/16 VPC; on-premises has no free 10.x space | Secondary CIDR 100.64.0.0/16 + custom networking (or IPv6 cluster) | CGNAT range need not be routable on-premises |
| 50 teams create VPCs; CIDRs keep overlapping | IPAM with Regional and environment pools shared via RAM | Allocation from pools prevents overlap; IPAM reports non-compliance |
| IPv6-only workload must call an IPv4-only SaaS | DNS64 on the subnet + 64:ff9b::/96 route to a NAT gateway | DNS64 synthesises, NAT64 translates |
| Private IPv6 instances must reach the internet but not be reachable | Egress-only internet gateway, ::/0 route | Stateful outbound-only for IPv6 |
| Need a subnet for 30 hosts with the smallest CIDR | /26 (59 usable) | /27 gives 27 usable after the 5 reserved |
Longest prefix match, internet and NAT gateways, ingress routing, prefix lists and Route Server.
Every subnet is associated with exactly one route table. The most specific matching route wins.
local route for each VPC CIDR is always present.| Target | Used for |
|---|---|
igw- | Internet (public IPv4 and IPv6) |
eigw- | Outbound-only IPv6 |
nat- | IPv4 outbound via NAT; NAT64 |
tgw- / vgw- / pcx- | Transit Gateway, VPN/DX, peering |
vpce- | Gateway endpoint (prefix list) or GWLB endpoint |
eni- / instance | Appliance, NAT instance, firewall |
cagw- / lgw- | Wavelength carrier, Outposts local gateway |
A subnet is public if its route table has 0.0.0.0/0 (or ::/0) → internet gateway. Nothing else.
An instance in a public subnet still needs a public IPv4 or Elastic IP to use the IGW. Public IPv4 addresses are billed per hour (since Feb 2024), in use or not.
The IGW is not a box with a throughput limit; it maps public IPv4 ↔ private IPv4 and passes IPv6 untouched.
ip addr shows only the private IP).Managed, redundant inside one AZ. Put one per AZ and route each AZ to its own.
ErrorPortAllocation (port exhaustion), PacketsDropCount.A single shared NAT gateway is a single-AZ failure point and adds inter-AZ data charges.
Regional NAT gateway (next slide) removes the per-AZ wiring: one ID, follows workloads across AZs.
One NAT gateway ID for the whole VPC that expands to every AZ where you have ENIs.
| Zonal | Regional | |
|---|---|---|
| Placement | One AZ, in a public subnet | No subnet; spans AZs automatically |
| Routes | One route table per AZ → its own NAT | Same route (0/0 → nat-…) in every AZ |
| IPs | Up to 8 per gateway | Up to 32 per AZ |
| Private NAT | Yes | No — use zonal for private NAT |
| Own route table | Uses its subnet's table | AWS creates one with a route to the IGW |
| New AZ | You create a new gateway | Expands itself (can take up to 60 min; cross-AZ meanwhile) |
aws ec2 create-nat-gateway --vpc-id vpc-0abc \
--availability-mode regional
# manual mode: you place IPs per AZ
aws ec2 associate-nat-gateway-address \
--nat-gateway-id nat-0123 \
--availability-zone us-east-1b \
--allocation-ids eipalloc-0456Pick by availability, throughput, control and cost.
| NAT gateway | NAT instance | Avoid NAT | |
|---|---|---|---|
| Availability | Managed, redundant in AZ (regional mode spans AZs) | You script failover (or Route Server) | — |
| Throughput | 5 → 100 Gbps auto | Instance bandwidth | — |
| Security group | No | Yes | — |
| Port forwarding / bastion | No | Yes | — |
| Setup | Create + route | Disable source/dest check, iptables MASQUERADE | — |
| Cost | Hourly + per-GB processing | Instance cost only | Gateway endpoints for S3/DynamoDB are free; interface endpoints for other AWS APIs |
| Pick when | Default for production | Tiny labs, custom filtering | Most traffic goes to AWS services |
Associate a route table with the IGW (or VGW). Its routes override where inbound packets go inside the VPC.
Targets in a gateway route table: an ENI or a GWLB endpoint, for destinations inside the VPC CIDR. The appliance subnet routes 0/0 → IGW, and the return path must cross the appliance too (symmetry). Used for third-party firewalls and GWLB designs (Chapter 7).
Used in route tables, security groups, Transit Gateway route tables and Network Firewall rules.
com.amazonaws.<region>.s3 / .dynamodb: gateway endpoint ranges.com.amazonaws.global.cloudfront.origin-facing: lock an ALB to CloudFront.com.amazonaws.<region>.ec2-instance-connect for EC2 Instance Connect.aws ec2 create-managed-prefix-list \
--prefix-list-name on-prem \
--max-entries 10 \
--address-family IPv4 --entries \
Cidr=192.168.0.0/16,Description=dc1 \
Cidr=172.20.0.0/14,Description=dc2
aws ec2 create-route --route-table-id rtb-0a \
--destination-prefix-list-id pl-0123 \
--transit-gateway-id tgw-0abc
aws ec2 authorize-security-group-ingress \
--group-id sg-0a --ip-permissions \
'IpProtocol=tcp,FromPort=443,
ToPort=443,PrefixListIds=
[{PrefixListId=pl-0123}]'Appliances advertise routes over BGP; Route Server installs the best next hop in your route tables and fails over automatically.
Wrong answers that look right.
0/0 → igw, they are public by accident.Scenario → answer → why.
| Scenario | Answer | Why |
|---|---|---|
NAT gateway shows ErrorPortAllocation when calling one SaaS endpoint | Add secondary IPs to the NAT gateway (up to 8 IPs: 1 primary + 7 secondary), or regional mode | Each IP adds 55,000 connections per destination |
| All inbound internet traffic must pass a third-party IDS before reaching web servers | IGW gateway route table → appliance ENI / GWLB endpoint | Ingress routing steers traffic entering the VPC |
| Active/standby firewall instances need sub-second route failover without Lambda | VPC Route Server with BGP + BFD | Installs next hop dynamically from BGP |
| Private subnets in 3 AZs; must survive an AZ loss with least route management | Regional NAT gateway | One ID, expands to every AZ with workloads |
| Allow SSH only from 40 office CIDRs, used by 30 security groups | Customer-managed prefix list referenced by the SGs | Edit once; but 40 max entries count as 40 rules per SG |
Stateful vs stateless filtering, connection tracking, SG referencing and VPC Block Public Access.
Know the mechanics, not just "stateful vs stateless".
| Security group | Network ACL | |
|---|---|---|
| Attached to | ENI (instance, RDS, ALB, Lambda, endpoint…) | Subnet (every ENI in it) |
| State | Stateful: replies allowed by connection tracking | Stateless: allow the return direction explicitly |
| Rules | Allow only; all rules evaluated together | Allow and deny; numbered, lowest first, first match wins |
| Default | New SG: no inbound, all outbound | Default NACL allows all; a new custom NACL denies all |
| Sources | CIDR, prefix list, other SGs | CIDR only |
| Quotas (default) | 60 inbound + 60 outbound per SG (IPv4 and IPv6 counted separately); 5 SGs per ENI (max 16); rules × SGs ≤ 1,000 | 20 rules per direction (max 40) |
| Use for | Primary micro-segmentation | Coarse subnet guardrail, explicit deny of a range, cutting live connections |
Inbound: subnet NACL in → SG in. Reply: SG (tracked, allowed) → NACL out (must allow ephemeral ports).
The reply goes to the client's source port. NACLs must allow the range the client actually uses.
| Client | Ephemeral range |
|---|---|
| Many Linux kernels (incl. Amazon Linux) | 32768–61000 |
| Windows Server 2008+ | 49152–65535 |
| NAT gateway | 1024–65535 |
| Elastic Load Balancing | 1024–65535 |
| Safe NACL rule | 1024–65535 |
Tracked flows keep working until they go idle. Untracked flows stop immediately.
--connection-tracking-specification on the ENI or at launch.conntrack_allowance_exceeded.An SG rule can name another SG as the source: "allow 5432 from members of sg-app". Membership changes, rules do not.
| Path | Notes |
|---|---|
| Same VPC | Always |
| Shared VPC (RAM) | Participants can reference SGs in the shared VPC |
| VPC peering | Same Region only; peer SG by ID |
| Transit Gateway (Sep 2024) | Enable on the TGW and the VPC attachments |
| Cloud WAN (Jun 2025) | Enable on the core network attachments |
Each tier accepts traffic only from the tier above, by SG ID.
An account-level (or Organization declarative policy) control that overrides route tables and SGs for IGW / egress-only IGW traffic.
aws ec2 \
modify-vpc-block-public-access-options \
--internet-gateway-block-mode \
block-bidirectional
aws ec2 \
create-vpc-block-public-access-exclusion \
--subnet-id subnet-0pub \
--internet-gateway-exclusion-mode \
allow-bidirectional
aws ec2 \
describe-vpc-block-public-access-options
# test first with Network Access
# Analyzer / Reachability AnalyzerWrong answers that look right.
Scenario → answer → why.
| Scenario | Answer | Why |
|---|---|---|
| Block a malicious /24 from reaching a public subnet now | NACL deny rule with a low rule number | SGs cannot deny; NACL also cuts live flows |
| DB must accept only from app instances that scale in and out | SG rule source = app tier SG ID | Membership-based, no IP maintenance |
| Security team: no VPC in any account may ever be internet-reachable, except one egress VPC | VPC BPA bidirectional via declarative policy + exclusion for the egress VPC | Organization-wide preventive control |
Firewall cluster drops connections under load; conntrack_allowance_exceeded rising | Lower TCP established timeout, bigger instances, or untracked rules | Conntrack table full |
| App in spoke VPC must allow traffic only from SG of a shared-services VPC via TGW | Enable SG referencing on TGW and attachments; inbound rule by SG ID | Supported same Region, inbound only |
ENIs, prefix delegation, ENA / ENA Express / EFA, bandwidth limits and MTU.
An ENI carries IPs, MAC, security groups and source/dest check. It lives in one subnet and one AZ.
Assign an IPv4 /28 (16 addresses) or IPv6 /80 to an ENI instead of single secondary IPs.
ENABLE_PREFIX_DELEGATION=true raises max pods per node.# reserve space in the subnet for /28 prefixes
aws ec2 \
create-subnet-cidr-reservation \
--subnet-id subnet-0app \
--cidr 10.20.48.0/20 \
--reservation-type prefix
# assign two /28 prefixes to an ENI
aws ec2 assign-private-ip-addresses \
--network-interface-id eni-0abc \
--ipv4-prefix-count 2
# EKS
kubectl set env ds aws-node \
-n kube-system \
ENABLE_PREFIX_DELEGATION=trueBy default an ENI drops packets that are neither from nor to its own IPs. Routers, NAT instances and firewalls forward other hosts' traffic.
GWLB-based and Network Firewall designs avoid this by using endpoints instead of instance ENIs as route targets.
All current-generation instances use ENA. ENA Express and EFA are add-ons for specific workloads.
| ENA | ENA Express | EFA | |
|---|---|---|---|
| What | Enhanced networking (SR-IOV) driver, default on Nitro | ENA + AWS SRD transport: multipath, congestion-aware | ENA + OS-bypass (libfabric) for HPC/ML |
| Single-flow bandwidth | 5 Gbps (10 Gbps in a cluster placement group) | Up to 25 Gbps (same Region, up to instance limit) | OS-bypass traffic, very low latency |
| Requirements | Nitro instance, ENA driver | Supported large types on both sides, enabled on both attachments, no middlebox in path | Supported types, EFA ENI at launch, MPI / NCCL apps |
| Traffic | Everything | TCP; UDP optional; same or cross-AZ in a Region | MPI, NCCL collective ops |
| Use for | General workloads | Big data shuffles, storage, media between instances | Tightly coupled HPC, distributed training |
ENA Express falls back to plain ENA silently if either side is not configured or the path crosses a middlebox (NLB, NAT, firewall).
Published "up to N Gbps" is the best case. Several other ceilings apply first.
bw_in_allowance_exceeded / bw_out_allowance_exceededpps_allowance_exceededconntrack_allowance_exceeded, conntrack_allowance_availablelinklocal_allowance_exceeded (DNS / IMDS / NTP)ethtool -S eth0; publish via CloudWatch agent.Inside a VPC the MTU is 9001. Every path that leaves the VPC is smaller.
| Path | MTU |
|---|---|
| Within a VPC, intra-Region peering | 9001 |
| Direct Connect private VIF | 9001 (when enabled) |
| Transit Gateway (VPC, DX, peering attachments) | 8500 |
| Inter-Region VPC peering | 8500 |
| Direct Connect transit VIF | 8500 (when enabled) |
| NAT gateway | 8500 |
| Internet gateway, public VIF, between Regions without peering or TGW | 1500 |
| Site-to-Site VPN (inside the tunnel) | Max 1446 / MSS 1406; 1422 / 1382 with AES-CBC + NAT-T |
ip route … mtu 1500) or a second ENI.Classic symptom: SSH and small HTTP work; large downloads or TLS handshakes with big certificates hang.
ip link show eth0 | grep mtu # 9001 on most Nitro instances
tracepath -n 10.50.0.20 # reports pmtu per hop
# largest unfragmented payload: MTU - 28 (IPv4 + ICMP headers)
ping -M do -s 8972 10.50.0.20 # tests 9000-byte packets
ping -M do -s 1472 10.50.0.20 # tests 1500-byte packets
# fix for one destination (e.g. on-prem via VPN)
sudo ip route add 10.50.0.0/16 via 10.0.1.1 mtu 1500
# or the whole interface
sudo ip link set dev eth0 mtu 1500/etc/dhcp/dhclient.conf / systemd-networkd, not just ip link.Wrong answers that look right.
Scenario → answer → why.
| Scenario | Answer | Why |
|---|---|---|
| HPC job needs lowest inter-node latency for MPI | EFA + cluster placement group | OS-bypass and close physical placement |
| Large file copy between two m7i.16xlarge in different AZs tops out at 5 Gbps | Parallel streams, or enable ENA Express on both | Single-flow limit; SRD raises it to 25 Gbps |
| EKS nodes hit max pods long before CPU | VPC CNI prefix delegation | /28 per slot multiplies IPs per ENI |
| Uploads to on-premises over VPN hang, small requests fine | Allow ICMP fragmentation-needed; set MTU/MSS for the VPN path | VPN tunnels carry at most 1446-byte packets (AES-GCM, no NAT-T) and do not support PMTUD; clamp MSS to ≤1406 (1382 with AES-CBC + SHA-256 + NAT-T) |
| Licensed app is tied to a MAC address and must survive instance failure | Secondary ENI moved to a standby instance | ENI keeps its MAC and IPs |
DHCP option sets, DNS attributes, the Route 53 VPC Resolver, hostnames and link-local services.
One option set per VPC at a time. Option sets are immutable: create a new one and associate it.
| Option | Meaning |
|---|---|
domain-name | Search suffix (e.g. corp.example.com). Use a domain you control; one value for Windows. |
domain-name-servers | AmazonProvidedDNS or up to 4 IPv4 + 4 IPv6 custom servers. Do not mix both. |
ntp-servers | Up to 4 IPv4 + 4 IPv6; Amazon Time Sync is 169.254.169.123 / fd00:ec2::123. |
netbios-name-servers, netbios-node-type | Windows; node type 2 (P-node) recommended. |
ipv6-address-preferred-lease-time | 140 s default, up to ~68 years. |
domain-name-servers to on-premises DNS bypasses Route 53 private zones — use Resolver endpoints instead (Chapter 5).enableDnsSupport and enableDnsHostnamesTwo VPC attributes control whether the Amazon DNS server works and whether instances get DNS names.
| enableDnsSupport | enableDnsHostnames | Result |
|---|---|---|
| true | true | Resolver works; instances with public IPs get public DNS names; private hosted zones resolve. Needed for PHZ and interface endpoint private DNS. |
| true | false | Resolver works; no public DNS hostnames for instances. |
| false | any | No Amazon DNS server at VPC+2 or 169.254.169.253. Custom DNS only. |
enableDnsSupport true, enableDnsHostnames false.aws ec2 modify-vpc-attribute --vpc-id vpc-0a --enable-dns-hostnames "{\"Value\":true}"Renamed from "Route 53 Resolver" in Mar 2026. Same addresses, same behaviour.
169.254.169.253, and fd00:ec2::253 (Nitro).linklocal_allowance_exceeded. Cache locally (systemd-resolved, nscd, NodeLocal DNSCache in EKS).Subnet setting --private-dns-hostname-type-on-launch decides the guest OS hostname for new instances.
| IP name | Resource name | |
|---|---|---|
| Format | ip-10-0-1-20.ap-northeast-1.compute.internal | i-0123456789abcdef0.ap-northeast-1.compute.internal |
| IPv6-only subnets | Not available | Required |
| Changes when IP changes | Yes | No |
| DNS records | A record | Optional A and AAAA records (enable-resource-name-dns-a/aaaa-record) |
us-east-1 uses ec2.internal instead of <region>.compute.internal.ec2-203-0-113-25.compute-1.amazonaws.com) resolve to the private IP inside the VPC and the public IP outside.They never leave the host; they share the 1,024 pps limit per ENI.
| Service | IPv4 | IPv6 (Nitro) |
|---|---|---|
| Instance Metadata Service | 169.254.169.254 | fd00:ec2::254 |
| Amazon DNS (VPC Resolver) | 169.254.169.253 (and VPC+2) | fd00:ec2::253 |
| Amazon Time Sync (NTP) | 169.254.169.123 | fd00:ec2::123 |
PUT /latest/api/token with X-aws-ec2-metadata-token-ttl-seconds (up to 21,600 s).HttpTokens=required) to stop SSRF-based credential theft.TOKEN=$(curl -sX PUT http://169.254.169.254/latest/api/token -H "X-aws-ec2-metadata-token-ttl-seconds: 300")
curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/network/interfaces/macs/
Wrong answers that look right.
enableDnsSupport and enableDnsHostnames on the VPC.Scenario → answer → why.
| Scenario | Answer | Why |
|---|---|---|
Instances must use corp.example.com as search domain and still resolve a private hosted zone | Option set: domain-name corp.example.com, servers AmazonProvidedDNS | Keeps the VPC Resolver in the path |
| Interface endpoint private DNS name does not resolve in a new VPC | Enable enableDnsHostnames (and DNS support) | Private DNS for endpoints relies on PHZs |
DNS timeouts on a log-shipping fleet under load; linklocal_allowance_exceeded rising | Local DNS cache on each host (or NodeLocal DNSCache) | 1,024 pps per ENI limit is fixed |
| IPv6-only subnet; instances need stable names | Resource-based hostnames with AAAA records | IP names are unavailable in IPv6-only subnets |
| Pods must not read node credentials from IMDS | IMDSv2 required + hop limit 1; use Pod Identity / IRSA | Extra hop blocks pod access |
Explain the answer out loud and name the exact setting or service.
Tasks 1.6, 3.1, 3.3 — reach AWS services, partner services and your own services without the internet.
Gateway endpoints, interface endpoints, endpoint policies, and private DNS for AWS services.
An instance in a private subnet that calls an AWS API uses the public service endpoint. The path decides cost and exposure.
Endpoints keep traffic on the AWS network, remove NAT data processing charges (gateway type), and let you write policies on aws:SourceVpce / aws:SourceVpc.
Three different mechanisms behind one console page.
| Gateway endpoint | Interface endpoint (PrivateLink) | GWLB endpoint | |
|---|---|---|---|
| Services | S3 and DynamoDB only | Most AWS services, S3 too, partner and your own services | Your inspection appliances behind a GWLB |
| Mechanism | Route table entry: AWS prefix list → vpce-… | ENI with a private IP in each chosen subnet | ENI used as a route target (next hop) |
| DNS | No change: public name, public IPs, routed privately | Regional + zonal names; optional private DNS overrides the public name | None — it is a routing target |
| Reach from on-premises / peered VPC / TGW | No | Yes (routed to the ENI IPs) | Via routing design |
| Security group | No (use the prefix list in instance SGs) | Yes, on the endpoint ENI | No |
| Cost | No charge | Per hour per AZ + per GB processed | Per hour per AZ + per GB processed |
| Quota (default) | 20 per Region (adjustable), max 255 per VPC | 50 interface + GWLB endpoints per VPC (adjustable) | (shared with interface) |
Free, simple, and limited to the VPC that owns them.
com.amazonaws.<region>.s3), target = vpce-….0.0.0.0/0 → NAT.aws:SourceVpce (one endpoint) or aws:SourceVpc (any endpoint in a VPC).aws:SourceIp does not work through an endpoint: S3 sees the private IP. Use aws:VpcSourceIp.s3:ResourceAccount, bucket ARNs).aws ec2 create-vpc-endpoint --vpc-id vpc-0abc --vpc-endpoint-type Gateway \
--service-name com.amazonaws.ap-northeast-1.s3 \
--route-table-ids rtb-0priv-a rtb-0priv-c --policy-document file://s3-endpoint-policy.json
One ENI per selected subnet (one subnet per AZ). The service is reached at that private IP.
vpce-0123-abcd.ec2.ap-northeast-1.vpce.amazonaws.com → all ENI IPs.vpce-0123-abcd-ap-northeast-1a.ec2… → the ENI in that AZ only (keeps traffic AZ-local, avoids cross-AZ charges).ec2.ap-northeast-1.amazonaws.com resolve to the ENIs inside the VPC.enableDnsSupport and enableDnsHostnames on the VPC.Same name, different answer depending on who asks.
From outside the VPC, either forward the service domain to a Resolver inbound endpoint, or use the endpoint-specific vpce-… name. For S3, "private DNS only for inbound endpoint" sends on-premises clients to the interface endpoint and keeps VPC clients on the free gateway endpoint.
It does not grant anything; it is one more policy that must allow the request (with IAM and the resource policy).
{
"Version": "2012-10-17",
"Statement": [{
"Sid": "OnlyOurOrgBuckets",
"Effect": "Allow",
"Principal": "*",
"Action": ["s3:GetObject", "s3:PutObject", "s3:ListBucket"],
"Resource": "*",
"Condition": {
"StringEquals": {
"aws:PrincipalOrgID": "o-a1b2c3d4e5",
"aws:ResourceOrgID": "o-a1b2c3d4e5"
}
}
}]
}aws:PrincipalOrgID, aws:ResourceOrgID). Stops data exfiltration to an attacker’s bucket.aws:SourceVpce is ours. Stops use of stolen credentials from outside.The consumer creates the endpoint in its own Region and names the service Region; AWS carries the traffic over its backbone.
--service-region.aws ec2 create-vpc-endpoint --vpc-id vpc-0abc --vpc-endpoint-type Interface \
--service-name com.amazonaws.vpce.us-east-1.vpce-svc-0123456789abcdef0 --service-region us-east-1 \
--subnet-ids subnet-a subnet-c --security-group-ids sg-0endpoint
Hundreds of VPCs × dozens of services × 3 AZs becomes expensive. Centralize, then make DNS point at the hub.
Turn private DNS off on the hub endpoints, create one private hosted zone per service name with an alias to the endpoint, and associate the zones with every spoke VPC. Trade-off: TGW processing charges per GB vs per-VPC endpoint hours.
Wrong answers that look right.
aws:SourceIpaws:SourceIp = VPC CIDR never matches through an endpoint. Use aws:VpcSourceIp, aws:SourceVpc or aws:SourceVpce.* does not give a principal any permission it did not have.Scenario → answer → why.
| Scenario | Answer | Why |
|---|---|---|
| Private subnets download large S3 objects; NAT bill is high | S3 gateway endpoint on the private route tables | Free, removes NAT processing |
| On-premises servers must upload to S3 over Direct Connect privately | S3 interface endpoint + Resolver inbound endpoint (or endpoint DNS names) | Gateway endpoints are not reachable from outside the VPC |
| Stop developers copying data to personal buckets from the VPC | Endpoint policy with aws:ResourceOrgID / s3:ResourceAccount | Restricts which resources the path can reach |
| Bucket must only be readable from one VPC | Bucket policy deny unless aws:SourceVpc matches | Identity-independent network condition |
| 100 VPCs each need 15 interface endpoints | Central endpoints in a shared-services VPC + PHZs associated with spokes | Cuts endpoint hours; DNS steers to the hub |
| App in AZ a must not pay cross-AZ for endpoint traffic | Use the zonal endpoint DNS name | Resolves only to the ENI in that AZ |
Publish a service behind an NLB or GWLB to other VPCs and accounts — one-way and overlap-tolerant.
The consumer gets an ENI in its own VPC; the provider sees connections from its NLB. Nothing else is routable.
Provider side, then consumer side. The NLB must be in the AZs the consumer wants to use.
# provider (account A)
aws ec2 create-vpc-endpoint-service-configuration \
--network-load-balancer-arns $NLB_ARN \
--acceptance-required \
--supported-ip-address-types ipv4 \
--private-dns-name api.example.com
aws ec2 modify-vpc-endpoint-service-permissions \
--service-id vpce-svc-0123 \
--add-allowed-principals arn:aws:iam::222233334444:root
# consumer (account B)
SVC=com.amazonaws.vpce.ap-northeast-1.vpce-svc-0123
aws ec2 create-vpc-endpoint --vpc-endpoint-type Interface \
--vpc-id vpc-0consumer --service-name $SVC \
--subnet-ids subnet-a subnet-c --security-group-ids sg-0ep
# provider accepts
aws ec2 accept-vpc-endpoint-connections \
--service-id vpce-svc-0123 --vpc-endpoint-ids vpce-0abc* with auto-accept makes the NLB effectively public.Without it consumers use vpce-… names or create their own zone.
api.example.com (wildcards allowed, lowercase).example.com.start-vpc-endpoint-service-private-dns-verification; status becomes verified.api.example.com resolves to their ENIs.enableDnsSupport and enableDnsHostnames.api.example.com to a Resolver inbound endpoint.The NLB performs source NAT for PrivateLink traffic; the client IP is hidden unless you ask for it.
vpce-… in a TLV).PrivateLink can now reach a single resource (a database, an IP, a domain) without an NLB, or a whole Lattice service network.
| Endpoint type | Connects to | Provider builds | Typical use |
|---|---|---|---|
| Interface | An endpoint service (NLB / GWLB) or an AWS service | NLB + endpoint service | SaaS APIs, AWS APIs |
| Resource | One resource configuration: IP, domain-name target, or RDS database | Resource gateway + resource configuration, shared via RAM | Give a partner access to one database port without an NLB |
| Service network | A VPC Lattice service network (its services and resources) | Lattice service network | On-premises or TGW-connected clients reaching Lattice services |
| Gateway Load Balancer | A GWLB endpoint service | GWLB + appliances | Transparent inspection |
Quotas: 200 resource endpoints and 50 service-network endpoints per VPC by default (both adjustable).
PrivateLink exposes one service; peering and TGW connect whole networks.
| Requirement | PrivateLink | VPC peering | Transit Gateway |
|---|---|---|---|
| Overlapping CIDRs | Works | Not possible | Only with NAT tricks |
| Consumer initiates only | Yes, by design | No — both sides routable | No — routing decides |
| Hundreds of consumers / SaaS | Yes | Quota-bound (50, up to 125 per VPC) | Yes, but full network access |
| Many ports and protocols, east-west | TCP via NLB listeners only | Yes | Yes |
| Cost model | Endpoint hours + per GB | Data transfer only | Attachment hours + per GB |
VPC Lattice (beyond the exam guide), AWS Cloud Map, and life after App Mesh.
A managed L7 proxy (and L4 for resources) between clients in associated VPCs and services anywhere in the organization.
Components, traffic path and security layers.
Auth type AWS_IAM requires clients to sign requests with SigV4 (SDKs, or a sidecar/proxy).
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::111122223333:role/orders-task"
},
"Action": "vpc-lattice-svcs:Invoke",
"Resource":
"arn:aws:vpc-lattice:*:444455556666:service/svc-0b/*",
"Condition": {
"StringEquals": {
"vpc-lattice-svcs:RequestMethod": "POST" },
"StringLike": {
"vpc-lattice-svcs:RequestPath": "/charge*" }
}
}]
}aws vpc-lattice create-service-network --name prod --auth-type AWS_IAM
aws vpc-lattice create-service --name billing --auth-type AWS_IAM
aws vpc-lattice create-service-network-service-association \
--service-network-identifier sn-0prod \
--service-identifier svc-0billing
aws vpc-lattice create-service-network-vpc-association \
--service-network-identifier sn-0prod \
--vpc-identifier vpc-0client \
--security-group-ids sg-0lattice
aws vpc-lattice put-auth-policy \
--resource-identifier svc-0billing \
--policy file://auth.jsonA registry of service instances, discoverable by DNS or by API, with health status.
DiscoverInstances).UpdateInstanceCustomHealthStatus).App Mesh support ended on 30 Sep 2026; it is still on the ANS-C01 in-scope list.
| Need | Best fit |
|---|---|
| L7 routing, auth per path, many services across accounts | VPC Lattice |
| Expose one TCP service to many consumers, overlap-safe | PrivateLink endpoint service |
| Full network reachability, many protocols, inspection | Transit Gateway / Cloud WAN |
| ECS-only service mesh features (retries, metrics) | ECS Service Connect |
| One database port to a partner | Resource endpoint (Lattice resource configuration) |
Scenario → answer → why.
| Scenario | Answer | Why |
|---|---|---|
| SaaS provider must serve 400 customer VPCs with overlapping CIDRs | NLB + endpoint service, allowed principals per customer | One-way, overlap-tolerant, scales without routing |
| Provider app must log the real client IP behind PrivateLink | Proxy protocol v2 on the NLB target group | NLB hides the source IP otherwise |
Consumers want api.vendor.com to resolve privately | Private DNS name on the endpoint service + TXT verification | Domain ownership must be proven |
| Removing a customer from allowed principals did not cut them off | Reject the existing endpoint connection | Permission changes affect only new connections |
| Microservices in 30 accounts need per-path IAM authorization, no routing changes | VPC Lattice with service-level auth policies | L7 auth without TGW or peering |
| ECS team is still on App Mesh in Oct 2026 | Migrate to ECS Service Connect | App Mesh support ended 30 Sep 2026 |
Explain the answer and name the exact feature.
aws:SourceIp fail behind an endpoint?Tasks 1.6, 2.2, 3.1, 3.3 — peering, Transit Gateway, Cloud WAN, sharing, and overlapping IP ranges.
A direct, non-transitive link between two VPCs: cheapest and simplest, until the mesh grows.
A peers with B and with C. B cannot reach C through A, and B cannot use A’s gateways.
Unicast reverse path forwarding is not supported on peering connections: if a reply would come back over a different path, it is dropped.
Most exam distractors break one of these rules.
Four steps; forgetting the route on one side is the classic outage.
# account A (requester), Region ap-northeast-1
aws ec2 create-vpc-peering-connection --vpc-id vpc-0aaa \
--peer-vpc-id vpc-0bbb --peer-owner-id 222233334444 \
--peer-region ap-southeast-1
# account B (accepter), Region ap-southeast-1
aws ec2 accept-vpc-peering-connection \
--vpc-peering-connection-id pcx-0123 --region ap-southeast-1
# both sides: route + DNS option
aws ec2 create-route --route-table-id rtb-0a \
--destination-cidr-block 10.2.0.0/16 \
--vpc-peering-connection-id pcx-0123
aws ec2 modify-vpc-peering-connection-options \
--vpc-peering-connection-id pcx-0123 \
--requester-peering-connection-options \
AllowDnsResolutionFromRemoteVpc=truen VPCs in a full mesh need n(n−1)/2 connections. Beyond a handful, the hub wins on operations.
| Peering | Transit Gateway | |
|---|---|---|
| Topology | Point-to-point | Hub-and-spoke, transitive |
| Hourly charge | None | Per attachment |
| Data processing | None (data transfer only; free within the same AZ) | Per GB processed |
| Bandwidth | No aggregate limit (instance limits apply) | Up to 100 Gbps per VPC attachment per AZ |
| MTU | 9001 (8500 inter-Region) | 8500 |
| Central inspection | No | Yes (appliance mode) |
| Max scale | 125 per VPC | 5,000 attachments per TGW |
A Regional router you program with route tables: attachments, association, propagation, segmentation and inspection.
Every packet enters on an attachment, is looked up in that attachment’s associated route table, and leaves on another attachment.
The TGW is reachable only from AZs where the attachment has a subnet.
Best practice: a dedicated small subnet per AZ for attachment ENIs, with its own route table and a permissive NACL. Traffic from an AZ without an attachment subnet cannot reach the TGW; traffic to an attachment missing the source AZ is sent to another AZ at no extra TGW charge.
Longest prefix first; for the same prefix, the attachment type decides; within one type, BGP attributes.
Associate by "who am I", propagate by "who may reach me".
Read each row as "attachments associated here can reach…".
| Destination | Target | Type |
|---|---|---|
| 10.1.0.0/16 | Prod A attachment | propagated |
| 10.2.0.0/16 | Prod B attachment | propagated |
| 10.100.0.0/16 | Shared attachment | propagated |
| 192.168.0.0/16 | VPN attachment | propagated |
| 10.3.0.0/16 | — | blackhole (explicit) |
| Table | Routes |
|---|---|
| RT-dev (Dev C) | 10.3.0.0/16 → Dev C; 10.100.0.0/16 → Shared |
| RT-shared (Shared, VPN) | All VPC CIDRs propagated + on-premises from VPN |
| Spoke VPC route tables | 10.0.0.0/8 and 192.168.0.0/16 → tgw-… |
aws ec2 create-transit-gateway --options DefaultRouteTableAssociation=disable,DefaultRouteTablePropagation=disable,VpnEcmpSupport=enable
aws ec2 associate-transit-gateway-route-table --transit-gateway-route-table-id tgw-rtb-prod --transit-gateway-attachment-id tgw-attach-a
aws ec2 enable-transit-gateway-route-table-propagation --transit-gateway-route-table-id tgw-rtb-shared --transit-gateway-attachment-id tgw-attach-a
Without appliance mode the TGW keeps each direction in its originating AZ — the reply may hit a firewall that never saw the request.
Enable it on the inspection VPC attachment only (--options ApplianceModeSupport=enable). Use exactly one TGW per inspection VPC: TGWs do not share flow state. Native Network Firewall attachments have appliance mode on automatically.
Spokes send everything to the inspection VPC; the inspection VPC sends traffic back to the TGW after the firewall.
| Table | Associated | Routes |
|---|---|---|
| RT-spokes | All spoke VPCs | 0.0.0.0/0 → inspection attachment (static) |
| RT-inspection | Inspection VPC | All spoke CIDRs propagated; 0.0.0.0/0 → egress VPC or local NAT |
| Subnet | Routes |
|---|---|
| TGW attachment subnet (per AZ) | 0.0.0.0/0 → firewall endpoint in the same AZ |
| Firewall subnet (per AZ) | 10.0.0.0/8 → tgw-…; 0.0.0.0/0 → NAT gateway |
| Public subnet (NAT) | 10.0.0.0/8 → firewall endpoint (return path); 0.0.0.0/0 → IGW |
With the native Network Firewall attachment (Jun 2025) the inspection VPC disappears: point routes in the TGW route tables at the network function attachment (static routes only). Details in Chapter 7.
Peering connects TGWs over the AWS backbone; routing across it is static only.
Limits: no BGP and no ECMP over peering; one peering between the same two TGWs; no PMTUD on peering, VPN or DX attachments (the TGW clamps MSS); SG referencing does not cross Regions. Many Regions? Cloud WAN builds the mesh and dynamic routing for you.
Features with a specific exam trigger.
Wrong answers that look right.
Scenario → answer → why.
| Scenario | Answer | Why |
|---|---|---|
| Two VPCs exchange 200 TB/month in the same AZ; cost must be minimal | VPC peering | No hourly or processing charge; free within the AZ |
| 80 VPCs and two data centers; dev must never reach prod | TGW with separate prod/dev route tables, no cross propagation | Transitive hub with segmentation |
| Inspection VPC firewalls drop return traffic intermittently | Enable appliance mode on the inspection VPC attachment | Keeps both directions in one AZ |
| Same prefix learned over DX and VPN; VPN should carry 10.50.0.0/24 | Advertise 10.50.0.0/24 over VPN (longer prefix) | Type priority prefers DX for equal prefixes |
| Connect TGWs in Tokyo and Virginia | Inter-Region TGW peering with static routes | No BGP over peering |
| Need more than 1.25 Gbps over IPsec to the TGW from one site | Multiple VPN tunnels with BGP + ECMP (or 5 Gbps tunnels, Ch 4) | ECMP needs dynamic routing |
Beyond the exam guide — a global network defined by a policy document instead of per-Region route tables.
One core network spans Regions; each Region has a core network edge (CNE); segments are global isolated routing domains.
Policies are versioned; each change produces a change set you review and then execute (rollback = execute an older version).
{
"version": "2021.12",
"core-network-configuration": {
"asn-ranges": ["64520-64530"],
"edge-locations": [{"location": "ap-northeast-1"},
{"location": "us-east-1"}]
},
"segments": [
{"name": "prod", "require-attachment-acceptance": true},
{"name": "dev", "isolate-attachments": false},
{"name": "hybrid"}
],
"segment-actions": [
{"action": "share", "mode": "attachment-route",
"segment": "hybrid", "share-with": ["prod", "dev"]}
],
"attachment-policies": [
{"rule-number": 100,
"conditions": [{"type": "tag-value", "operator": "equals",
"key": "env", "value": "prod"}],
"action": {"association-method": "constant",
"segment": "prod"}}
]
}isolate-attachments blocks traffic between attachments of the same segment.share routes between segments, create-route static routes, send-via / send-to steer traffic through a network function group.send-via / send-to.Attachments land in a Region’s CNE and are mapped to a segment by the attachment policy.
| Attachment | Notes |
|---|---|
| VPC | Subnet per AZ, like TGW; appliance mode option; tags drive segment mapping |
| Site-to-Site VPN | Dynamic routing; 5 Gbps large-bandwidth tunnels supported (Nov 2025) |
| Connect | GRE or tunnel-less (Oct 2023, BGP straight to the appliance) over a VPC transport |
| Transit Gateway route table | Via TGW ↔ CNE peering; lets existing TGW estates join gradually |
| Direct Connect gateway | Native attachment; DX routes propagate into segments without a TGW |
Security group referencing across Cloud WAN VPC attachments arrived in Jun 2025 (same Region, inbound rules).
Both are valid; the choice is about scale and how you want to operate.
| Factor | TGW + peering | Cloud WAN |
|---|---|---|
| Regions | One TGW per Region, peer in a mesh | One core network, CNE per Region |
| Inter-Region routing | Static routes you maintain | Dynamic, automatic |
| Segmentation | Route tables per TGW, repeated per Region | Global segments in one policy |
| Onboarding | Associate + propagate per attachment | Tag the attachment; policy maps it |
| Change control | API calls / IaC | Versioned policy, change sets, rollback |
| Exam (ANS-C01) | In scope | Not on the in-scope list |
Many accounts, one network: the owner runs routing, participants run workloads.
The network account owns the VPC and shares subnets through AWS RAM with accounts or OUs in the same organization.
Participants consume the network; they cannot change it.
account-id/sg-id.apne1-az1).aws ram enable-sharing-with-aws-organization # management account, once
aws ram create-resource-share --name prod-subnets \
--resource-arns arn:aws:ec2:ap-northeast-1:111122223333:subnet/subnet-0app-a \
--principals arn:aws:organizations::999988887777:ou/o-a1b2c3d4e5/ou-ab12-workloads
Build once in the network account, consume everywhere.
| Resource | What the consumer gets | Typical pattern |
|---|---|---|
| Subnets | Launch resources into the owner’s VPC | VPC sharing per environment |
| Transit Gateway | Create attachments from their own VPCs (owner may need to accept) | Central TGW in the network account |
| Customer-managed prefix lists | Reference in SGs and route tables; updates flow automatically | Corporate CIDRs, partner ranges |
| VPC Resolver rules / Route 53 Profiles | Forwarding rules or a bundle of DNS settings to associate with VPCs | Hybrid DNS (Chapter 5) |
| IPAM pools | Allocate CIDRs from a central plan | No overlapping VPCs (Chapter 1) |
| Network Firewall policies | Use a central policy on local firewalls | Distributed inspection |
| Cloud WAN core network, VPC Lattice service networks / services / resource configurations | Attach or associate across accounts | Global network, app networking |
Not RAM: Direct Connect gateways are shared across accounts with association proposals; VPC peering uses requests and acceptance.
Scenario → answer → why.
| Scenario | Answer | Why |
|---|---|---|
| 50 application accounts, one network team, apps need low-latency east-west in the same VPC | VPC sharing from a network account | Implicit local routing, central control, separate billing |
| Participant cannot add a route to the NAT gateway | Expected: only the owner manages route tables | Participants consume, owner configures |
| Every account must use the corporate CIDR list in SGs, kept current | Customer-managed prefix list shared with RAM | One update propagates everywhere |
| Five Regions, identical prod/dev isolation, frequent onboarding | Cloud WAN with segments and tag-based attachment policies | Global policy, automatic mapping |
| Team in account B must attach its VPC to the central TGW | Share the TGW with RAM; B creates the attachment | Cross-account attachments need the share |
| Subnet placement differs between accounts though both chose "AZ a" | Use AZ IDs | AZ names are mapped per account |
Acquisitions, partners and Kubernetes pods: how to connect networks that reuse the same addresses.
Routing cannot choose between two identical destinations; something must translate or avoid routing.
| Option | How | Good for | Cost / limits |
|---|---|---|---|
| PrivateLink | Expose a service via NLB endpoint service; consumer gets a local IP | Client → specific service, many consumers | One-way, TCP, per-service |
| VPC Lattice | Service network; no routing between VPCs | Service-to-service across accounts | L7 / TCP resources; not on the exam |
| Private NAT gateway + TGW | Translate a non-routable range to a small routable range | Many-to-one, initiated from the NATed side | NAT + TGW processing; inbound needs an LB |
| IPv6 | Give workloads globally unique IPv6 addresses | New builds, EKS IPv6 clusters | All components must support IPv6 |
| Re-IP | Renumber one side (IPAM to prevent recurrence) | Long-term fix after an acquisition | Project effort, downtime risk |
Prevent it first: allocate every VPC from Amazon VPC IPAM pools (Chapter 1).
Both VPCs reuse 10.0.0.0/16 internally; each also gets a small unique range that the TGW routes.
Routes: A non-routable subnet 100.64.2.0/24 → private NAT; A routable subnet 100.64.2.0/24 → TGW; TGW (propagation off) static 100.64.1.0/24 → A, 100.64.2.0/24 → B; B routable subnet 100.64.1.0/24 → TGW.
Same service as a public NAT gateway, without an Elastic IP or internet path.
Wrong answers that look right.
Scenario → answer → why.
| Scenario | Answer | Why |
|---|---|---|
| Acquired company uses the same 10.0.0.0/16; their app needs one API in our VPC | PrivateLink endpoint service for the API | Overlap-tolerant, least change |
| Many workloads in overlapping VPCs must call on-premises systems | Routable /24 per VPC + private NAT gateway + TGW static routes | Translates many-to-one into a unique range |
| Partner firewall only allows 100.64.10.0/28 | Private NAT gateway in a subnet from that range | Present allow-listed source IPs |
| EKS pods exhaust routable IPs in every cluster | Secondary non-routable CIDR for pods + custom networking (or IPv6) | Reuse the same pod range everywhere |
| Need permanent fix and avoid future overlaps | Re-IP one side; allocate all VPCs from IPAM pools | Removes the root cause |
Explain the answer and name the exact feature.
share, send-via and send-to do?Tasks 1.5, 2.1, 2.2, 3.1, 4.1, 4.3 — BGP, Site-to-Site VPN, Direct Connect, resiliency, SD-WAN and multicloud.
The one routing protocol every hybrid design depends on, and the AWS-specific knobs that steer it.
Every Direct Connect VIF, dynamic VPN tunnel and Connect peer is an eBGP session between your router and AWS.
Applies to virtual private gateway (VGW) designs. Longest prefix always wins first.
| # | Rule | Example |
|---|---|---|
| 1 | Longest prefix match | 10.10.2.0/24 beats 10.10.0.0/16 |
| 2 | Static routes | IGW, NAT, peering, TGW, ENI, gateway endpoint, VGW static |
| 3 | Prefix list routes | A route referencing a managed prefix list |
| 4a | Propagated: Direct Connect BGP | Learned over a private VIF on the VGW |
| 4b | Propagated: VPN static | Static routes on a VPN connection |
| 4c | Propagated: VPN BGP | Dynamic routes over the VPN tunnels |
Most specific first. For the same CIDR from different attachment types, a fixed order applies; within one type, BGP attributes decide.
| Priority | Route source |
|---|---|
| 1 | Static routes (including VPN static) |
| 2 | Prefix list referenced routes |
| 3 | VPC-propagated routes |
| 4 | Direct Connect gateway-propagated |
| 5 | Transit Gateway Connect-propagated |
| 6 | Site-to-Site VPN over private Direct Connect |
| 7 | Site-to-Site VPN-propagated |
| 8 | VPN Concentrator, then Client VPN, then TGW peering (Cloud WAN) |
On private and transit VIFs, AWS evaluates prefix length, then local preference communities, then AS_PATH, then MED.
| Order | Lever | How you set it | Use it for |
|---|---|---|---|
| 1 | Longest prefix | Advertise /24s on the primary VIF and the /16 on both | Strong active/passive; AWS recommends it |
| 2 | Local preference community | 7224:7300 high, 7224:7200 medium, 7224:7100 low on each prefix | Active/passive with identical prefixes; beats AS_PATH |
| 3 | AS_PATH length | Prepend your ASN on the backup path | When prefix and local pref are equal |
| 4 | MED | Lower MED preferred | Last resort; AWS does not recommend it |
| — | ECMP | Same prefix, same attributes (same AS_PATH length) on several VIFs | Active/active load sharing |
Default without communities: a Region prefers Direct Connect locations associated with that Region (treated as medium, 7224:7200); other locations get a lower value. That is why a remote DX location can lose to a local one even with a shorter AS_PATH.
Primary tagged high preference, backup tagged low. Your router mirrors the choice for the return direction.
Without the on-premises local preference, traffic to AWS can leave through Edge B while replies come back through Edge A. Stateful firewalls on the edges then drop it.
Communities 7224:1–7224:65535 are reserved by Direct Connect; unsupported communities on a public VIF are removed.
| Community | Meaning |
|---|---|
7224:7100 | Low local preference |
7224:7200 | Medium local preference |
7224:7300 | High local preference |
| same tag on all paths | ECMP active/active |
| Community | Your prefix reaches |
|---|---|
7224:9100 | Local AWS Region only |
7224:9200 | All Regions on the continent |
7224:9300 | All public Regions (global) |
| none | Global (default) |
| Community | AWS prefix originates in |
|---|---|
7224:8100 | Same Region as the DX location |
7224:8200 | Same continent |
| no tag | Other continents |
NO_EXPORT | On every route AWS advertises |
Filter on 7224:8100 to accept only local-Region AWS prefixes over a public VIF and send everything else to the internet. Local preference communities are evaluated before AS_PATH.
Default BGP hold time on Direct Connect is 90 seconds. BFD cuts detection to roughly 300 ms × 3.
| Setting | Value |
|---|---|
| Hold timer | default 90 s, minimum 3 s |
| Keepalive | default 30 s, minimum 1 s |
| BFD minimum interval | 300 ms |
| BFD minimum multiplier | 3 |
| Graceful restart timer | 120 s (do not combine with BFD) |
| BGP TTL | 1: no multihop BGP |
| MD5 authentication | Always on; key yours or AWS-generated |
Tag the primary with high preference, the backup with low; prefer the primary for outbound too.
router bgp 65010
neighbor 169.254.10.1 remote-as 64512 # VIF 1 (primary)
neighbor 169.254.10.1 password <md5-key>
neighbor 169.254.10.1 bfd
neighbor 169.254.20.1 remote-as 64512 # VIF 2 (backup)
neighbor 169.254.20.1 password <md5-key>
neighbor 169.254.20.1 bfd
address-family ipv4 unicast
network 10.20.0.0/16
neighbor 169.254.10.1 route-map TO-AWS-PRI out
neighbor 169.254.20.1 route-map TO-AWS-BAK out
neighbor 169.254.10.1 route-map FROM-AWS-PRI in
exit-address-family
!
route-map TO-AWS-PRI permit 10
set community 7224:7300 # high: AWS -> us via VIF 1
route-map TO-AWS-BAK permit 10
set community 7224:7100 # low: backup
route-map FROM-AWS-PRI permit 10
set local-preference 200 # us -> AWS via VIF 1show bgp summary: both neighbors Established.show bgp ipv4 unicast 10.0.0.0/16: best path via 169.254.10.1.aws directconnect describe-virtual-interfaces → bgpPeers[].bgpStatus = up.search-transit-gateway-routes shows the DX gateway attachment.Wrong answers that look right.
Scenario → answer → why.
| Scenario | Answer | Why |
|---|---|---|
| Two DX connections; all AWS → on-prem traffic must use connection 1 unless it fails; same prefixes on both | Tag prefixes on VIF 1 with 7224:7300 and VIF 2 with 7224:7100 | Local pref communities are the recommended active/passive lever |
| Same requirement, but the team may only change what it advertises, not tags | Advertise more specific prefixes on VIF 1, the summary on both | Longest prefix is evaluated first |
| Load share across two DX connections in different locations | Same prefixes, same community (e.g. 7224:7200), same AS_PATH | ECMP needs identical attributes |
| Public VIF: on-prem prefixes must be reachable only from the local Region | Tag with 7224:9100 | Scope communities limit propagation |
| Failover from DX to VPN takes over a minute | Enable BFD on the DX BGP sessions | Default hold time is 90 s; BFD detects in about 900 ms |
| On-prem router advertises 150 prefixes and the VIF keeps going down | Summarize routes (or request higher prefix allocation) | Default limit is 100 per session |
IPsec over the internet or over Direct Connect: tunnels, options, routing, scale, and remote-user access.
Each tunnel has its own AWS public IP in a different AZ. Configure both: AWS maintenance takes one down at a time.
Traffic from on-premises to AWS can use both tunnels. Traffic from AWS to on-premises prefers one tunnel (unless ECMP on Transit Gateway or Cloud WAN) and fails over automatically.
AWS evaluates proposals starting from the lowest configured value; restrict the lists with modify-vpn-tunnel-options.
| Option | Default | Allowed |
|---|---|---|
| Inside tunnel CIDR (IPv4) | Assigned by AWS | /30 from 169.254.0.0/16; 169.254.0.0/30–169.254.5.0/30 and 169.254.169.252/30 reserved |
| Inside tunnel CIDR (IPv6) | Assigned by AWS | /126 from fd00::/8 (TGW only) |
| Pre-shared key | Generated | 8–64 chars, alphanumeric, . and _, not starting with 0 (or certificate auth) |
| IKE versions | ikev1, ikev2 | Use ikev2 only where the device supports it |
| Phase 1 / Phase 2 lifetime | 28,800 s / 3,600 s | 900–28,800 s / 900–3,600 s (Phase 2 < Phase 1) |
| Rekey margin / fuzz | 270 s / 100% | 60 s to half of Phase 2 / 0–100% |
| Replay window | 1,024 packets | 64–2,048 |
| DPD timeout / action | 30 s / clear | ≥ 30 s / clear, none, restart |
| Startup action | add (your device initiates) | start: AWS initiates IKE |
Encryption: AES128, AES256, AES128-GCM-16, AES256-GCM-16. Integrity: SHA1, SHA2-256/384/512. DH groups 2 and 14–24 (Phase 2 also allows 5). Tunnel endpoint lifecycle control lets you schedule AWS maintenance replacements yourself.
Dynamic (BGP) is the default choice: failover, ECMP and route updates without tickets.
| Direction | VGW | TGW |
|---|---|---|
| CGW → AWS, dynamic | 100 | 1,000 |
| CGW → AWS, static | 100 | — |
| AWS → CGW | 1,000 | 5,000 |
Advertised routes on TGW come from the route table associated with the VPN attachment; on a VGW, from the VPC. Exceeding 100 routes on a VGW is a classic reason to move to a Transit Gateway.
A tunnel is the unit of scale. Go wider with ECMP, or bigger with Large Bandwidth Tunnels.
| Tunnel type | Bandwidth | PPS |
|---|---|---|
| Standard | up to 1.25 Gbps | up to 140,000 |
| Large Bandwidth Tunnel (Nov 2025) | up to 5 Gbps | up to 400,000 |
| VPN Concentrator tunnel | up to 100 Mbps | up to 10,000 |
Three options that change where the tunnel lands or how many sites share it.
| Variant | What it does | Rules and limits |
|---|---|---|
| Accelerated VPN | Two AWS-managed Global Accelerator accelerators (one per tunnel) bring traffic onto the AWS backbone at the nearest edge | Transit Gateway only; set at creation (cannot toggle later); NAT-T required; CGW must initiate IKE; not with a DX public VIF; not with LBT |
| VPN Concentrator (Nov 2025) | One TGW / Cloud WAN attachment aggregates many low-bandwidth sites | Up to 100 remote sites per concentrator; up to 100 Mbps per tunnel; no ECMP; aimed at 25+ small sites |
| Private IP VPN | IPsec over a Direct Connect transit VIF with private outside IPs (RFC 1918 or RFC 6598) | Needs TGW + DX gateway; the TGW CIDR block must be in the DX gateway allowed prefixes; higher route scale than DX alone |
| VPN over a public VIF | Standard VPN whose AWS endpoints are reached over a public VIF | Older pattern: exposes public endpoints; private IP VPN is preferred |
Private IP VPN solves "encrypt everything on Direct Connect without public IPs". MACsec (Part 4.3) is the alternative at layer 2, only on dedicated 10/100/400 Gbps connections.
Encrypted and unencrypted traffic can share the same transit VIF, using different TGW route tables.
Order of build: give the TGW a CIDR block → associate the DX gateway with the TGW and put that CIDR in allowed prefixes → create the VPN with outside IP type PrivateIpv4 and transport attachment = the DX gateway attachment.
CLI on the AWS side, strongSwan + FRR on a Linux "on-premises" router (used in Lab 4).
aws ec2 create-customer-gateway --type ipsec.1 \
--bgp-asn 65010 --ip-address 198.51.100.7
aws ec2 create-vpn-connection --type ipsec.1 \
--customer-gateway-id cgw-0abc \
--transit-gateway-id tgw-0def \
--options file://vpn-opts.json
# vpn-opts.json: StaticRoutesOnly=false,
# TunnelOptions[].TunnelInsideCidr =
# 169.254.11.0/30 and 169.254.12.0/30, IKEv2 only
aws ec2 describe-vpn-connections \
--vpn-connection-ids vpn-0123 --query \
'VpnConnections[0].VgwTelemetry[].[OutsideIpAddress,Status]'
# download a device config (strongSwan, Cisco, Palo Alto...)
aws ec2 get-vpn-connection-device-sample-configuration \
--vpn-connection-id vpn-0123 \
--vpn-connection-device-type-id <id># customer gateway (Linux), tunnel 1 of 2
# xfrm interface carries the BGP session
ip link add xfrm1 type xfrm dev ens5 if_id 1
ip addr add 169.254.11.2/30 dev xfrm1
ip link set xfrm1 mtu 1422 up # AES-CBC+SHA-256+NAT-T
iptables -t mangle -A FORWARD -o xfrm1 -p tcp \
--tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1382
# FRR (vtysh): BGP over the tunnel inside /30
router bgp 65010
neighbor 169.254.11.1 remote-as 64512
address-family ipv4 unicast
network 10.200.0.0/24
# strongSwan config: next slideOne key = value per line (the swanctl.conf grammar has no ; separator). if_id binds the child SA to xfrm1.
# /etc/swanctl/conf.d/aws.conf (tunnel 1)
connections {
aws1 {
version = 2
local_addrs = 10.200.0.10
remote_addrs = 203.0.113.10
proposals = aes256-sha256-modp2048
local {
auth = psk
id = 198.51.100.7
}
remote {
auth = psk
id = 203.0.113.10
}
children {
aws1 {
local_ts = 0.0.0.0/0
remote_ts = 0.0.0.0/0
if_id_in = 1
if_id_out = 1
start_action = start
dpd_action = restart
}
}
}
}secrets {
ike-aws1 {
id-1 = 198.51.100.7
id-2 = 203.0.113.10
secret = "<tunnel 1 pre-shared key>"
}
}
swanctl --load-all
swanctl --list-sas
# IKE_SA ESTABLISHED
# tunnel 2: copy as aws2 with the second
# outside IP, its PSK, if_id 2, xfrm2 and
# 169.254.12.2/30local_addrs is the private IP and the IKE id is the public IP AWS knows.esp_proposals can be added; the defaults cover the AWS tunnel proposals.0.0.0.0/0 selectors + if_id = route-based: BGP, not IPsec policy, picks the tunnel.Managed OpenVPN-based endpoint: users connect to a VPC, then reach peered VPCs, TGW, on-premises or the internet through it.
0.0.0.0/0 route via a subnet with NAT and an authorization rule.Most VPN outages are a mismatched proposal, a DPD timeout, or MTU.
TunnelState (1 up / 0 down), TunnelDataIn, TunnelDataOut.TunnelState max < 1 for either tunnel.Wrong answers that look right.
Scenario → answer → why.
| Scenario | Answer | Why |
|---|---|---|
| Need 4 Gbps of encrypted bandwidth to 30 VPCs over the internet | TGW + dynamic VPN with ECMP (or Large Bandwidth Tunnels) | Standard tunnels are 1.25 Gbps; ECMP needs BGP on TGW |
| Branch in another continent sees jitter on VPN to the Region | Accelerated Site-to-Site VPN on TGW | Enters the AWS backbone at the nearest edge |
| Compliance: all DX traffic must be IPsec-encrypted, no public IPs | Private IP VPN over transit VIF | IPsec with private outside addresses |
| 80 retail stores, each needing ~20 Mbps to AWS | VPN Concentrator on TGW | Built for many low-bandwidth sites, up to 100 per concentrator |
| VPN up, SSH works, file copies hang | Lower MTU / clamp MSS on the CGW | Packets over 1,446 bytes are dropped; no PMTUD |
| Remote staff need access to a VPC with Okta groups controlling which subnets | Client VPN with SAML + authorization rules per group | Group-based authorization on Client VPN |
A private circuit into AWS: ports, cross connects, LAGs, MACsec, virtual interfaces, and the DX gateway.
A Direct Connect location is a colocation facility where AWS has routers. You (or a partner) bring a fiber there.
A connection is layer 1–2 (a port and a fiber). A virtual interface (VIF) is layer 3: a VLAN plus a BGP session. Nothing flows until a VIF exists.
The physical steps are the slow part; plan weeks, not hours.
Billing starts when the port is active or 90 days after the LOA is issued, whichever comes first. Delete the port before then to avoid charges.
Hosted connections come from a Direct Connect Partner who already has a port at the location.
| Dedicated connection | Hosted connection | |
|---|---|---|
| Speeds | 1, 10, 100, 400 Gbps | 50, 100, 200, 300, 400, 500 Mbps; 1, 2, 5, 10, 25 Gbps (higher tiers only from qualified partners) |
| Who orders | You, in the console or API | The partner creates it; you accept it |
| VIFs | 50 private/public + up to 4 transit (51 total) | 1 VIF of any type |
| Change speed | No: create a new connection | Partner can upgrade or downgrade (if supported) |
| LAG | Yes | No |
| MACsec | 10, 100, 400 Gbps at selected locations | Not supported |
| Jumbo frames | Per VIF | Only if the partner’s parent connection has it |
| Rate behaviour | Port speed | Policed at the purchased rate: bursts are dropped |
"Need 200 Mbps quickly from a city without our own colo" → hosted connection. "Need 100 Gbps with MACsec" → dedicated.
A link aggregation group (LAG) uses LACP to make several dedicated ports act as one.
should_encrypt (default): falls back to clear if negotiation fails.must_encrypt: no MACsec, no traffic.Pick the VIF type by what you need to reach. Each is one VLAN plus one BGP session per address family.
| Private VIF | Transit VIF | Public VIF | |
|---|---|---|---|
| Reaches | One VPC via VGW, or VPCs via DX gateway + VGWs | Transit Gateways or a Cloud WAN core network via DX gateway | All AWS public prefixes (S3, DynamoDB, APIs, EC2 public IPs) |
| Addresses | Private (or public) prefixes | Private (or public) prefixes | Public prefixes you own (RIR-registered) |
| MTU | 1500 or 9001 | 1500 or 8500 | 1500 |
| Prefixes you → AWS | 100 default, up to 1,000 (prefix controls) | 100 default, up to 1,000 | 1,000 (fixed) |
| Prefixes AWS → you | VPC CIDRs (filtered by allowed prefixes) | Allowed prefixes; 200 per TGW (v4+v6) | All Region and edge prefixes (NO_EXPORT) |
| Per dedicated connection | 50 private + public combined | Up to 4 | Counts in the 50 |
| SiteLink | Yes, if via DX gateway | Yes | No |
BGP peer IPs: IPv4 /30 or /31 (you choose or AWS assigns); IPv6 /125 always assigned by Amazon. Multiple BGP sessions of the same address family on one VIF are not allowed.
A DX gateway is not in the data path. It is a distributed set of BGP route reflectors, so one is enough.
A DX gateway can associate VGWs and TGWs in any Region (and accounts) except China. Use private VIFs for VGW associations and transit VIFs for TGW associations.
A frequent exam detail: filter on one, advertisement on the other.
| Allowed prefix | VPC 10.0.0.0/16 advertised? |
|---|---|
22.0.0.0/24 | No route |
10.0.0.0/24 | No (narrower than the VPC CIDR) |
10.0.0.0/15 | Yes: 10.0.0.0/16 |
| Allowed prefix | You receive |
|---|---|
22.0.0.0/24 | 22.0.0.0/24 (even though no VPC uses it) |
10.0.0.0/24 | 10.0.0.0/24 |
10.0.0.0/8 | 10.0.0.0/8 (a summary) |
0.0.0.0/0 with two TGWs).Traffic to S3, DynamoDB or public endpoints goes over your circuit instead of the internet.
NO_EXPORT; do not leak them to the internet.7224:8100 / 7224:8200 to keep only nearby prefixes.SiteLink lets two Direct Connect locations talk to each other directly, without going through a Region.
With SiteLink enabled, Regions prefer the path with the shorter AS_PATH regardless of associated Region. SiteLink fails if your router advertises the same route on multiple VIFs. Jumbo up to 8500 or 9001 depending on VIF type.
Ethernet frames of 1522 or 9023 bytes at layer 2; the IP MTU is what you configure.
| Path | MTU |
|---|---|
| Private VIF | 1500 or 9001 |
| Transit VIF | 1500 or 8500 |
| Transit Gateway (any jumbo) | 8500 |
| Public VIF | 1500 |
| Site-to-Site VPN | 1,446 inside the tunnel |
The minimal AWS-side commands; the BGP side is the FRR config from Part 4.1.
# 1. DX gateway (private ASN) and TGW association
aws directconnect create-direct-connect-gateway \
--direct-connect-gateway-name corp-dxgw --amazon-side-asn 64600
aws directconnect create-direct-connect-gateway-association \
--direct-connect-gateway-id dxgw-111 --gateway-id tgw-0def \
--add-allowed-prefixes-to-direct-connect-gateway cidr=10.0.0.0/12
# 2. Transit VIF on the connection (or LAG)
aws directconnect create-transit-virtual-interface \
--connection-id dxcon-abc --new-transit-virtual-interface \
'virtualInterfaceName=tvif-1,vlan=301,asn=65010,mtu=8500,
addressFamily=ipv4,amazonAddress=169.254.10.1/30,
customerAddress=169.254.10.2/30,authKey=<md5>,
directConnectGatewayId=dxgw-111'
# 3. Verify
aws directconnect describe-virtual-interfaces \
--virtual-interface-id dxvif-123 \
--query 'virtualInterfaces[0].[virtualInterfaceState,bgpPeers[0].bgpStatus,mtu]'
aws ec2 search-transit-gateway-routes --transit-gateway-route-table-id tgw-rtb-1 \
--filters Name=type,Values=propagated10.20.0.0/16 → tgw-0def in VPC subnet route tables.ConnectionState and on VirtualInterfaceBpsEgress.Wrong answers that look right.
Scenario → answer → why.
| Scenario | Answer | Why |
|---|---|---|
| One 10G DX must reach 40 VPCs in 3 Regions | Transit VIF → DX gateway → one TGW per Region | DX gateway is global; up to 6 TGWs per DX gateway |
| Need 30 Gbps on one logical link at one location | LAG of 4 × 10G dedicated (minimum links 3) | LAG bundles same-speed ports |
| Compliance needs layer 2 encryption on a 100G link | MACsec with GCM-AES-XPN-256 | XPN required at 100G |
| On-prem must reach S3 public endpoints without the internet | Public VIF (or interface endpoint via private VIF) | Public VIF carries AWS public prefixes |
| Offices in Tokyo and Sydney must talk over AWS without a Region hop | SiteLink on both VIFs to a DX gateway | Backbone path between DX locations |
| After enabling jumbo, all VIFs on the port dropped briefly | Expected: connection update, up to 30 s | Plan MTU changes in a window |
Resiliency models, active/active vs active/passive, VPN backup, route limits, asymmetric routing, and failover tests.
Pick the model from the SLA you need; the toolkit orders the right number of ports and keeps them on separate devices.
SLAs apply only when you meet every requirement in the Direct Connect SLA. A "Classic" connection is ordered one by one without a model. Add a VPN as a low-cost backup on top of any model.
Both designs need redundancy; they differ in capacity planning and predictability.
| Active/active | Active/passive | |
|---|---|---|
| AWS → on-prem | Same prefixes, same community (e.g. 7224:7200), same AS_PATH → ECMP | Primary 7224:7300, backup 7224:7100 (or more specifics on primary) |
| On-prem → AWS | Equal local pref, ECMP on your routers | Higher local pref on the primary edge |
| Capacity rule | Each path must carry 100% if the other fails (keep each under 50%) | Backup sized for full load, or accept degraded mode |
| Pros | Uses all bandwidth; failures are smaller events | Predictable path, simple troubleshooting, easier for stateful firewalls |
| Cons | Flows hash across paths: stateful devices must see both directions | Backup capacity idle; failover is a bigger change |
| Typical | Two DX connections in the same associated Region | DX primary + DX or VPN backup; cross-Region DX |
Cheapest resilient design. AWS prefers DX for the same prefix; your side must do the same.
Exceeding a Direct Connect limit is not a soft warning: the BGP session goes idle.
| Session | Limit |
|---|---|
| Private/transit VIF, you → AWS | 100 per family (prefix controls up to 1,000) |
| Public VIF, you → AWS | 1,000 (fixed) |
| Transit VIF, AWS → you per TGW | 200 (IPv4 + IPv6) |
| Cloud WAN DX gateway attachment → you | 5,000 |
| VPN on VGW, you → AWS | 100 |
| VPN on TGW, you → AWS / AWS → you | 1,000 / 5,000 |
| Private IP VPN | 1,000 in / 5,000 out |
10.0.0.0/12 instead of 40 VPC CIDRs).Routing still "works", but stateful firewalls, NAT and Network Firewall drop the half they never saw.
Fixes: advertise identical prefixes on both paths and steer with communities/local pref, keep stateful devices on a shared session table, or place inspection where both directions pass (TGW appliance mode for inspection VPCs, Ch 7).
The Resiliency Toolkit failover test shuts down BGP sessions on chosen VIFs for a set time.
# bring BGP down on one VIF for 60 minutes
aws directconnect start-bgp-failover-test \
--virtual-interface-id dxvif-primary \
--test-duration-in-minutes 60
# watch: routes move, traffic shifts
aws ec2 search-transit-gateway-routes \
--transit-gateway-route-table-id tgw-rtb-1 \
--filters Name=route-search.exact-match,Values=10.20.0.0/16
# end early and review history
aws directconnect stop-bgp-failover-test \
--virtual-interface-id dxvif-primary
aws directconnect list-virtual-interface-test-history \
--virtual-interface-id dxvif-primaryAlarm on the physical layer and on BGP, not only on application symptoms.
| Metric | Level | Use |
|---|---|---|
ConnectionState | Connection | 1 up / 0 down: alarm when < 1 |
ConnectionBpsEgress / Ingress, ConnectionPpsEgress / Ingress | Connection | Capacity planning; keep active/active paths under 50% |
ConnectionLightLevelTx / Rx | Connection (dedicated) | Optical health; dropping dBm predicts a failing fiber |
ConnectionErrorCount | Connection | MAC-level errors incl. CRC (Sum): dirty optics or bad patching |
ConnectionEncryptionState | Connection (MACsec) | Alarm if encryption drops in should_encrypt mode |
VirtualInterfaceBpsEgress / Ingress | VIF | Per-VIF usage, chargeback |
Alarm on VirtualInterfaceBgpStatus for BGP down. Maintenance appears in AWS Health events; route changes are visible in Transit Gateway Network Manager (Ch 8).
Wrong answers that look right.
Scenario → answer → why.
| Scenario | Answer | Why |
|---|---|---|
| Critical workload, must survive loss of a whole DX location, 99.99% | Maximum resiliency: 2 locations × 2 connections on separate devices | Only model with 99.99% SLA |
| Cost-sensitive, one DX already, needs a backup | Site-to-Site VPN on the same TGW as backup | DX routes win; VPN takes over on withdrawal |
| Prove to auditors that failover works | Direct Connect failover test on the primary VIF | Controlled BGP shutdown with history |
| Firewall logs show half-open sessions after a DX change | Asymmetric routing: align prefixes and local pref | Request and reply on different edges |
| 50 VPC CIDRs exceed 200 prefixes per TGW toward on-prem | Summarize with allowed prefixes on the DX gateway association | TGW allowed prefixes are advertised exactly |
| Failover to backup DX takes 90 s | Enable BFD on both VIFs | Detects in about 900 ms |
Overlay networks into Transit Gateway and Cloud WAN, existing WANs, and private links to other clouds.
A Connect attachment rides on a transport attachment (a VPC attachment or a DX gateway attachment).
| Item | Rule |
|---|---|
| Inside CIDR | /29 from 169.254.0.0/16 (first IP on the appliance); 169.254.0.0/29–169.254.5.0/29 and 169.254.169.248/29 reserved; optional /125 from fd00::/8 |
| BGP | Two sessions per Connect peer (configure both); eBGP needs multihop TTL 2; MP-BGP carries IPv6 over IPv4 peering |
| Not supported | Static routes, BFD, graceful restart |
| ECMP | Same prefixes, same AS_PATH and ASN across peers; routes propagate by default |
| MTU | GRE tunnel MTU = outer MTU − 24 (e.g. 1476 over 1500) |
AWS side first, then a Linux GRE tunnel and FRR on the appliance (a stand-in for an SD-WAN box).
aws ec2 modify-transit-gateway --transit-gateway-id tgw-0def \
--options AddTransitGatewayCidrBlocks=192.0.2.0/24
aws ec2 create-transit-gateway-connect \
--transport-transit-gateway-attachment-id tgw-attach-vpc \
--options Protocol=gre
aws ec2 create-transit-gateway-connect-peer \
--transit-gateway-attachment-id tgw-attach-connect \
--peer-address 10.1.1.10 --transit-gateway-address 192.0.2.1 \
--inside-cidr-blocks 169.254.6.0/29 \
--bgp-options PeerAsn=65050
# appliance subnet route table: 192.0.2.0/24 -> tgw-0def# on the appliance (Linux)
ip tunnel add gre1 mode gre local 10.1.1.10 remote 192.0.2.1 ttl 255
ip addr add 169.254.6.1/29 dev gre1
ip link set gre1 mtu 8476 up # 8500 TGW MTU - 24
# FRR: two sessions, to .2 and .3 of the /29
router bgp 65050
neighbor 169.254.6.2 remote-as 64512
neighbor 169.254.6.2 ebgp-multihop 2
neighbor 169.254.6.3 remote-as 64512
neighbor 169.254.6.3 ebgp-multihop 2
address-family ipv4 unicast
network 172.16.0.0/12Verify: describe-transit-gateway-connect-peers shows both BGP sessions up; on-premises branch prefixes appear as Connect-propagated routes in the TGW route table.
Choose the overlay by what the existing WAN already runs.
| Option | How it works | Pick it when |
|---|---|---|
| TGW Connect (GRE) | GRE + BGP over VPC or DX transport; higher throughput than IPsec, no encryption | SD-WAN appliances in a VPC or on-prem over DX |
| Cloud WAN Connect | Same GRE model on a core network edge; tunnel-less Connect (Oct 2023) peers BGP directly without GRE | Global SD-WAN on Cloud WAN segments |
| IPsec VPN | Encrypted; 1.25 or 5 Gbps per tunnel | Encryption required over the internet |
| MPLS provider | The carrier is a Direct Connect partner and hands off a hosted connection or VIF at a DX location | An enterprise WAN already on MPLS: AWS becomes another site |
| Transit VPC (legacy) | VPN from every VPC to router instances in a hub VPC | Only for exam history: replaced by Transit Gateway (managed, scalable, no instances) |
| Third-party appliances | Marketplace firewalls/routers behind GWLB or as Connect peers | Existing vendor tooling and skills |
GRE is not encryption. If the exam says "encrypt", the overlay must be IPsec (or MACsec on the DX hop), not Connect.
A managed private link from your VPCs to another cloud provider, carried on the AWS backbone and handed off directly.
Not in the ANS-C01 guide. For the exam, multicloud and branch problems still map to DX + partner, VPN, or SD-WAN via TGW Connect.
Read the constraint words in the stem.
| Requirement | Choose | Not |
|---|---|---|
| Fast to set up, encrypted, < ~1 Gbps | Site-to-Site VPN (TGW for scale) | Direct Connect (weeks to provision) |
| Consistent latency, high throughput, lower data transfer rate | Direct Connect (dedicated or hosted) | VPN over internet |
| Encrypted and private at high speed | DX + MACsec, or private IP VPN over transit VIF | Public VIF + VPN |
| Many VPCs and Regions over one DX | Transit VIF + DX gateway + TGWs (or Cloud WAN) | One private VIF per VPC |
| Many small branches | VPN Concentrator, or SD-WAN via Connect | One VPN per branch on a VGW |
| Remote employees | Client VPN or Verified Access | Site-to-Site VPN |
| Branch-to-branch over AWS | DX SiteLink, or Cloud WAN | Hairpin through a VPC |
Wrong answers that look right.
Scenario → answer → why.
| Scenario | Answer | Why |
|---|---|---|
| SD-WAN appliances in a VPC must exchange routes with 50 VPCs dynamically at high throughput | TGW Connect attachment with Connect peers (GRE + BGP) | No IPsec overhead; BGP propagation |
| Same, but the SD-WAN head-ends are on-premises behind DX | Connect attachment using the DX gateway attachment as transport | Connect supports DX transport |
| Existing MPLS WAN; add AWS as a site with minimum change | Ask the MPLS carrier (DX partner) for a hosted connection / VIF | Carrier hands off at a DX location |
| Private link from AWS VPCs to Google Cloud VPCs without managing routers | AWS Interconnect – multicloud | Managed, backbone-carried link (GA Apr 2026) |
| Company runs EC2-based transit VPC; routing scale and HA are problems | Migrate to Transit Gateway | Managed regional router, ECMP, route tables |
| Two Connect peers; traffic uses only one | Advertise the same prefixes with the same AS_PATH and ASN from both | ECMP requires identical attributes |
Explain the answer out loud and name the exact feature or value.
Tasks 1.2, 2.3, 3.3, 4.3 — public and private zones, routing policies, hybrid resolution, DNSSEC and DNS security.
Records, delegation, alias records, domain registration and DNSSEC signing.
Recursive resolvers do the walking; authoritative servers only answer for their own zone.
Lower the TTL (for example to 60 s) a day before a planned cut-over, then raise it again afterwards. Changing TTL at the moment of the change does not help: resolvers already cached the old value with the old TTL.
The exam asks which record solves a requirement, and what a record cannot do.
| Type | Purpose | Watch out for |
|---|---|---|
| A / AAAA | Name → IPv4 / IPv6 address | Dual-stack clients query both; publish AAAA only if the IPv6 path works |
| CNAME | Name → another name | Not allowed at the zone apex; cannot coexist with other records of the same name |
| Alias (Route 53 only) | Name → AWS resource or another record in the zone, answered as A/AAAA | Works at the apex; no TTL of your own when the target is an AWS resource |
| MX / TXT | Mail routing; free text (SPF, DKIM, domain verification) | TXT values over 255 characters are split into several strings |
| PTR | IP → name (reverse lookup), in in-addr.arpa / ip6.arpa zones | Reverse zones for VPC CIDRs are answered by VPC Resolver automatically |
| NS / SOA | Delegation to name servers; zone metadata incl. negative-caching TTL | The NS set in the zone must match the NS set at the registrar |
| SRV / CAA / DS | Service location; which CAs may issue certs; DNSSEC chain to the parent | CAA must allow amazon.com (or amazontrust.com) for ACM issuance |
Use an alias whenever the target is an AWS resource: it is free, works at the apex and follows IP changes.
dig never shows "ALIAS".Exam pattern: "point example.com (the apex) at an ALB" → alias A record. A CNAME at the apex is never a valid answer.
Registering a domain and hosting its zone are separate things; Route 53 can do either or both.
example.com).us-east-1.dev.example.com (it can live in another account).dev in the parent zone.dig NS dev.example.com +trace.dig shows what resolvers see; the Route 53 API shows what you configured. Check both.
# Follow the delegation chain from the root
dig +trace www.example.com
# Ask a Route 53 name server directly (bypasses caches)
dig @ns-123.awsdns-15.com www.example.com A +norecurse
# Remaining TTL from a recursive resolver
dig www.example.com +noall +answer
# Create or update a record (UPSERT is idempotent)
aws route53 change-resource-record-sets \
--hosted-zone-id Z0123456789ABC --change-batch '{
"Changes": [{"Action": "UPSERT", "ResourceRecordSet": {
"Name": "app.example.com", "Type": "A",
"AliasTarget": {
"HostedZoneId": "Z14GRHDCWA56QT",
"DNSName": "dualstack.my-alb-123.ap-northeast-1.elb.amazonaws.com",
"EvaluateTargetHealth": true }}}]}'
aws route53 get-change --id C2682N5HXP0BZ4 # PENDING -> INSYNCHostedZoneId is the target’s zone ID (the ELB’s canonical zone for that Region), not your own zone.get-change returns INSYNC once every Route 53 name server has the change.ResourceRecord elements and 32,000 characters; UPSERT counts twice.Throttling; a concurrent change to the same zone returns PriorRequestNotComplete.DNSSEC proves a response was not tampered with. It signs data; it does not encrypt queries.
ECC_NIST_P256, sign/verify only.DNSSECInternalFailure and DNSSECKeySigningKeysNeedingAction.Wrong answers that look right.
HostedZoneId is the target service’s zone ID, not yours.us-east-1 and asymmetric ECC_NIST_P256.Steer users with weights, latency, geography and health; fail over automatically.
Each record set with the same name and type uses one policy; mix policies by nesting alias records.
| Policy | Decides by | Typical use |
|---|---|---|
| Simple | One record, possibly several values (returned in random order) | Single resource; no health checks on the record |
| Weighted | Relative weight 0–255 per record | Canary / blue-green, gradual migration, A/B tests |
| Latency | Lowest measured latency between the user and AWS Regions | Multi-Region active/active for performance |
| Failover | Primary while healthy, otherwise secondary | Active/passive DR, static S3 "sorry" page |
| Geolocation | Continent, country or US state of the user | Compliance, localized content, licensing |
| Geoproximity | Distance to the resource, adjusted by a bias (−99 to 99) | Shift traffic between Regions by expanding or shrinking areas |
| Multivalue answer | Up to eight healthy records, random order | Simple client-side load spreading with health checks |
| IP-based | Client source CIDR, from a CIDR collection | Route by ISP or known network; cost or performance |
All three use the resolver’s IP, or the client subnet when the resolver sends EDNS Client Subnet (ECS).
Biased distance = actual distance × (1 − bias/100).* catches the rest.Health checks run from Route 53’s global checkers, outside your VPC. That decides what they can reach.
HealthyHostCount) → alarm → alarm-based health check.Alias records to ELB, CloudFront or another record can use "Evaluate target health" instead of a separate health check.
Combine policies by pointing alias records at other records: build a tree.
Active/active: all records healthy and answering (weighted, latency, multivalue). Active/passive: failover policy. If every record in a group is unhealthy, Route 53 treats them all as healthy and keeps answering.
For large policy trees and for controlled, auditable failover.
Scenario → answer → why.
| Scenario | Answer | Why |
|---|---|---|
| Send 5% of users to the new stack, then increase | Weighted records (95/5), health checks on both | Weights shift share without changing clients |
| EU users must be served only from eu-central-1 for compliance | Geolocation (Europe → EU stack) + Default record | Latency could pick a non-EU Region |
| Users should hit the fastest of three Regions | Latency-based alias records with evaluate target health | Latency is measured, not distance |
| Move more traffic to Tokyo without moving users by country | Geoproximity with a positive bias for Tokyo | Bias resizes the catchment area |
| Health check an internal ALB behind a private hosted zone record | CloudWatch alarm on HealthyHostCount + alarm-based health check | Checkers cannot reach private IPs |
| Return several healthy EC2 IPs without a load balancer | Multivalue answer with a health check per record | Up to eight healthy values, random order |
| Route a partner’s ISP ranges to a dedicated endpoint | IP-based routing with a CIDR collection | You know their CIDRs exactly |
Private hosted zones, VPC Resolver endpoints, forwarding rules, Profiles and a central DNS design.
Renamed from "Route 53 Resolver" in March 2026; exam questions and older docs use the old name.
10.0.0.2).169.254.169.253 and fd00:ec2::253 from any subnet.enableDnsSupport = true; hostnames need enableDnsHostnames.ip-10-0-1-5.ec2.internal, interface endpoint names)..2 from another VPC (it serves its own VPC).Authoritative only for the VPCs you associate; invisible from the internet.
create-vpc-association-authorization.associate-vpc-with-hosted-zone.delete-vpc-association-authorization (cleanup).example.com and dev.example.com both associated: the most specific zone answers names under dev.dig from an instance in the VPC, not from your laptop.aws route53 create-vpc-association-authorization --hosted-zone-id Z0PHZ123 \
--vpc VPCRegion=ap-northeast-1,VPCId=vpc-0spoke # in the zone owner account
aws route53 associate-vpc-with-hosted-zone --hosted-zone-id Z0PHZ123 \
--vpc VPCRegion=ap-northeast-1,VPCId=vpc-0spoke # in the VPC owner account
Inbound: on-premises asks AWS. Outbound: AWS asks on-premises. Both ride your VPN or Direct Connect.
Blue: on-premises → inbound endpoint → private hosted zone. Green: workload → VPC Resolver → forwarding rule → outbound endpoint → corporate DNS. Endpoint ENIs need SG rules for TCP and UDP 53 (and 443 for DoH).
Endpoints are ENIs in your subnets; they are billed per ENI hour and per query.
Rules tell VPC Resolver which domains to send elsewhere. They apply only to VPCs they are associated with.
# Forward corp.example.com to two on-premises DNS servers
aws route53resolver create-resolver-rule \
--name corp-forward --rule-type FORWARD \
--domain-name corp.example.com \
--resolver-endpoint-id rslvr-out-0abc \
--target-ips "Ip=172.16.0.10,Port=53" "Ip=172.16.1.10,Port=53" \
--creator-request-id corp-2026-10
# Carve out one subdomain that AWS should resolve itself
aws route53resolver create-resolver-rule --name aws-local \
--rule-type SYSTEM --domain-name aws.corp.example.com \
--creator-request-id aws-local-2026-10
aws route53resolver associate-resolver-rule \
--resolver-rule-id rslvr-rr-0123 --vpc-id vpc-0spoke
# Share the rule with the organization via RAM
RULE=arn:aws:route53resolver:ap-northeast-1:1111…:resolver-rule/rslvr-rr-0123
ORG=arn:aws:organizations::1111…:organization/o-abc123
aws ram create-resource-share --name dns-rules \
--resource-arns $RULE --principals $ORGOne pair of endpoints for the whole organization; spokes reach on-premises names through shared rules.
Spokes need no endpoints of their own: queries that match a shared rule leave through the owner’s outbound endpoint. On-premises DNS forwards AWS zones to the central inbound endpoint IPs, so only the shared VPC needs a DNS path to on-premises.
Launched in 2024. Bundle DNS settings once, share with RAM, associate with up to 1,000 VPCs per Profile.
Scenario → answer → why.
| Scenario | Answer | Why |
|---|---|---|
| On-premises servers must resolve records in a private hosted zone | Inbound endpoint; conditional forwarder on-premises to its IPs | The zone is only answered inside associated VPCs |
EC2 must resolve corp.example.com hosted in Active Directory | Outbound endpoint + FORWARD rule to the AD DNS IPs | VPC Resolver cannot reach on-premises by itself |
| 50 accounts need the same on-premises forwarding with least effort | Central outbound endpoint; share rules via RAM (or a Profile) | One endpoint pair, consumers only associate |
Forward all of corp.example.com except aws.corp.example.com | FORWARD rule for the parent + SYSTEM rule for the subdomain | Most specific rule wins |
| Associate a private zone in account A with a VPC in account B | Authorization in A, association in B (CLI/API) | The console cannot do cross-account associations |
| Inbound endpoint drops queries at peak | Add IPs/ENIs; avoid connection tracking in SG rules | Tracked flows reduce per-IP capacity |
Query logs, DNS Firewall, encrypted DNS transport and Route 53 Global Resolver.
Public logs show who asked your authoritative zone; Resolver logs show what your workloads asked.
us-east-1 only.firewall_rule_action, firewall_domain_list_id).Filters outbound DNS queries from your VPCs. It acts on names, not on IP traffic.
DNS Firewall does not stop a client that uses an IP address directly or another DNS server. Combine it with Network Firewall egress rules and blocking outbound port 53/853 to the internet.
Two separate problems: integrity of answers (DNSSEC) and privacy of queries (DoH / DoT).
| Need | Use | Notes |
|---|---|---|
| Prove answers from my public zone are authentic | DNSSEC signing on the public hosted zone | KSK from a us-east-1 KMS key; DS record at the parent |
| Reject forged answers for workloads | DNSSEC validation in VPC Resolver | Per VPC, or set by a Route 53 Profile |
| Encrypt hybrid DNS between on-premises and AWS | DoH on inbound and outbound Resolver endpoints | Allow TCP 443 in the endpoint SG; DoH-FIPS for inbound only |
| Encrypt everything at the link layer | IPsec VPN or MACsec on Direct Connect | Covers all traffic, not just DNS |
| Stop DNS bypass from workloads | Block outbound 53/853 except to VPC Resolver; DNS Firewall | Clients with hard-coded resolvers otherwise skip controls |
An internet-reachable anycast resolver for your organization’s clients outside any VPC.
Wrong answers that look right.
.2.Explain the answer and name the exact feature.
Tasks 1.1, 1.3, 2.1, 3.3 — ALB, NLB, GWLB, the Load Balancer Controller, CloudFront, Global Accelerator and API Gateway.
ALB, NLB and GWLB: layers, target types, attributes, TLS and Auto Scaling.
Choose by the OSI layer you need to act on, then check the features the requirement names.
| Application LB | Network LB | Gateway LB | |
|---|---|---|---|
| Layer | 7 (HTTP/HTTPS, HTTP/2, gRPC, WebSocket) | 4 (TCP, UDP, TLS, TCP_UDP, QUIC) | 3 (all IP packets, all ports) |
| Routing | Host, path, header, method, query, source IP rules | Listener port → target group, flow hash | All traffic → appliance target group (GENEVE 6081) |
| Targets | Instance, IP, Lambda | Instance, IP, ALB | Instance, IP (virtual appliances) |
| Static IP | No (use NLB in front, or Global Accelerator) | One per AZ; Elastic IPs for internet-facing | Reached through GWLB endpoints |
| Client IP to target | X-Forwarded-For header | Preserved (defaults vary), or proxy protocol v2 | Original packet inside GENEVE |
| Security | SGs, AWS WAF, TLS, mTLS, OIDC/Cognito auth | SGs (only if attached at creation), TLS termination | Inspection by the appliances |
| Typical use | Web apps, microservices, Ingress | Extreme throughput, non-HTTP, PrivateLink services | Third-party firewalls, IDS/IPS |
ALB is a proxy (two connections); NLB forwards flows; GWLB wraps packets for an appliance and returns them.
GWLB adds 68 bytes of GENEVE header and supports packets up to 8,500 bytes, so appliances need an MTU of at least 8,568. GWLB does not fragment and does not send ICMP "fragmentation needed": Path MTU Discovery does not work through it.
Most "the load balancer is wrong" tickets are one of these settings.
| Attribute | ALB | NLB |
|---|---|---|
| Cross-zone | Always on at the LB; can be turned off per target group | Off by default; turning it on adds inter-AZ data charges |
| Algorithm | Round robin (default), least outstanding requests, weighted random (+ anomaly mitigation) | Flow hash on protocol, IPs, ports (and TCP sequence) |
| Stickiness | Duration cookie AWSALB or app cookie (AWSALBAPP-*), 1 s – 7 days | Source IP; not for TLS or QUIC listeners |
| Deregistration delay | 300 s default; target drains, then unused | 300 s default; optional connection termination |
| Slow start | 30–900 s ramp; only with round robin | Not available |
| Client IP | X-Forwarded-For, X-Forwarded-Proto, X-Forwarded-Port | Instance targets: preserved; IP targets with TCP/TLS: off by default |
| Proxy protocol | No | v2 header; carries the VPC endpoint ID for PrivateLink consumers |
The single most tested NLB detail. Know the defaults and the cases where preservation cannot work.
preserve_client_ip.enabled.0xEA).aws elbv2 modify-target-group-attributes --target-group-arn $TG \
--attributes Key=preserve_client_ip.enabled,Value=false Key=proxy_protocol_v2.enabled,Value=true
Decide where TLS ends: at the load balancer (inspect, route, offload) or at the target (end-to-end).
ELBSecurityPolicy-TLS13-1-2-2021-06).X-Amzn-Mtls-Clientcert; the target verifies.X-Amzn-Mtls-Clientcert-Subject, -Issuer, -Serial-Number, -Validity, -Leaf.An ALB is a programmable reverse proxy; many "add a proxy fleet" answers are wrong because the ALB already does it.
ALBRequestCountPerTarget for request-driven scaling.ELB scales automatically, but the shape of the traffic still matters.
Wrong answers that look right.
The AWS Load Balancer Controller: Ingress to ALB, Service to NLB, IP mode and TargetGroupBinding.
A controller in the cluster watches Kubernetes objects and calls the ELB API. It needs IAM (IRSA or Pod Identity).
Subnet discovery uses tags: kubernetes.io/role/elb = 1 for public subnets and kubernetes.io/role/internal-elb = 1 for private subnets (or set the subnets annotation). EKS Auto Mode has built-in load balancing: IngressClass controller: eks.amazonaws.com/alb with IngressClassParams, Services loadBalancerClass: eks.amazonaws.com/nlb (the cluster default); the same subnet tags apply.
Annotations carry the ALB settings; group.name lets many Ingresses share one ALB.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: shop
annotations:
alb.ingress.kubernetes.io/scheme: internet-facing # default internal
alb.ingress.kubernetes.io/target-type: ip # default instance
alb.ingress.kubernetes.io/group.name: public-web
alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS": 443}]'
alb.ingress.kubernetes.io/ssl-redirect: '443'
alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:...:certificate/abcd
alb.ingress.kubernetes.io/healthcheck-path: /healthz
alb.ingress.kubernetes.io/wafv2-acl-arn: arn:aws:wafv2:...:regional/webacl/...
spec:
ingressClassName: alb
rules:
- host: shop.example.com
http:
paths:
- path: /api
pathType: Prefix
backend: { service: { name: api, port: { number: 8080 } } }group.name, group.order) merges rules from several Ingresses into one ALB — fewer ALBs, shared cost.elbv2.k8s.aws/pod-readiness-gate-inject=enabled) stop rollouts from outrunning ALB registration.deletion_protection.enabled=true in load-balancer-attributes.Use an NLB for TCP/UDP or when you need static IPs; bind to load balancers that Terraform owns.
apiVersion: v1
kind: Service
metadata:
name: mqtt
annotations:
service.beta.kubernetes.io/aws-load-balancer-type: external
service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
spec:
type: LoadBalancer
ports: [{ port: 8883, targetPort: 8883, protocol: TCP }]
selector: { app: mqtt }
---
apiVersion: elbv2.k8s.aws/v1beta1
kind: TargetGroupBinding
metadata: { name: api-tgb }
spec:
serviceRef: { name: api, port: 8080 }
targetGroupARN: arn:aws:elasticloadbalancing:...:targetgroup/api/0123
targetType: ipaws-load-balancer-type: external hands the Service to the AWS Load Balancer Controller instead of the legacy in-tree controller.Bring users onto the AWS global network: cache at the edge, or accelerate any TCP/UDP flow.
Both use AWS edge locations. CloudFront is an HTTP cache and proxy; Global Accelerator is an anycast network entry point.
| Amazon CloudFront | AWS Global Accelerator | |
|---|---|---|
| Protocols | HTTP/HTTPS (HTTP/1.1, 2, 3), WebSocket | Any TCP or UDP |
| Caching | Yes: cache behaviors, policies, Origin Shield | No |
| Addressing | DNS name (d111.cloudfront.net); IPs change | 2 static anycast IPv4 (4 IPs with dual stack); BYOIP |
| Failover | Origin groups (GET/HEAD/OPTIONS) | Health-checked endpoint groups; fast failover without DNS TTL |
| Endpoints / origins | S3, ALB, NLB, EC2, API Gateway, any HTTP server, VPC origins | ALB, NLB, EC2, Elastic IP (standard); VPC subnets (custom routing) |
| Security | AWS WAF, Shield, OAC, signed URLs/cookies, geo restriction | Shield Standard; WAF on the ALB endpoint |
| Pick when | Cacheable or dynamic web content, APIs | Gaming, VoIP, IoT, MQTT, static IP allow-lists, multi-Region failover |
Users hit the nearest edge location; misses go via regional edge caches (and optional Origin Shield) to the origin.
VPC origins: CloudFront reaches an ALB, NLB (with an SG, no TLS listener) or EC2 in a private subnet through a service-managed ENI. Allow the CloudFront-VPCOrigins-Service-SG group or the com.amazonaws.global.cloudfront.origin-facing prefix list in the origin SG. Not supported: GWLB origins, gRPC, Lambda@Edge origin request/response triggers; inbound NACL rules are not evaluated, outbound NACLs must allow ephemeral ports.
Map each requirement to the feature and where it runs.
Two anycast IPs announced from every edge location; TCP is terminated at the edge and carried over the AWS backbone.
Scenario → answer → why.
| Scenario | Answer | Why |
|---|---|---|
| Partners need two fixed IPs to allow-list for an ALB-based app in two Regions | Global Accelerator with ALB endpoints | Static anycast IPs, health-based Regional failover |
| UDP game servers need lower jitter for players worldwide | Global Accelerator (standard or custom routing) | CloudFront does not carry UDP |
| Serve a private ALB through CloudFront with no public IPs | CloudFront VPC origin | Private subnet, service-managed ENI |
| App needs the client IP; NLB with IP targets over TCP | Enable client IP preservation (same VPC) or proxy protocol v2 | Off by default for IP/TCP targets |
| Inspect all inbound traffic with a third-party firewall fleet that scales | GWLB + GWLB endpoints in ingress routing | Transparent bump in the wire |
| TLS private keys must stay on the instances | NLB TCP listener (passthrough) | ALB and NLB TLS listeners terminate TLS |
| Blue/green on EKS with a gradual shift | ALB weighted forward to two target groups (TargetGroupBinding) | No DNS TTL delay |
Endpoint types, private APIs through interface endpoints, and VPC links to private back ends.
For REST APIs you choose edge-optimized, Regional or private. HTTP APIs and WebSocket APIs are Regional.
us-east-1.execute-api interface VPC endpoint.aws:SourceVpce or aws:SourceVpc.The interface endpoint and its private DNS setting decide which URL works.
# A) Private DNS enabled on the endpoint
curl https://01234567ab.execute-api.\
ap-northeast-1.amazonaws.com/prod/pets
# B) Private DNS disabled: endpoint name + header
curl https://vpce-0abc-def.execute-api.\
ap-northeast-1.vpce.amazonaws.com/prod/pets \
-H 'x-apigw-api-id: 01234567ab'
# C) Endpoint associated with the API:
# Route 53 alias form {api-id}-{vpce-id}
curl https://01234567ab-vpce-0abc.execute-api.\
ap-northeast-1.amazonaws.com/prod/pets{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Deny", "Principal": "*",
"Action": "execute-api:Invoke",
"Resource": "execute-api:/*",
"Condition": { "StringNotEquals": {
"aws:SourceVpce": "vpce-0abc" } }
}, {
"Effect": "Allow", "Principal": "*",
"Action": "execute-api:Invoke",
"Resource": "execute-api:/*"
}]
}Trap: with private DNS enabled on the execute-api endpoint, that VPC can no longer reach the default endpoints of public APIs (all *.execute-api names resolve to the endpoint). Use custom domains for public APIs, a separate VPC without private DNS, or disable private DNS and create a private hosted zone per private API.
The API stays public (or private); the back end stays in private subnets.
| API type | VPC link | Back-end targets | Notes |
|---|---|---|---|
| REST API | VPC link V1 (legacy) | Network Load Balancer | Still supported; AWS recommends not creating new V1 links |
| REST API | VPC link V2 (Nov 2025) | Application Load Balancer, directly | Removes the NLB hop; API, link and ALB in the same account |
| HTTP API | VPC link V2 | ALB, NLB, or AWS Cloud Map services | Link ENIs live in your subnets with a security group |
| Any | No VPC link | Public HTTP endpoint, Lambda, AWS service | Lambda in a VPC needs no VPC link |
Exam pattern: "expose a private ALB through API Gateway REST API with least cost and latency" → VPC link V2 to the ALB (since Nov 2025). Older answer keys say NLB + VPC link; both are valid designs, the newer one has one less hop.
Wrong answers that look right.
us-east-1; ALB and Regional APIs use the local Region.Explain the answer and name the exact setting.
group.name change?Domain 4 · 24% — threat models, inspection architectures, governance at scale, and encryption in transit.
Threat models for inbound, outbound and east-west flows; AWS WAF and AWS Shield from the network point of view.
The exam describes a flow and a threat. Name the layer, then the cheapest control that stops it there.
| Flow | Typical threats | Network controls | Where they sit |
|---|---|---|---|
| Inbound (internet → app) | DDoS, L7 exploits (SQLi, XSS), bots, scanning | Shield, WAF, CloudFront, ALB, SG, Network Firewall (ingress) | Edge, then public subnets |
| Outbound (app → internet) | Data exfiltration, malware callbacks (C2), crypto-mining pools | Network Firewall domain / IP rules, DNS Firewall, proxies, NAT | Egress VPC or per-VPC firewall subnet |
| East-west (VPC ↔ VPC) | Lateral movement after a compromise | SG referencing, NACLs, Transit Gateway route segmentation, Network Firewall | TGW route tables, inspection VPC |
| Hybrid (on-premises ↔ AWS) | Untrusted branch networks, sniffing on the circuit | Inspection of VPN / DX traffic, IPsec, MACsec, prefix filtering | TGW + inspection VPC, DX location |
| To AWS services | Data copied to a bucket outside the company | VPC endpoint policies, aws:PrincipalOrgID, aws:ResourceOrgID | Endpoint ENIs, resource policies |
| Name resolution | DNS tunneling, lookups of malicious domains | Route 53 Resolver DNS Firewall, query logs | VPC Resolver (VPC+2) |
Defense in depth: one control per layer, never one control for everything. Security groups stay on every ENI even when a firewall sits in front.
Each arrow crosses a control. Only the ALB is reachable from the internet; the database tier has no route out at all.
Exam pattern: "only CloudFront can reach the ALB" → SG allows the managed prefix list com.amazonaws.global.cloudfront.origin-facing, or use CloudFront VPC origins with an internal ALB.
WAF is a layer 7 filter on specific resources. Anything that is not HTTP needs Shield, Network Firewall or security groups.
us-east-1, scope CLOUDFRONT).Outbound and east-west threats are the ones WAF never sees.
| Requirement | Control | Notes |
|---|---|---|
| Allow only listed internet domains | Network Firewall stateful domain list (HTTP Host, TLS SNI) | No decryption needed for SNI matching; TLS inspection for full URLs |
| Block lookups of malicious domains | Resolver DNS Firewall | Cheap first layer; does not stop direct-to-IP traffic |
| Block known C2 / malware IPs | Network Firewall managed threat signature and domain list rule groups | Managed by AWS, updated automatically |
| Inspect with a third-party NGFW | Gateway Load Balancer + appliance fleet | Palo Alto, Fortinet, Check Point, … |
| Stop app tier talking to other VPCs | Separate TGW route tables, blackhole routes | Routing is the coarse control |
| Limit which services a tier can call | Security groups referencing other SGs | Works across TGW and Cloud WAN for inbound rules in the same Region |
| Keep S3 traffic inside the company | Gateway endpoint policy with aws:ResourceOrgID | Stops copying data to a personal bucket |
Names the exam uses for common designs.
ec2:CreateInternetGateway outside the network account.Testing compliance (Task 4.1): re-run Reachability Analyzer paths and failover tests after every change, and alert on drift with AWS Config.
Wrong answers that look right.
Scenario → answer → why.
| Scenario | Answer | Why |
|---|---|---|
| Workload VPCs must never have direct internet access; central team controls egress | Egress / perimeter VPC behind Transit Gateway + SCP denying IGW creation | Routing forces the path; SCP stops bypass |
| Game server on UDP behind an NLB is hit by floods | Shield Advanced on the NLB Elastic IPs (or Global Accelerator) | WAF is HTTP-only; Shield covers L3/L4 |
Only allow outbound HTTPS to *.github.com and *.amazonaws.com | Network Firewall stateful domain allow list with default drop | SNI-based filtering needs no decryption |
| Partner network connects over VPN and must reach only one API | PrivateLink endpoint service for the API (or a dedicated TGW route table) | Exposes one service, not the whole network |
| Developers copy data to personal S3 buckets from EC2 | S3 gateway endpoint policy with aws:ResourceOrgID | Endpoint policy limits which buckets are reachable |
Stateless and stateful engines, Suricata rules, TLS inspection, and the distributed, centralized and GWLB deployment models.
A managed, horizontally scaling firewall built on Gateway Load Balancer technology, with a Suricata-compatible engine.
vpce-…).Stateless runs first on every packet; stateful sees whole flows and application protocols.
vpce-….Fragmented and asymmetric traffic breaks stateful inspection. If only one direction of a flow crosses the firewall, the stateful engine drops it.
Pick the simplest rule type that meets the requirement; stateful rules cost more capacity.
| Rule group type | Matches on | Use for |
|---|---|---|
| Stateless | Source/destination IP and port, protocol, TCP flags | Fast drops of obvious junk; forwarding everything else to stateful |
| Stateful 5-tuple (standard) | IP, port, protocol, direction | Simple allow/deny lists without Suricata syntax |
| Stateful domain list | HTTP Host header and TLS SNI | Egress allow lists such as .github.com |
| Suricata-compatible IPS rules | Suricata keywords such as tls.sni, http.host, http.uri, geoip | Detailed L7 policy and intrusion signatures |
| AWS managed rule groups | Threat signatures, malware/botnet domain lists, active threat defense | Baseline protection maintained by AWS |
| Partner managed rule groups | Vendor threat intelligence from AWS Marketplace | Extra feeds without running appliances |
Geographic IP filtering matches the source or destination country of IPv4 and IPv6 traffic, for example to block all traffic to or from a sanctioned country.
AWS recommends strict order. Rule groups and the policy must use the same ordering.
pass → drop → reject → alert.priority keyword (1–65535) orders rules.pass stops inspection of the rest of that flow.Each rule needs a unique sid. In strict order these run top to bottom.
# Allow HTTPS only to approved domains (match on TLS SNI)
pass tls $HOME_NET any -> $EXTERNAL_NET 443 (tls.sni; dotprefix;
content:".github.com"; endswith; nocase; flow:to_server;
msg:"allow github"; sid:100001; rev:1;)
pass tls $HOME_NET any -> $EXTERNAL_NET 443 (tls.sni;
content:"api.example-partner.com"; startswith; endswith;
nocase; flow:to_server; sid:100002; rev:1;)
# Allow plain HTTP to the OS package mirror by Host header
pass http $HOME_NET any -> $EXTERNAL_NET 80 (http.host;
content:"deb.debian.org"; startswith; endswith;
flow:to_server; sid:100003; rev:1;)
# Block SSH to the internet and log it
drop tcp $HOME_NET any -> $EXTERNAL_NET 22 (flow:to_server;
msg:"egress ssh blocked"; sid:100010; rev:1;)
# Alert on TLS to a non-standard port (possible tunnel)
alert tls $HOME_NET any -> $EXTERNAL_NET !443 (flow:to_server;
msg:"tls on odd port"; sid:100020; rev:1;)$HOME_NET defaults to the firewall VPC CIDR. For a central firewall, set it to all spoke CIDRs in the rule variables.dotprefix + endswith on .github.com matches subdomains but not evilgithub.com.flow:to_server matches client-to-server packets only.tls, http) wait for the app layer; tcp rules can match the first SYN.Without decryption, the firewall sees only SNI and certificates. With it, rules can match URLs, headers and payloads.
Each VPC has its own firewall endpoints; ingress routing on the IGW sends inbound traffic through them.
Pros: no single choke point, blast radius of one VPC, no Transit Gateway data processing. Cons: many firewalls to pay for and manage (use Firewall Manager), and east-west traffic between VPCs is not inspected unless it also crosses a firewall.
Spokes send everything to the inspection VPC; the firewall returns it to TGW, which routes it to the destination.
Two TGW route tables: spokes are associated with the "spoke" table (default route to the inspection attachment); the inspection attachment is associated with the "firewall" table, where spoke routes are propagated.
Transit Gateway keeps traffic in the source AZ by default. Two spokes in different AZs then cross different firewall endpoints.
aws ec2 modify-transit-gateway-vpc-attachment --transit-gateway-attachment-id tgw-attach-0abc \
--options ApplianceModeSupport=enable
Available in all Regions with both services since July 2025. The firewall becomes a TGW attachment type.
GWLB is a layer 3 load balancer: it keeps flows sticky to one appliance and wraps packets in GENEVE.
From cheapest to most capable. Many designs combine DNS Firewall with one of the inline options.
| Option | How it filters | Strengths | Limits |
|---|---|---|---|
| NAT gateway only | No filtering; hides private IPs | Simple, managed | Any destination allowed |
| Resolver DNS Firewall | Blocks DNS answers for listed domains | Cheap, catches DNS tunneling | Bypassed by hard-coded IPs or other resolvers |
| Network Firewall (transparent) | Domain lists, SNI, IP, Suricata, TLS inspection | Managed, no client changes | Endpoint hours + per-GB processing |
| Network Firewall proxy mode | Explicit proxy (HTTP CONNECT), no source preservation; uses the attached NAT gateway | Same policy for proxy and transparent filtering | Preview (re-introduced Aug 2026, us-east-2 only) |
| Self-managed proxy (Squid) behind NLB | URL allow lists, authentication | Full control, URL-level rules | You patch and scale it; clients need proxy settings |
| Third-party firewall via GWLB | Vendor URL filtering, IPS, user-ID | Feature parity with on-premises | Licenses and appliance operations |
Logging is configured per firewall and per log type; each type can go to a different destination.
drop, reject or alert.AWS/NetworkFirewall, per AZ endpoint.ReceivedPackets, PassedPackets, DroppedPackets, RejectedPackets, OtherDroppedPackets (includes throttling).TLSErrors, TLSDroppedPackets, revocation-status counters.Wrong answers that look right.
$HOME_NET only treats the inspection VPC as home. Add the spoke CIDRs.pass rule wins over a later, specific drop. Use strict order.Scenario → answer → why.
| Scenario | Answer | Why |
|---|---|---|
| Inspect all traffic between 50 VPCs and to on-premises with least operational effort | Network Firewall with native Transit Gateway attachment | No inspection VPC to build; inspects VPC, VPN and DX traffic |
| Security team must use its existing Palo Alto policy in AWS | GWLB with the vendor appliances; GWLB endpoints in spokes or inspection VPC | GWLB is built for third-party inline appliances |
| Some flows through the central firewall drop randomly; others work | Enable appliance mode on the inspection VPC attachment | Cross-AZ flows were asymmetric |
| Block outbound traffic to any domain not on an approved list, without decrypting | Strict order, Drop established, domain list allow rules on SNI and Host | SNI is visible in the TLS Client Hello |
| Need to block a specific malicious URL path over HTTPS | Network Firewall TLS inspection (outbound) + http.uri rule | The path is encrypted; only decryption exposes it |
Firewall Manager across the organization, configuration audits, AWS Verified Access, and guardrails as code.
Central management of network security policies across an AWS Organization, including accounts created later.
| Policy | Enforces |
|---|---|
| AWS WAF (and WAF Classic) | First and last rule groups in every web ACL; accounts add rules in between |
| Shield Advanced | Protections on chosen resource types |
| Security group | Common baseline SGs, content audit, usage audit |
| Network ACL | Baseline first and last NACL rules |
| Network Firewall | Firewalls and policies in VPCs (distributed or centralized) |
| Resolver DNS Firewall | Rule group associations on VPCs |
| Third-party firewall | Palo Alto Cloud NGFW, Fortigate CNF |
Scope by account, OU, resource type or tag. Choose "identify only" first to see non-compliant resources, then turn on auto-remediation.
The most frequently tested Firewall Manager feature.
0.0.0.0/0 on 22 or 3389.Firewall Manager enforces; these tools continuously report on what exists.
| Tool | What it checks | Example |
|---|---|---|
| AWS Config managed rules | Resource configuration against a rule, with history | restricted-ssh, restricted-common-ports, vpc-sg-open-only-to-authorized-ports, vpc-default-security-group-closed, vpc-flow-logs-enabled |
| Config conformance packs | A bundle of rules deployed org-wide | Operational best practices for networking; remediation with SSM Automation |
| AWS Trusted Advisor | Account-level best-practice checks | "Security Groups – Specific Ports Unrestricted", "Security Groups – Unrestricted Access" |
| Network Access Analyzer | Paths that violate an intent (e.g. internet → database) | Finds unintended reachability across the account |
| Security Hub CSPM | Aggregated findings with controls such as EC2 security group checks | One dashboard across accounts |
| CloudTrail | Who changed the network and when | AuthorizeSecurityGroupIngress, CreateRoute, ModifyVpcEndpoint |
Per-request access decisions based on identity and device posture, without a VPN.
Preventive controls stop teams from building around the network design.
{
"Sid": "OnlyNetworkAccountBuildsEdges",
"Effect": "Deny",
"Action": [
"ec2:CreateInternetGateway",
"ec2:AttachInternetGateway",
"ec2:CreateEgressOnlyInternetGateway",
"ec2:CreateVpcPeeringConnection",
"ec2:AcceptVpcPeeringConnection"
],
"Resource": "*",
"Condition": {
"ArnNotLike": { "aws:PrincipalArn":
"arn:aws:iam::*:role/NetworkAdmin" }
}
}{
"Statement": [{
"Sid": "OnlyOurOrgBuckets",
"Effect": "Allow",
"Principal": "*",
"Action": ["s3:GetObject", "s3:PutObject"],
"Resource": "*",
"Condition": {
"StringEquals": {
"aws:ResourceOrgID": "o-abc123",
"aws:PrincipalOrgID": "o-abc123"
}
}
}]
}Left: SCP statement in workload OUs. Right: S3 gateway endpoint policy that limits both who uses the endpoint and which buckets it reaches. More data perimeter patterns are in the AWS Security & Governance deck.
Scenario → answer → why.
| Scenario | Answer | Why |
|---|---|---|
| Remove any SG rule that opens SSH to the world in 200 accounts, automatically | Firewall Manager content audit SG policy with auto-remediation | Central, covers new accounts, remediates |
| Attach the security team’s scanner SG to every EC2 instance | Firewall Manager common SG policy | Baseline SG on every in-scope resource |
| Firewall Manager policy shows no resources | Enable AWS Config in those accounts and Regions | Firewall Manager depends on Config |
| Contractors need SSH to a few hosts, no VPN, with device checks | Verified Access non-HTTP endpoint + device trust provider | Per-connection, posture-aware access |
| Workload accounts keep creating VPC peerings that bypass inspection | SCP denying ec2:CreateVpcPeeringConnection except for network admins | Preventive, organization-wide |
| Report which accounts have SGs open to 0.0.0.0/0 on 3389 | AWS Config rule restricted-common-ports with an aggregator | Continuous evaluation and history |
Who encrypts what, Nitro encryption, VPC Encryption Controls, IPsec and MACsec for hybrid links, and TLS everywhere.
AWS encrypts several layers automatically. Your data between your endpoints is still yours to protect.
| Layer | Encrypted by | Conditions |
|---|---|---|
| Between AWS Regions and between AZs | AWS, at the physical layer | All traffic on the AWS global network leaving secured facilities |
| Inter-Region VPC peering and TGW peering | AWS, bulk encryption | Automatic when traffic exits a Region |
| Between EC2 instances | Nitro hardware (AEAD, 256-bit), no performance impact | Supported instance types on both ends, same Region, same or peered VPC, no load balancer or TGW in the path |
| VPC Encryption Controls enforce mode | Nitro + application layer, verified by AWS | All resources in the VPC must be compatible or excluded |
| Site-to-Site VPN, Client VPN | IPsec / TLS by AWS and your device | Over the internet or over Direct Connect |
| Direct Connect | Not encrypted by default | Add MACsec (L2) or IPsec VPN over DX (L3) |
| Application data | You: TLS, mTLS | Load balancers, databases, S3, APIs |
aws ec2 describe-instance-types --filters Name=network-info.encryption-in-transit-supported,Values=true \
--query "InstanceTypes[*].InstanceType" --output text
Launched Nov 2025; paid per non-empty VPC per hour since 1 Mar 2026. Uses Nitro encryption plus application-layer TLS.
get-vpc-resources-blocking-encryption-enforcement).encryption-status (0–3).Direct Connect is private but not encrypted. Pick the layer that matches the requirement.
associate-mac-sec-key.should_encrypt (default) and must_encrypt. No extra charge.Where TLS ends decides who can see plaintext and which certificates you manage.
us-east-1; ELB certificates in the LB Region.DaysToExpiry.Deny non-TLS requests at the resource so a misconfigured client fails loudly.
{
"Sid": "DenyInsecureTransport",
"Effect": "Deny",
"Principal": "*",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::payroll-data",
"arn:aws:s3:::payroll-data/*"
],
"Condition": {
"Bool": { "aws:SecureTransport": "false" }
}
}s3:TlsVersion to require a minimum TLS version.rds.force_ssl = 1.require_secure_transport = ON.aws:SecureTransport deny in the queue or topic policy.Scenario → answer → why.
| Scenario | Answer | Why |
|---|---|---|
| Encrypt a 100 Gbps dedicated DX at line rate | MACsec with GCM-AES-XPN-256 | IPsec tunnels cannot reach that throughput |
| Encrypt DX traffic end to end into the VPCs without using the internet | Private IP VPN over a transit VIF to Transit Gateway | IPsec from your router to TGW with private outer IPs |
| Prove all east-west traffic in a VPC is encrypted for an auditor | VPC Encryption Controls in monitor, then enforce; flow logs with encryption-status | Authoritative status per flow |
| Enforce mode is blocked by the central firewall in the VPC | Keep that VPC in monitor mode (GWLB / Network Firewall unsupported in enforce) | Documented limitation |
| Instances on supported types talk via an NLB; is Nitro encryption automatic? | No; add TLS | Nitro encryption excludes paths through load balancers or TGW |
| Hosted 1 Gbps DX must be encrypted | IPsec VPN over DX | MACsec is not available on hosted connections |
Explain the answer and name the exact control.
Flow logs, mirroring, metrics and analyzers — then a repeatable playbook for the failures you will actually see.
VPC Flow Logs in depth, Athena queries, Traffic Mirroring, access logs, CloudWatch metrics and network monitors.
Pick the tool from the question. Most investigations use two or three of them together.
| Question | Tool | What you get |
|---|---|---|
| Was this flow accepted or rejected, and by which ENI? | VPC Flow Logs | 5-tuple, bytes, ACCEPT / REJECT per ENI |
| What is inside the packets? | VPC Traffic Mirroring | Full packet copies (VXLAN) to an IDS or tcpdump host |
| Should this path work, per the configuration? | Reachability Analyzer | Hop-by-hop path or the blocking component |
| Which paths exist that should not? | Network Access Analyzer | Findings for paths matching a scope (e.g. internet → DB) |
| Do TGW route tables send this prefix where I expect? | Network Manager Route Analyzer | Route path through TGW route tables only |
| Who changed the route table or SG? | CloudTrail | API call, principal, time, parameters |
| Is a link, tunnel or gateway healthy or saturated? | CloudWatch metrics and alarms | Tunnel state, DX light levels, NAT port errors, drops |
| Is latency or loss from AWS or from my app? | Network Flow Monitor, Internet Monitor, Network Synthetic Monitor | RTT, retransmissions, health indicators |
Metadata about IP flows per ENI. Not packet contents, not real time.
169.254.169.254 and Time Sync 169.254.169.123.dstaddr shows the primary IP; add pkt-dstaddr for the real one.pkt-srcaddr / pkt-dstaddr.log-status SKIPDATA means records were dropped during that interval.The default format is fixed: the 14 version-2 fields in this order. Anything else needs a custom format.
# version account-id interface-id srcaddr dstaddr srcport dstport
# protocol packets bytes start end action log-status
2 111122223333 eni-0a1b2c3d4e5f60718 203.0.113.45 10.0.1.25
51522 22 6 4 240 1760947200 1760947260 REJECT OK
2 111122223333 eni-0a1b2c3d4e5f60718 10.0.1.25 10.0.10.40
41030 5432 6 18 3120 1760947200 1760947260 ACCEPT OK
2 111122223333 eni-0a1b2c3d4e5f60718 - - - - - - -
1760947260 1760947320 - NODATA| Field | Value → meaning |
|---|---|
| srcaddr → dstaddr | Internet host → instance private IP |
| srcport → dstport | Ephemeral 51522 → SSH 22 |
| protocol | 6 = TCP (17 UDP, 1 ICMP, 58 ICMPv6) |
| packets / bytes | 4 packets, 240 bytes: SYN retries |
| start / end | Unix seconds of the capture window |
| action | REJECT by SG or NACL |
| log-status | OK, NODATA, SKIPDATA |
A REJECT does not say which control blocked it. Inbound REJECT with no matching outbound record: SG or inbound NACL. Inbound ACCEPT followed by outbound REJECT: the outbound NACL (stateless) is missing ephemeral ports.
The record version becomes the highest version of the fields you choose.
| Field | Ver. | Why add it |
|---|---|---|
vpc-id, subnet-id, instance-id | 3 | Group by subnet or instance without joins |
tcp-flags | 3 | FIN 1, SYN 2, RST 4, SYN-ACK 18; OR-ed per interval (19 = SYN-ACK + FIN) |
pkt-srcaddr, pkt-dstaddr | 3 | Original IPs behind NAT gateways, EKS pods, secondary IPs |
region, az-id | 4 | Find cross-AZ traffic (cost, latency) |
pkt-src-aws-service, pkt-dst-aws-service | 5 | Label AWS ranges: S3, DYNAMODB, EC2, ROUTE53_RESOLVER, … |
flow-direction, traffic-path | 5 | ingress / egress, and which gateway egress used |
ecs-* fields | 7 | Cluster, service, task ARN for awsvpc tasks |
reject-reason | 8 | BPA (Block Public Access) or EC (Encryption Controls) |
encryption-status | 10 | 0 none, 1 Nitro, 2 application, 3 both |
next-hop-*, interface-type, tag fields | 11 | Next-hop ENI/subnet/VPC; NAT, NLB, TGW, endpoint ENIs; instance/ASG tags |
Two custom-format examples: which gateway the traffic used, and who is really behind a NAT gateway.
| Value | Egress went through |
|---|---|
| 1 | Another resource in the same VPC (incl. AWS-managed ENIs) |
| 2 | Internet gateway or gateway VPC endpoint |
| 3 | Virtual private gateway |
| 4 | Intra-Region VPC peering |
| 5 | Inter-Region VPC peering |
| 6 | Local Zone or Wavelength Zone |
| 7 | Gateway VPC endpoint |
| 8 | Internet gateway |
# format: interface-id srcaddr pkt-srcaddr dstaddr
# pkt-dstaddr dstport flow-direction traffic-path
# 1) On the instance ENI: leaving towards the NAT GW
eni-app01 10.0.10.25 10.0.10.25 52.94.8.10
52.94.8.10 443 egress 1
# 2) On the NAT GW ENI: arriving from the instance
eni-nat01 10.0.10.25 10.0.10.25 10.0.0.9
52.94.8.10 443 ingress -
# 3) On the NAT GW ENI: leaving to the internet
eni-nat01 10.0.0.9 10.0.10.25 52.94.8.10
52.94.8.10 443 egress 8Record 3 is the one that answers "which instance is generating NAT data processing charges": srcaddr is the NAT gateway, pkt-srcaddr is the real source.
Choose by how you will query them and how long you keep them.
delivery.logs.amazonaws.com; no role.fields @timestamp, srcAddr, dstAddr, dstPort, action
| filter action = "REJECT" and dstPort in [22, 3389]
| stats count(*) as attempts by srcAddr, dstPort
| sort attempts desc | limit 20 # CloudWatch Logs Insights
Partition projection avoids running MSCK REPAIR or crawlers as new days arrive.
CREATE EXTERNAL TABLE vpc_flow_logs (
version int, account_id string, interface_id string,
srcaddr string, dstaddr string, srcport int, dstport int,
protocol bigint, packets bigint, bytes bigint,
start bigint, `end` bigint, action string, log_status string)
PARTITIONED BY (`date` date)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ' '
LOCATION
's3://flowlogs/AWSLogs/111122223333/vpcflowlogs/ap-northeast-1/'
TBLPROPERTIES (
"skip.header.line.count"="1",
"projection.enabled"="true",
"projection.date.type"="date",
"projection.date.format"="yyyy/MM/dd",
"projection.date.range"="2026/01/01,NOW",
"projection.date.interval"="1",
"projection.date.interval.unit"="DAYS",
"storage.location.template"=
"s3://flowlogs/AWSLogs/111122223333/vpcflowlogs/ap-northeast-1/${date}");AWSLogs/<account>/vpcflowlogs/<region>/yyyy/MM/dd/.date to scan only the days you need.Run against the table on the previous slide.
-- 1) Who is knocking on SSH/RDP from outside?
SELECT srcaddr, dstport, count(*) AS hits
FROM vpc_flow_logs
WHERE date = DATE '2026-10-20' AND action = 'REJECT'
AND dstport IN (22, 3389)
AND NOT (srcaddr LIKE '10.%')
GROUP BY 1, 2 ORDER BY hits DESC LIMIT 20;
-- 2) Top talkers by bytes in the last 7 days
SELECT srcaddr, dstaddr, sum(bytes)/1e9 AS gb
FROM vpc_flow_logs
WHERE date >= current_date - INTERVAL '7' DAY
GROUP BY 1, 2 ORDER BY gb DESC LIMIT 20;-- 3) Which hosts use the NAT gateway most?
-- (custom-format table with pkt_srcaddr, flow_direction)
SELECT pkt_srcaddr, sum(bytes)/1e9 AS gb
FROM vpc_flow_logs_v5
WHERE date >= current_date - INTERVAL '1' DAY
AND interface_id = 'eni-nat01'
AND flow_direction = 'egress'
GROUP BY 1 ORDER BY gb DESC LIMIT 10;
-- 4) Was a specific flow ever accepted?
SELECT from_unixtime(start) AS t, action, packets
FROM vpc_flow_logs
WHERE date = DATE '2026-10-20'
AND srcaddr = '10.0.10.25' AND dstaddr = '10.0.11.40'
AND dstport = 5432
ORDER BY t;Copies packets from a source ENI, wraps them in VXLAN, and sends them to a target for IDS or capture.
aws ec2 create-traffic-mirror-target --network-load-balancer-arn $NLB_ARN
aws ec2 create-traffic-mirror-filter --description "web only"
aws ec2 create-traffic-mirror-filter-rule --traffic-mirror-filter-id tmf-0abc \
--traffic-direction ingress --rule-number 100 --rule-action accept \
--protocol 6 --destination-port-range FromPort=443,ToPort=443 \
--source-cidr-block 0.0.0.0/0 --destination-cidr-block 0.0.0.0/0
aws ec2 create-traffic-mirror-session --network-interface-id eni-src01 \
--traffic-mirror-target-id tmt-0abc --traffic-mirror-filter-id tmf-0abc \
--session-number 1 --packet-length 128 # first 128 bytes only--packet-length to copy headers only.Flow logs stop at L4. These logs carry the L7 and change history you need to correlate.
| Source | Contents | Delivery | Network use |
|---|---|---|---|
| ALB access logs | Client IP:port, target, status codes, latencies, TLS cipher, trace ID | S3 (every 5 min) | Find 5xx by target, slow targets, client IPs |
| ALB connection logs | TLS handshake details, client certificate info for mTLS | S3 | Debug mTLS and TLS version issues |
| NLB access logs | TLS listener connections: TLS version, cipher, handshake time | S3 | TLS listeners only |
| CloudFront standard logs | Edge location, client IP, cache result, time taken | S3, CloudWatch Logs, Firehose | Cache hit ratio, geographic issues |
| CloudFront real-time logs | Configurable fields within seconds | Kinesis Data Streams | Live dashboards and alerting |
| CloudTrail | CreateRoute, AuthorizeSecurityGroupIngress, ModifyTransitGatewayVpcAttachment | S3, CloudWatch Logs, EventBridge | "What changed right before the outage?" |
| VPC Resolver query logs | Query name, type, response code, source ENI | S3, CloudWatch Logs, Firehose | DNS failures, malicious lookups |
Default metrics most teams forget, by service.
| Service | Metric | Alarm when |
|---|---|---|
| NAT gateway | ErrorPortAllocation, PacketsDropCount, IdleTimeoutCount | > 0: port exhaustion (55,000 connections per unique destination), drops |
| Transit Gateway | PacketDropCountBlackhole, PacketDropCountNoRoute, BytesIn/Out | Drops > 0 after a routing change |
| Site-to-Site VPN | TunnelState, TunnelDataIn, TunnelDataOut | TunnelState < 1 (per tunnel) |
| Direct Connect | ConnectionState, ConnectionLightLevelRx/Tx, ConnectionErrorCount, ConnectionBpsEgress | State 0, light level out of range, CRC errors |
| ALB | HTTPCode_ELB_5XX_Count, TargetResponseTime, RejectedConnectionCount | ELB 5xx vs target 5xx tells LB vs app |
| NLB | TCP_Target_Reset_Count, TCP_ELB_Reset_Count, UnHealthyHostCount, PortAllocationErrorCount | Resets or port allocation errors > 0 |
EC2 (ENA, via ethtool -S) | bw_in/out_allowance_exceeded, pps_allowance_exceeded, conntrack_allowance_exceeded, linklocal_allowance_exceeded | Any increase: instance network limits hit |
Three monitors with different vantage points. Capture a baseline before you need it.
Baseline (Task 1.4): record normal RTT, loss, tunnel throughput, NAT connections and TGW bytes per hour for a few weeks; use CloudWatch anomaly detection alarms instead of fixed thresholds.
Reachability Analyzer, Network Access Analyzer, and Network Manager with Route Analyzer.
A configuration analysis of routes, SGs, NACLs, gateways and load balancers between a source and a destination.
# 1) Define the path: IGW -> instance on TCP 22
aws ec2 create-network-insights-path \
--source igw-0797cccdc9d73b0e5 \
--destination i-0495d385ad28331c7 --protocol TCP \
--filter-at-source '{"DestinationPortRange":
{"FromPort":22,"ToPort":22}}'
# 2) Run the analysis (takes a few minutes)
aws ec2 start-network-insights-analysis \
--network-insights-path-id nip-0b26f224f1d131fa8
# 3) Read the verdict
aws ec2 describe-network-insights-analyses \
--network-insights-analysis-ids nia-0abc123def456789 \
--query 'NetworkInsightsAnalyses[0].[NetworkPathFound,
Explanations[0].ExplanationCode]'
# [false, "ENI_SG_RULES_MISMATCH"]NetworkPathFound, ForwardPathComponents, ReturnPathComponents, Explanations.ENI_SG_RULES_MISMATCH.Reachability Analyzer checks one path. Network Access Analyzer searches for every path matching a pattern.
{
"MatchPaths": [{
"Source": { "ResourceStatement": {
"ResourceTypes": ["AWS::EC2::InternetGateway"] } },
"Destination": {
"ResourceStatement": {
"ResourceTypes": ["AWS::EC2::NetworkInterface"] },
"PacketHeaderStatement": {
"DestinationPorts": ["3306", "5432"] } }
}],
"ExcludePaths": [{
"ThroughResources": [{ "ResourceStatement": {
"ResourceTypes": ["AWS::NetworkFirewall::Firewall"] } }]
}]
}AWS Network Manager is the console for global networks built on Transit Gateway and Cloud WAN.
Scenario → answer → why.
| Scenario | Answer | Why |
|---|---|---|
| Need the payload of suspicious traffic for an IDS | VPC Traffic Mirroring to an NLB in front of IDS sensors | Flow logs have no payload |
| Flow logs show the NAT gateway IP, not the instance | Custom format with pkt-srcaddr | srcaddr is the intermediate hop |
| Validate before go-live that the app subnet can reach RDS on 5432 | Reachability Analyzer path | Configuration proof without traffic |
| Continuously prove no database is reachable from the internet | Network Access Analyzer scope, scheduled | Searches all paths, not one |
| A TGW route change dropped traffic between two VPCs; find which table | Network Manager Route Analyzer | It walks TGW route tables |
| DNS lookups are failing, but flow logs show nothing | Resolver query logs | Flow logs do not capture Amazon DNS traffic |
| Alert within minutes when a VPN tunnel goes down | CloudWatch alarm on TunnelState (or Network Manager event in EventBridge) | Native metric per tunnel |
One decision tree and six real failure patterns: symptom → checks → root cause → fix.
Each step has a tool. Stop at the first step that fails.
Timeouts usually mean a route or a filter silently dropped the packet. "Connection refused" means the packet reached the host and nothing listens. Learn to tell them apart first.
Symptom: yum update and curl https://… time out from instances in a private subnet.
0.0.0.0/0 → nat-…?0.0.0.0/0 → igw-…?available, public NAT has an EIP?ErrorPortAllocation, PacketsDropCount.egress 8.Symptom: app.internal.example.com resolves in the VPC but returns NXDOMAIN from the data center.
internal.example.com → inbound endpoint IPs.10.0.0.2): not reachable from outside the VPC.dig @<endpoint-ip> app.internal.example.com from on-prem.InboundQueryVolume on the endpoint.Symptom: after adding a central firewall, some VPC-to-VPC connections hang, others work; failures vary by instance.
ApplianceModeSupport?Symptom: SSH works and small API calls succeed over Site-to-Site VPN, but file transfers and large responses hang.
ping -M do -s 1472 <host> (Linux) then lower the size.ip link) sending into a 1,446-byte tunnel?ip route … mtu 1422 for CBC + NAT-T).tracepath <host> shows the discovered path MTU.Symptom: on-prem routes disappear from TGW route tables; traffic fails over to VPN or stops.
ConnectionState up? Light levels ConnectionLightLevelRx/Tx in range (physical layer)?down vs idle).ConnectionState and VIF BGP status; test with the Resiliency Toolkit failover test.Symptom: new pods stay Pending / ContainerCreating with "failed to assign an IP address"; nodes do not scale.
AvailableIpAddressCount in the console or CLI.WARM_ENI_TARGET, WARM_IP_TARGET, MINIMUM_IP_TARGET.ENABLE_PREFIX_DELEGATION)?Explain the answer and name the exact tool.
traffic-path 8 mean, and what does 7 mean?Tasks 2.4, 4.1 — repeatable network configuration, no hardcoded values, change safety, and event-driven operations.
Build VPCs, Transit Gateways and DNS from code, allocate addresses from IPAM, and prove every change before it ships.
The exam asks for "repeatable", "consistent across accounts" and "least operational overhead": that is infrastructure as code.
All are in scope. Pick by team skills and by how many accounts and Regions you deploy to.
| Tool | Model | Strengths for networking | Watch out for |
|---|---|---|---|
| AWS CloudFormation | Declarative JSON/YAML; state kept by AWS | StackSets to every account/Region in an OU; change sets; drift detection | Replacement on some property changes; verbose templates |
| AWS CDK | TypeScript/Python code that synthesizes CloudFormation | High-level ec2.Vpc construct, loops, unit tests | Defaults hide resources (NAT per AZ); review cdk diff |
| Terraform | Declarative HCL; state in S3 (lock file or DynamoDB) | One tool for AWS, on-prem routers and SD-WAN vendors; plan shows replacements | State is yours to secure; provider version drift |
| AWS CLI / SDK | Imperative calls | Glue, one-off fixes, Lambda automation, tests | No state, no plan; easy to make snowflakes |
| Ansible / vendor APIs | Imperative or declarative device config | Customer gateway and firewall config from AWS-generated VPN files | Keep it in the same pipeline as the AWS side |
Hybrid automation: aws ec2 get-vpn-connection-device-sample-configuration returns a vendor-specific CGW config that a pipeline can template and push to the on-premises router.
Separate what changes rarely (core network) from what changes often (workload attachments), and keep state per account and Region.
network-iac/
├── modules/
│ ├── vpc/ # subnets per AZ, route tables, endpoints, flow logs
│ ├── tgw/ # TGW, route tables, RAM share
│ ├── tgw-attachment/ # spoke attachment + association/propagation
│ ├── dns/ # PHZs, Resolver endpoints, rules, RAM share
│ └── inspection/ # Network Firewall, appliance mode, routes
├── live/
│ ├── network-prod/ap-northeast-1/ # core: TGW, inspection, DNS
│ ├── workload-a-prod/ap-northeast-1/
│ └── workload-a-dev/ap-northeast-1/
└── policies/ # OPA/Checkov rules: no 0.0.0.0/0 SSH, flow logs onHardcoded values work in one account and break in the next. The exam calls this out explicitly in Task 2.4.
| Hardcoded value | Why it breaks | Use instead |
|---|---|---|
| CIDR blocks | Overlaps appear as soon as a second team copies the template | IPAM pool allocation (ipv4_ipam_pool_id + netmask length) |
AZ names (ap-northeast-1a) | AZ names map to different physical zones per account; some accounts lack a zone | AZ IDs (apne1-az1) or data.aws_availability_zones / Fn::GetAZs |
| Subnet math | Off-by-one errors and overlaps | cidrsubnet() in Terraform, Fn::Cidr in CloudFormation |
| Resource IDs (TGW, PHZ, prefix list) | IDs differ per account and Region | SSM parameters, remote state, data lookups, CloudFormation exports |
| Account IDs and ARNs | Break when promoted between accounts | data.aws_caller_identity, AWS::AccountId, AWS::Partition |
| Partner or SaaS IP lists | They change without notice | Customer-managed prefix lists updated by automation |
| AMI IDs for appliances | Region-specific, deprecated over time | SSM public parameters (/aws/service/...) |
The network account owns the pools; workload accounts ask for "a /20" and IPAM guarantees no overlap.
# network account: IPAM, top-level pool, regional pool
resource "aws_vpc_ipam" "main" {
operating_regions { region_name = "ap-northeast-1" }
}
resource "aws_vpc_ipam_pool" "apne1" {
address_family = "ipv4"
ipam_scope_id = aws_vpc_ipam.main.private_default_scope_id
locale = "ap-northeast-1"
allocation_default_netmask_length = 20
allocation_min_netmask_length = 16
allocation_max_netmask_length = 24
}
resource "aws_vpc_ipam_pool_cidr" "apne1" {
ipam_pool_id = aws_vpc_ipam_pool.apne1.id
cidr = "10.64.0.0/12"
}
# share the pool with the organization via RAM (not shown)
# workload account: no CIDR literal anywhere
resource "aws_vpc" "app" {
ipv4_ipam_pool_id = var.ipam_pool_id # from SSM / remote state
ipv4_netmask_length = 20
}
locals {
private = [for i in range(3) : cidrsubnet(aws_vpc.app.cidr_block, 4, i)]
}locale ties the pool to a Region; a VPC can only draw from a pool in its own Region.cidrsubnet(), so the VPC CIDR is the only input.Both read the pool from a parameter and derive subnets, so the template is identical in every account.
# CloudFormation (YAML)
Parameters:
IpamPoolId:
Type: AWS::SSM::Parameter::Value<String>
Default: /network/ipam/apne1-pool-id
Resources:
Vpc:
Type: AWS::EC2::VPC
Properties:
Ipv4IpamPoolId: !Ref IpamPoolId
Ipv4NetmaskLength: 20
SubnetA:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref Vpc
AvailabilityZone: !Select [0, !GetAZs ""]
CidrBlock: !Select [0, !Cidr [!GetAtt Vpc.CidrBlock, 6, 8]]// CDK (TypeScript)
const poolId = ssm.StringParameter.valueForStringParameter(
this, '/network/ipam/apne1-pool-id');
const vpc = new ec2.Vpc(this, 'App', {
ipAddresses: ec2.IpAddresses.awsIpamAllocation({
ipv4IpamPoolId: poolId,
ipv4NetmaskLength: 20,
defaultSubnetIpv4NetmaskLength: 24,
}),
maxAzs: 3,
natGateways: 1, // default is one per AZ: cost!
subnetConfiguration: [
{ name: 'public', subnetType: ec2.SubnetType.PUBLIC },
{ name: 'private', subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS },
],
});!Cidr [ipBlock, count, cidrBits] uses host bits: 8 bits = /24 subnets. In CDK, check cdk diff before deploy: the Vpc construct creates NAT gateways, route tables and an IGW for you.
Most real designs span a network account and many workload accounts. Automate both sides of every handshake.
create-vpc-association-authorization then associate-vpc-with-hosted-zone.A network change is safe when you know what it replaces and you can prove reachability did not regress.
terraform plan / change set.terraform plan -refresh-only on a schedule.Encode "app can reach DB on 5432" and "DB cannot reach the internet" as resources; fail the pipeline when the answer changes.
resource "aws_ec2_network_insights_path" "app_to_db" {
source = aws_instance.app.primary_network_interface_id
destination = var.db_eni_id # the DB's ENI ID
protocol = "tcp"
destination_port = 5432
}
resource "aws_ec2_network_insights_analysis" "app_to_db" {
network_insights_path_id = aws_ec2_network_insights_path.app_to_db.id
wait_for_completion = true
lifecycle {
replace_triggered_by = [aws_security_group.db]
postcondition {
condition = self.path_found
error_message = "app -> db:5432 is no longer reachable"
}
}
}replace_triggered_by re-runs the analysis when the SG changes.!self.path_found.create-network-insights-path → start-network-insights-analysis → read NetworkPathFound and Explanations.React to network events with EventBridge, Lambda, Config and Systems Manager, and report incidents automatically.
EventBridge is the bus. Know which service emits which event and in which Region.
| Source | Example events | Notes |
|---|---|---|
| CloudTrail (API calls) | AuthorizeSecurityGroupIngress, CreateRoute, DeleteFlowLogs | Management events; arrive as "AWS API Call via CloudTrail" |
| Network Manager | Topology change, routing update (route added/removed in TGW), VPN/Connect peer status | Delivered in us-west-2 (Network Manager home Region); TGW must be registered |
| CloudWatch alarms | TunnelState = 0 (AWS/VPN), ConnectionState (AWS/DX), NAT ErrorPortAllocation | Alarm state change events, or alarm actions to SNS |
| AWS Health | Direct Connect maintenance, VPN tunnel endpoint replacement | Plan failover before scheduled maintenance |
| AWS Config | Compliance change for restricted-ssh, vpc-flow-logs-enabled | Pair with SSM Automation remediation |
| GuardDuty / Security Hub | Port probing, DNS exfiltration, unusual traffic findings | Security incident reporting (Task 4.1) |
SNS AmazonIpSpaceChanged | AWS published ip-ranges.json changed | Trigger prefix list or firewall updates |
Detect the API call, then revoke the rule and notify. Prevention (SCPs, Firewall Manager) is better; this is the safety net.
{
"source": ["aws.ec2"],
"detail-type": ["AWS API Call via CloudTrail"],
"detail": {
"eventName": ["AuthorizeSecurityGroupIngress"],
"requestParameters": {
"ipPermissions": { "items": {
"ipRanges": { "items": {
"cidrIp": ["0.0.0.0/0"] } } } }
}
}
}# Lambda (Python) target of the rule
import boto3
ec2 = boto3.client("ec2")
def handler(event, _):
p = event["detail"]["requestParameters"]
gid = p["groupId"]
for perm in p["ipPermissions"]["items"]:
if perm.get("fromPort") in (22, 3389):
ec2.revoke_security_group_ingress(
GroupId=gid,
IpPermissions=[{
"IpProtocol": perm["ipProtocol"],
"FromPort": perm["fromPort"],
"ToPort": perm["toPort"],
"IpRanges": [{"CidrIp": "0.0.0.0/0"}]}])
# then publish to SNS with the caller identity
# event["detail"]["userIdentity"]["arn"]Managed alternative with less code: Config rule restricted-ssh + automatic remediation with the SSM document AWS-DisablePublicAccessForSecurityGroup. Org-wide: a Firewall Manager security group content audit policy.
VPN and Direct Connect publish state metrics; alarm on them and route the alarm to people and to automation.
# VPN: alarm when any tunnel of the connection is down
aws cloudwatch put-metric-alarm \
--alarm-name vpn-onprem-tunnel-down \
--namespace AWS/VPN --metric-name TunnelState \
--dimensions Name=VpnId,Value=vpn-0abc1234 \
--statistic Minimum --period 60 --evaluation-periods 3 \
--threshold 1 --comparison-operator LessThanThreshold \
--treat-missing-data breaching \
--alarm-actions arn:aws:sns:ap-northeast-1:111122223333:netops
# Direct Connect: connection down
aws cloudwatch put-metric-alarm \
--alarm-name dx-primary-down \
--namespace AWS/DX --metric-name ConnectionState \
--dimensions Name=ConnectionId,Value=dxcon-abc123 \
--statistic Minimum --period 60 --evaluation-periods 2 \
--threshold 1 --comparison-operator LessThanThreshold \
--alarm-actions arn:aws:sns:ap-northeast-1:111122223333:netopsTunnelState (1 up, 0 down), TunnelDataIn/Out per tunnel.ConnectionState, ConnectionBpsEgress/Ingress, ConnectionLightLevelRx (optics) for DX.VirtualInterfaceBpsEgress per VIF to spot a failover.ErrorPortAllocation, PacketsDropCount.PacketDropCountNoRoute, PacketDropCountBlackhole.Minimum for state metrics so a single down sample counts.Turn findings into tickets and chat messages with no human copy-paste.
REJECT spikes.aws.guardduty with severity ≥ 7.GuardDuty network findings use VPC Flow Logs, DNS logs and CloudTrail as foundational sources: you do not need to enable flow logs for GuardDuty to analyze them.
IP lists for SaaS partners, CloudFront origins or on-premises ranges change. Automate the update, not the rule.
# subscribe a Lambda to AWS IP range changes (topic is in us-east-1)
aws sns subscribe --region us-east-1 --protocol lambda \
--topic-arn \
arn:aws:sns:us-east-1:806199016981:AmazonIpSpaceChanged \
--notification-endpoint \
arn:aws:lambda:us-east-1:111122223333:function:sync-ranges
# the Lambda rewrites a customer-managed prefix list
aws ec2 get-managed-prefix-list-entries \
--prefix-list-id pl-0abc --query 'Entries'
aws ec2 modify-managed-prefix-list --prefix-list-id pl-0abc \
--current-version 7 \
--add-entries Cidr=203.0.113.0/24,Description=partner-a \
--remove-entries Cidr=198.51.100.0/24--current-version gives optimistic locking: concurrent writers fail instead of overwriting.com.amazonaws.global.cloudfront.origin-facing.Task 2.4 asks for efficiency "while maintaining the lowest possible cost". Automation is how you get there at scale.
Wrong answers that look right.
us-east-1a in one account is not the same physical AZ as in another.Scenario → answer → why.
| Scenario | Answer | Why |
|---|---|---|
| 200 accounts each create VPCs; CIDRs keep overlapping | IPAM pools shared with RAM; IaC allocates by netmask length | IPAM guarantees unique allocations and tracks utilization |
| Every new account must get flow logs and S3/DynamoDB gateway endpoints | CloudFormation StackSet with service-managed permissions and auto-deployment on the OU | Deploys automatically to new accounts with no custom code |
| Ops must know within a minute when a VPN tunnel drops | CloudWatch alarm on TunnelState → SNS | Native metric; no polling script |
| Pipeline must prove the DB is never reachable from the internet | Network Access Analyzer scope (or Reachability Analyzer path) in CI | Encodes intent; fails the build on regression |
| Partner IP list changes monthly; 30 SGs reference it | Customer-managed prefix list referenced by the SGs, updated by automation | One update, all rules follow |
| Template works in account A but fails in B with "subnet AZ not supported" | Replace hardcoded AZ names with AZ IDs or GetAZs | AZ names map per account |
Explain the answer and name the exact feature.
--current-version protect on a prefix list update?Tasks 3.1, 3.3 — get the throughput you pay for, stay inside the quotas, and stop paying for bytes you do not need to move.
Instance bandwidth, per-flow limits, interface types, placement, MTU per path, and multicast.
The instance type sets the ceiling; the number of flows, the destination and the credit bucket decide what you actually get.
describe-instance-types --query ...NetworkCards[0].BaselineBandwidthInGbps.bw_out_allowance_exceeded, pps_allowance_exceeded, conntrack_allowance_exceeded.Throughput fix order: more flows (parallel streams, MPTCP) → cluster placement group or ENA Express → bigger instance → jumbo frames on paths that support them.
Task 3.3 skill: "selecting the right network interface for the best performance".
| Interface | What it is | Use when | Limits |
|---|---|---|---|
| ENI | The virtual NIC object: IPs, SGs, MAC, source/dest check | Always; extra ENIs for dual-homed appliances or management networks | Number per instance depends on type |
| ENA (enhanced networking) | SR-IOV driver; required on current-generation types | Default for high throughput and low latency | Per-flow 5 Gbps outside a cluster PG |
| ENA Express | ENA + AWS SRD transport: multipath, congestion-aware | Single large flows (storage, replication, media) between supported instances | Both sides enabled and supported types; no middleboxes in the path; slight latency increase when idle |
| EFA | ENA + OS-bypass interface (libfabric) for MPI and NCCL | Tightly coupled HPC and ML training | Same subnet; SG must allow all traffic to and from itself; OS-bypass traffic is not routable |
aws ec2 modify-network-interface-attribute \
--network-interface-id eni-0abc1234 \
--ena-srd-specification \
'EnaSrdEnabled=true,EnaSrdUdpSpecification={EnaSrdUdpEnabled=true}'
ethtool -S ens5 | grep -E 'ena_srd|allowance' # SRD counters, dropsThe smallest MTU on the path wins. Know these values cold.
| Path | Max MTU | PMTUD |
|---|---|---|
| Inside a VPC, intra-Region peering | 9001 | Yes (allow ICMP) |
| Transit Gateway (VPC, DX, Connect, peering attachments) | 8500 | VPC and Connect attachments only |
| Inter-Region VPC peering | 8500 | — |
| VPC endpoints (PrivateLink), NAT gateway | 8500 | Endpoints: no; NAT: yes |
| Direct Connect private VIF | 9001 (opt-in) | — |
| Direct Connect transit VIF | 8500 (opt-in) | — |
| Internet gateway, public VIF, between Regions without peering or TGW | 1500 | — |
| Site-to-Site VPN tunnel | Max 1446 / MSS 1406 (AES-GCM, no NAT-T); 1422 / 1382 with AES-CBC + SHA-256 + NAT-T | No |
ip link set dev ens5 mtu 9001 (persist in netplan or the ifcfg file).ip route add 0.0.0.0/0 via 10.0.0.1 mtu 1500.tracepath <ip> or ping -M do -s 8972 <ip> (8972 + 28 = 9000).Task 3.2 skill: "troubleshooting packet size mismatches in a VPC to restore network connectivity".
scp stalls; TLS handshake hangs at certificate exchange.ACCEPT both ways: nothing is "blocked".When one box is not enough, add more of them and spread the flows. These are the numbers that decide that.
| Service | Throughput | Scale-out pattern |
|---|---|---|
| NAT gateway | 5 Gbps scaling to 100 Gbps; 1 M scaling to 10 M pps; 55,000 connections per unique destination per IP | Add secondary IPs (up to 8 IPs per NAT gateway: 1 primary + 7 secondary), or one NAT per subnet |
| Interface / GWLB endpoint | 10 Gbps per AZ scaling to 100 Gbps | Spread across AZs |
| Transit Gateway VPC attachment | Up to 100 Gbps per direction per AZ; up to 7.5 M pps | Multiple AZs, multiple attachments |
| Site-to-Site VPN tunnel | Up to 1.25 Gbps / 140,000 pps (standard); up to 5 Gbps / 400,000 pps (large) | ECMP across tunnels on TGW (dynamic routing only) |
| TGW Connect peer (GRE) | Up to 5 Gbps, 4 peers per attachment (20 Gbps) | ECMP across peers and attachments |
| Direct Connect | Port speed: 1, 10, 100 or 400 Gbps dedicated; hosted from 50 Mbps | LAG (up to 4 links below 100G, 2 at 100G), more connections |
A VPN over TGW with ECMP across 4 tunnels of 1.25 Gbps can reach ~5 Gbps aggregate, but any single flow still rides one tunnel.
The only way to do multicast in AWS. Task 3.3 skill: "implementing a multicast capability within a VPC and on-premises".
0.0.0.0/32 to 224.0.0.1 every two minutes: SG and NACL must allow protocol 2.Users far from the Region suffer on the public internet. Put them on the AWS backbone as early as possible.
Exam keyword: "static IPs", "UDP", "non-HTTP", "fast failover without DNS" → Global Accelerator. "Cache", "static content", "HTTP" → CloudFront.
Wrong answers that look right.
Default quotas that shape designs, which ones can be raised, and how to watch them.
Defaults per Region per account unless noted. "No" means you must design around it.
| Quota | Default | Adjustable |
|---|---|---|
| VPCs per Region | 5 | Yes (hundreds) |
| IPv4 CIDR blocks per VPC | 5 | Yes, up to 50 |
| Routes per route table (non-propagated) | 500 | Yes, up to 1,000 |
| Propagated routes per route table | 100 | No — advertise a summary or default route |
| Inbound or outbound rules per security group | 60 (IPv4 and IPv6 counted separately) | Yes; rules × SGs per ENI ≤ 1,000 |
| Security groups per ENI | 5 | Yes, up to 16 |
| Rules per network ACL (each direction) | 20 | Yes, up to 40 |
| Active peering connections per VPC | 50 | Yes, up to 125 |
| Interface + GWLB endpoints per VPC | 50 | Yes |
| Network Address Usage units per VPC | 64,000 | Yes, up to 256,000 |
Exceeding a BGP prefix limit does not drop a few routes: on Direct Connect the session goes down.
| Quota | Value | Adjustable / what happens |
|---|---|---|
| Prefixes from on-premises per private or transit VIF BGP session | 100 each for IPv4 and IPv6 | Up to 1,000 with prefix controls; over the limit the session goes idle (DOWN) |
| Prefixes per public VIF BGP session | 1,000 | No |
| Prefixes per TGW from AWS to on-premises on a transit VIF | 200 (IPv4 + IPv6 combined) | Contact your SA/TAM — summarize VPC CIDRs in allowed prefixes |
| Dynamic routes from a CGW to a VPN on a VGW | 100 | No |
| Dynamic routes from a CGW to a VPN on a TGW | 1,000 | No |
| Total routes across all route tables of one TGW | 10,000 | Contact your SA/TAM |
| VIFs per dedicated connection (private/public + transit) | 51 total, of which up to 4 transit | No |
| VGWs / TGWs per Direct Connect gateway | 20 / 6 | No |
Design reflex: summarize. Allocate each Region or environment a contiguous block (IPAM) so it can be advertised as one prefix.
These do not fail a deployment; they fail some requests under load, which is harder to find.
systemd-resolved, dnsmasq, NodeLocal DNSCache on EKS).ErrorPortAllocation > 0, failed connections to one popular endpoint.Service Quotas exposes current values and usage; CloudWatch alarms on usage turn quotas into normal monitoring.
# current value of a quota (SG rules per SG)
aws service-quotas get-service-quota \
--service-code vpc --quota-code L-0EA8095F
# list all VPC quotas with their values
aws service-quotas list-service-quotas --service-code vpc \
--query 'Quotas[].[QuotaName,Value,Adjustable]' --output table
# request an increase
aws service-quotas request-service-quota-increase \
--service-code vpc --quota-code L-0EA8095F --desired-value 100
# Network Address Usage of a VPC (must be enabled per VPC)
aws ec2 modify-vpc-attribute --vpc-id vpc-0abc \
--enable-network-address-usage-metricsWhere bytes cost money in AWS networking, and the design patterns that remove those charges.
Relative, not absolute: check the pricing pages for your Region. Inbound from the internet is free.
Charges stack: a byte from a private subnet to the internet pays NAT processing and internet data transfer out; through a centralized egress VPC it also pays TGW processing.
Know the billing dimensions; the exam asks for "most cost-effective", not exact prices.
| Service | Billed on | Cost lever |
|---|---|---|
| Data transfer | Per GB: internet out, inter-Region, inter-AZ (each direction); same AZ via private IP free | Keep chatty traffic in one AZ; cache at the edge |
| Public IPv4 addresses | Per hour for every public IPv4 address, attached or not (since Feb 2024) | IPv6, fewer public IPs, release idle EIPs |
| NAT gateway | Per hour per gateway + per GB processed | Gateway endpoints for S3/DynamoDB; interface endpoints for heavy AWS API use |
| VPC peering | No hourly or processing charge; normal inter-AZ / inter-Region transfer | Cheapest for a few high-volume VPC pairs |
| Transit Gateway | Per attachment-hour + per GB processed (sender side by default; flexible allocation since Nov 2025) | Peer the heaviest pairs directly |
| PrivateLink | Per endpoint per AZ per hour + per GB processed | Share endpoints from a central VPC |
| Direct Connect | Port-hours + data transfer out at DX rates; inbound free | Steady high volume; hosted connection for smaller needs |
| Site-to-Site VPN | Per connection-hour + normal data transfer out | Cheap for low volume and backup paths |
Most savings come from removing a hop or keeping traffic in one AZ.
Decide with data, not intuition: compare GB processed (CloudWatch BytesOutToDestination on NAT, BytesIn on TGW attachments) with each option's hourly cost.
A network bill is a list of usage types. Map them back to flows.
APN1-NatGateway-Bytes, APN1-DataTransfer-Regional-Bytes (inter-AZ), APN1-DataTransfer-Out-Bytes, APN1-TransitGateway-Bytes.pkt-srcaddr, pkt-dstaddr, az-id, flow-direction and bytes → Athena: top talkers across AZs.Wrong answers that look right.
Scenario → answer → why.
| Scenario | Answer | Why |
|---|---|---|
| Replication job between two instances is stuck at 5 Gbps on 25 Gbps instances | ENA Express on both (or cluster PG + parallel streams) | Single-flow limit, not instance limit |
| Large NAT gateway bill; workloads mostly read S3 in the same Region | S3 gateway endpoint on the private route tables | Removes NAT processing; gateway endpoints are free |
| Random DNS timeouts on a busy proxy fleet; Resolver itself is healthy | Local DNS cache on each instance | 1,024 pps link-local limit per ENI |
| BGP over a private VIF keeps going down after a new site was added | Summarize prefixes or raise the limit with prefix controls | Over 100 prefixes takes the session idle |
| Need 4 Gbps to on-premises over VPN | TGW VPN with BGP and ECMP across tunnels, or 5 Gbps large tunnels | Standard tunnel is 1.25 Gbps; ECMP needs dynamic routing |
| Market data feed must reach 40 receivers in 3 VPCs | TGW multicast domain | Only native multicast option |
Explain the answer and name the exact number or feature.
ErrorPortAllocation tell you?The baseline every other design builds on. Each tier has its own subnets and route tables.
Rules: ALB in public subnets only; app in private subnets with a NAT gateway per AZ; DB in isolated subnets (no route to 0.0.0.0/0); SG chain ALB-SG → App-SG → DB-SG by reference; S3/DynamoDB gateway endpoints everywhere.
The network account owns the TGW and its route tables; workload accounts only attach.
Isolation comes from TGW route tables: prod and dev never propagate into each other's table, both reach shared services. Shared services hosts Resolver endpoints and centralized interface endpoints; PHZs and Resolver rules are shared with RAM or Route 53 Profiles.
One inspection VPC filters spoke-to-internet and spoke-to-spoke traffic for the whole organization.
Appliance mode keeps both directions of a flow on the same firewall AZ. Since Jun 2025 Network Firewall can also attach natively to the TGW, which removes the inspection VPC plumbing; the routing logic stays the same.
High resiliency: two DX locations, BGP chooses the active path, VPN takes over only when both DX paths fail.
For identical prefixes, AWS already prefers routes learned over DX to VPN routes; a more specific prefix over the VPN would win, so never advertise one. On-premises, use local preference so the return path also prefers DX.
One JSON policy defines Regions, segments, attachment rules and inter-segment routing; Cloud WAN builds the global mesh.
Attachments land in a segment by tag (attachment policies). Segments are isolated by default; share them or send traffic through an inspection segment with segment actions. Alternative at smaller scale: one TGW per Region with inter-Region TGW peering and static routes.
Most questions have two technically valid answers. The constraint in the stem picks the winner.
Matching questions: every pair must be right for credit. Start with the pairs you are sure of and eliminate.
Common stems and the answer they almost always point to.
| If the stem says… | Think first of |
|---|---|
| Connect thousands of VPCs and on-premises, transitive routing, segmentation | Transit Gateway with multiple route tables |
| Global network across Regions defined by policy, segments | AWS Cloud WAN |
| Expose a service to other VPCs or accounts with overlapping CIDRs | PrivateLink (NLB endpoint service) |
| Consistent private connection, predictable latency, high volume | Direct Connect (+ VPN or second DX for resiliency) |
| Encrypt Direct Connect | MACsec (layer 2, dedicated) or IPsec VPN over DX (private IP VPN on TGW) |
| On-premises must resolve private hosted zone names | Resolver inbound endpoint + conditional forwarder |
| VPC must resolve on-premises domain | Resolver outbound endpoint + forwarding rule (share with RAM) |
Common stems and the answer they almost always point to.
| If the stem says… | Think first of |
|---|---|
| Static anycast IPs, TCP/UDP, fast regional failover | AWS Global Accelerator |
| Third-party firewall appliances inline, transparent, scalable | Gateway Load Balancer (GENEVE) + GWLB endpoints |
| Stateful inspection across AZs breaks through TGW | Enable appliance mode on the inspection VPC attachment |
| Preserve client IP at the target | NLB (client IP preservation) or ALB X-Forwarded-For; proxy protocol v2 for TLS passthrough |
| Find a configuration reason two ENIs cannot talk | VPC Reachability Analyzer |
| Inspect packet payloads for an intermittent problem | VPC Traffic Mirroring |
| Large transfers hang but small requests work | MTU mismatch: allow ICMP, lower MTU, clamp MSS |
One sandbox account (plus a second for Lab 2), a budget alarm, and a simulated data center.
lab-a; second account lab-b (CLI profile) for Lab 2.jq, dig, iperf3.export AWS_REGION=ap-northeast-1 (AZ IDs apne1-az1… differ per Region).Project=netlab and find leftovers with the Resource Groups Tagging API.onprem 10.200.0.0/16 stands in for the data center.cgw (10.200.0.10) with an Elastic IP and source/destination check disabled.onprem route table: 10.0.0.0/8 → the cgw ENI.# budget alarm first
aws budgets create-budget \
--account-id $ACCT --budget '{
"BudgetName":"netlab","TimeUnit":"MONTHLY",
"BudgetType":"COST","BudgetLimit":
{"Amount":"30","Unit":"USD"}}' \
--notifications-with-subscribers '[{
"Notification":{
"NotificationType":"ACTUAL",
"ComparisonOperator":"GREATER_THAN",
"Threshold":80},
"Subscribers":[{"SubscriptionType":
"EMAIL","Address":"you@example.com"}]}]'
# shell into a private instance
aws ec2 create-instance-connect-endpoint \
--subnet-id $PRV \
--security-group-ids $SG_EICE
aws ec2-instance-connect ssh \
--instance-id $I --os-user ec2-user \
--connection-type eiceEach lab = Build, Verify, Break it, Clean up. Save every lab's commands as labN.sh: Lab 8 rebuilds Lab 1 from its script.
Goal: subnets, IGW, egress-only IGW, NAT and flow logs by hand, so every later abstraction is familiar.
VPC=$(aws ec2 create-vpc --cidr-block 10.10.0.0/16 \
--amazon-provided-ipv6-cidr-block --query Vpc.VpcId --output text)
V6=$(aws ec2 describe-vpcs --vpc-ids $VPC --output text --query \
'Vpcs[0].Ipv6CidrBlockAssociationSet[0].Ipv6CidrBlock')
echo $V6 # e.g. 2406:da14:abc:de00::/56
P6=${V6%00::/56} # prefix for the /64 subnets
sub() { aws ec2 create-subnet --vpc-id $VPC --cidr-block $1 \
--availability-zone-id apne1-az1 --ipv6-cidr-block $2 \
--query Subnet.SubnetId --output text; }
PUB=$(sub 10.10.0.0/24 ${P6}00::/64)
PRV=$(sub 10.10.10.0/24 ${P6}01::/64)
IGW=$(aws ec2 create-internet-gateway --output text \
--query InternetGateway.InternetGatewayId)
aws ec2 attach-internet-gateway --internet-gateway-id $IGW \
--vpc-id $VPC
EIGW=$(aws ec2 create-egress-only-internet-gateway --vpc-id $VPC \
--output text \
--query EgressOnlyInternetGateway.EgressOnlyInternetGatewayId)
EIP=$(aws ec2 allocate-address --domain vpc \
--query AllocationId --output text)
NAT=$(aws ec2 create-nat-gateway --subnet-id $PUB \
--allocation-id $EIP --query NatGateway.NatGatewayId --output text)
aws ec2 wait nat-gateway-available --nat-gateway-ids $NATrtb() { aws ec2 create-route-table --vpc-id $VPC \
--query RouteTable.RouteTableId --output text; }
RTPUB=$(rtb); RTPRV=$(rtb)
aws ec2 create-route --route-table-id $RTPUB \
--destination-cidr-block 0.0.0.0/0 --gateway-id $IGW
aws ec2 create-route --route-table-id $RTPRV \
--destination-cidr-block 0.0.0.0/0 --nat-gateway-id $NAT
aws ec2 create-route --route-table-id $RTPRV \
--destination-ipv6-cidr-block ::/0 \
--egress-only-internet-gateway-id $EIGW
aws ec2 associate-route-table --route-table-id $RTPUB --subnet-id $PUB
aws ec2 associate-route-table --route-table-id $RTPRV --subnet-id $PRV
aws logs create-log-group --log-group-name /netlab/flow
ROLE=arn:aws:iam::$ACCT:role/netlab-flowlogs
FL=$(aws ec2 create-flow-logs --resource-type VPC --resource-ids $VPC \
--log-destination-type cloud-watch-logs --log-group-name /netlab/flow \
--traffic-type ALL --deliver-logs-permission-arn $ROLE \
--query 'FlowLogIds[0]' --output text)
AMI=/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64
I=$(aws ec2 run-instances --image-id resolve:ssm:$AMI \
--instance-type t3.micro --subnet-id $PRV --ipv6-address-count 1 \
--security-group-ids $SG_APP \
--query 'Instances[0].InstanceId' --output text)Prerequisites: IAM role netlab-flowlogs trusted by vpc-flow-logs.amazonaws.com; SG_APP allows TCP 22 from the EICE security group; an EICE in $PRV.
Every break has one symptom and one tool that explains it.
curl -4 -s https://checkip.amazonaws.com prints the NAT EIP, not the instance IP.ping -6 -c3 2001:4860:4860::8888 works through the egress-only IGW./netlab/flow: fields srcaddr, dstaddr, dstport, action | sort @timestamp desc | limit 20.0.0.0.0/0 route in $RTPRV → IPv4 curl times out, IPv6 still works. Reachability Analyzer from the ENI to the IGW names the missing route.curl hangs; Flow Logs show inbound REJECT to high ports. Fix: allow inbound TCP 1024–65535.aws ec2 terminate-instances \
--instance-ids $I
aws ec2 delete-instance-connect-endpoint \
--instance-connect-endpoint-id $EICE
aws ec2 delete-nat-gateway \
--nat-gateway-id $NAT
aws ec2 wait nat-gateway-deleted \
--nat-gateway-ids $NAT
aws ec2 release-address --allocation-id $EIP
aws ec2 delete-flow-logs --flow-log-ids $FL
aws logs delete-log-group \
--log-group-name /netlab/flow
aws ec2 detach-internet-gateway \
--internet-gateway-id $IGW --vpc-id $VPC
aws ec2 delete-internet-gateway \
--internet-gateway-id $IGW
aws ec2 delete-egress-only-internet-gateway \
--egress-only-internet-gateway-id $EIGW
for S in $PUB $PRV; do
aws ec2 delete-subnet --subnet-id $S; done
for R in $RTPUB $RTPRV; do
aws ec2 delete-route-table \
--route-table-id $R; done
aws ec2 delete-vpc --vpc-id $VPCGoal: reach S3 and STS without the internet, restrict them, and publish a service to another account that uses the same CIDR.
# start from the Lab 1 VPC; bucket netlab-$ACCT exists
cat > s3-ep.json <<EOF
{"Statement":[{"Effect":"Allow","Principal":"*",
"Action":["s3:GetObject","s3:PutObject","s3:ListBucket"],
"Resource":["arn:aws:s3:::netlab-$ACCT",
"arn:aws:s3:::netlab-$ACCT/*"]}]}
EOF
S3EP=$(aws ec2 create-vpc-endpoint --vpc-id $VPC \
--service-name com.amazonaws.$AWS_REGION.s3 \
--route-table-ids $RTPRV --policy-document file://s3-ep.json \
--query VpcEndpoint.VpcEndpointId --output text)
aws ec2 modify-vpc-attribute --vpc-id $VPC \
--enable-dns-hostnames '{"Value":true}'
STSEP=$(aws ec2 create-vpc-endpoint --vpc-id $VPC \
--vpc-endpoint-type Interface \
--service-name com.amazonaws.$AWS_REGION.sts \
--subnet-ids $PRV --security-group-ids $SG_443 \
--private-dns-enabled \
--query VpcEndpoint.VpcEndpointId --output text)# provider (lab-a): nginx instance $I behind an internal NLB
NLB=$(aws elbv2 create-load-balancer --name netlab-nlb \
--type network --scheme internal --subnets $PRV \
--query 'LoadBalancers[0].LoadBalancerArn' --output text)
TG=$(aws elbv2 create-target-group --name netlab-tg --protocol TCP \
--port 80 --vpc-id $VPC --target-type instance \
--query 'TargetGroups[0].TargetGroupArn' --output text)
aws elbv2 register-targets --target-group-arn $TG --targets Id=$I
aws elbv2 create-listener --load-balancer-arn $NLB --protocol TCP \
--port 80 --default-actions Type=forward,TargetGroupArn=$TG
SVC=$(aws ec2 create-vpc-endpoint-service-configuration \
--network-load-balancer-arns $NLB --acceptance-required \
--query ServiceConfiguration.ServiceId --output text)
aws ec2 modify-vpc-endpoint-service-permissions --service-id $SVC \
--add-allowed-principals arn:aws:iam::$ACCT_B:root
# consumer (lab-b): its VPC is also 10.10.0.0/16, on purpose
EP_B=$(aws ec2 create-vpc-endpoint --profile lab-b \
--vpc-id $VPC_B --vpc-endpoint-type Interface \
--service-name com.amazonaws.vpce.$AWS_REGION.$SVC \
--subnet-ids $SUB_B --security-group-ids $SG_B \
--query VpcEndpoint.VpcEndpointId --output text)
aws ec2 accept-vpc-endpoint-connections --service-id $SVC \
--vpc-endpoint-ids $EP_BSG_443 allows TCP 443 from 10.10.0.0/16. The nginx SG allows TCP 80 from 10.10.0.0/16: PrivateLink traffic arrives from NLB private IPs, not from the consumer.
Prove the path in DNS, in the route table, and in the policy.
describe-route-tables --route-table-ids $RTPRV shows a pl-… route to the gateway endpoint.aws s3 ls s3://netlab-$ACCT works; another bucket returns AccessDenied (endpoint policy).dig +short sts.$AWS_REGION.amazonaws.com returns 10.10.x.x.lab-b: curl http://<endpoint DNS name> returns nginx although both VPCs are 10.10.0.0/16.SG_443 → aws sts get-caller-identity hangs while dig still returns private IPs: it resolves, but is not allowed.modify-vpc-endpoint --vpc-endpoint-id $STSEP --no-private-dns-enabled → dig returns public IPs; calls go through NAT again.reject-vpc-endpoint-connections --service-id $SVC --vpc-endpoint-ids $EP_B → consumer state rejected, curl fails.aws ec2 delete-vpc-endpoints \
--profile lab-b --vpc-endpoint-ids $EP_B
aws ec2 \
delete-vpc-endpoint-service-configurations \
--service-ids $SVC
aws elbv2 delete-load-balancer \
--load-balancer-arn $NLB
aws elbv2 delete-target-group \
--target-group-arn $TG
aws ec2 delete-vpc-endpoints \
--vpc-endpoint-ids $STSEP $S3EP
# lab-b: terminate the test instance,
# delete its VPC
# lab-a: keep the Lab 1 VPC for Lab 10,
# or run the Lab 1 cleanupGoal: prod 10.11.0.0/16, dev 10.12.0.0/16, shared 10.13.0.0/16; two TGW route tables; Route Analyzer to explain paths.
TGW=$(aws ec2 create-transit-gateway --description netlab \
--options AmazonSideAsn=64512,DefaultRouteTableAssociation=disable,\
DefaultRouteTablePropagation=disable \
--query TransitGateway.TransitGatewayId --output text)
until [ "$(aws ec2 describe-transit-gateways \
--transit-gateway-ids $TGW --output text \
--query 'TransitGateways[0].State')" = available ]; do sleep 15; done
att() { aws ec2 create-transit-gateway-vpc-attachment \
--transit-gateway-id $TGW --vpc-id $1 --subnet-ids $2 \
--output text --query \
TransitGatewayVpcAttachment.TransitGatewayAttachmentId; }
A_PROD=$(att $VPC_PROD $SUB_PROD)
A_DEV=$(att $VPC_DEV $SUB_DEV)
A_SH=$(att $VPC_SH $SUB_SH)
until [ -z "$(aws ec2 describe-transit-gateway-attachments --output text \
--transit-gateway-attachment-ids $A_PROD $A_DEV $A_SH \
--query 'TransitGatewayAttachments[?State!=`available`].State')" ]
do sleep 15; done
rt() { aws ec2 create-transit-gateway-route-table --output text \
--transit-gateway-id $TGW \
--query TransitGatewayRouteTable.TransitGatewayRouteTableId; }
RT_SPOKE=$(rt); RT_SH=$(rt)assoc() { aws ec2 associate-transit-gateway-route-table \
--transit-gateway-route-table-id $1 \
--transit-gateway-attachment-id $2; }
prop() { aws ec2 enable-transit-gateway-route-table-propagation \
--transit-gateway-route-table-id $1 \
--transit-gateway-attachment-id $2; }
assoc $RT_SPOKE $A_PROD; assoc $RT_SPOKE $A_DEV; assoc $RT_SH $A_SH
prop $RT_SPOKE $A_SH # spokes see shared only
prop $RT_SH $A_PROD; prop $RT_SH $A_DEV # shared sees both
for R in $RT_PROD $RT_DEV $RT_SHVPC; do # the VPC route tables
aws ec2 create-route --route-table-id $R \
--destination-cidr-block 10.0.0.0/8 --transit-gateway-id $TGW
done
# Network Manager (home Region us-west-2) for Route Analyzer
GN=$(aws networkmanager create-global-network --region us-west-2 \
--query GlobalNetwork.GlobalNetworkId --output text)
TGW_ARN=arn:aws:ec2:$AWS_REGION:$ACCT:transit-gateway/$TGW
aws networkmanager register-transit-gateway --region us-west-2 \
--global-network-id $GN --transit-gateway-arn $TGW_ARNEach VPC: one attachment subnet (same AZ ID in all three) and a test instance whose SG allows ICMP from 10.0.0.0/8. Create an EICE in prod and in shared.
Isolation is just "no route". Prove it with the route tables, then with Route Analyzer.
search-transit-gateway-routes --transit-gateway-route-table-id $RT_SPOKE --filters Name=state,Values=active lists only 10.13.0.0/16.ping works; prod → dev fails, by design.aws networkmanager start-route-analysis --region us-west-2 --global-network-id $GN --source TransitGatewayAttachmentArn=<prod att ARN>,IpAddress=10.11.0.10 --destination TransitGatewayAttachmentArn=<dev att ARN>,IpAddress=10.12.0.10 --include-return-path, then get-route-analysis.disable-transit-gateway-route-table-propagation of $A_SH from $RT_SPOKE → prod → shared fails; TGW metric PacketDropCountNoRoute rises; Route Analyzer shows no route. Re-enable to fix.$A_DEV in $RT_SPOKE. Does prod reach dev? Explain with the table dev is associated with.for A in $A_PROD $A_DEV $A_SH; do
aws ec2 \
delete-transit-gateway-vpc-attachment \
--transit-gateway-attachment-id $A
done
# wait until the attachments are deleted
for R in $RT_SPOKE $RT_SH; do
aws ec2 \
delete-transit-gateway-route-table \
--transit-gateway-route-table-id $R
done
aws networkmanager \
deregister-transit-gateway \
--region us-west-2 \
--global-network-id $GN \
--transit-gateway-arn $TGW_ARN
aws networkmanager delete-global-network \
--region us-west-2 --global-network-id $GN
aws ec2 delete-transit-gateway \
--transit-gateway-id $TGW
# then instances, EICEs and the three VPCs
# (keep TGW + prod VPC for Lab 4)Goal: two IPsec tunnels from the Lab 3 TGW to cgw (strongSwan + FRR), BGP over both. AWS side is .1 of each /30, cgw is .2.
CGW=$(aws ec2 create-customer-gateway --type ipsec.1 \
--bgp-asn 65010 --ip-address $CGW_EIP --output text \
--query CustomerGateway.CustomerGatewayId)
VPN=$(aws ec2 create-vpn-connection --type ipsec.1 \
--customer-gateway-id $CGW --transit-gateway-id $TGW \
--options '{"TunnelOptions":[
{"TunnelInsideCidr":"169.254.10.0/30","PreSharedKey":"netlabTunnelOne"},
{"TunnelInsideCidr":"169.254.11.0/30","PreSharedKey":"netlabTunnelTwo"}]}' \
--query VpnConnection.VpnConnectionId --output text)
aws ec2 describe-vpn-connections --vpn-connection-ids $VPN --query \
'VpnConnections[0].Options.TunnelOptions[].OutsideIpAddress'
aws ec2 wait vpn-connection-available --vpn-connection-ids $VPN
A_VPN=$(aws ec2 describe-transit-gateway-attachments --output text \
--filters Name=resource-id,Values=$VPN Name=state,Values=available \
--query 'TransitGatewayAttachments[0].TransitGatewayAttachmentId')
assoc $RT_SH $A_VPN; prop $RT_SPOKE $A_VPN # helpers from Lab 3
# on cgw, as root
apt-get install -y strongswan-swanctl charon-systemd frr
sysctl -w net.ipv4.ip_forward=1 net.ipv4.conf.all.rp_filter=0
for n in 1 2; do
ip link add xfrm$n type xfrm dev ens5 if_id $n
ip addr add 169.254.1$((n-1)).2/30 dev xfrm$n
ip link set xfrm$n up mtu 1422
done# FRR: set bgpd=yes in /etc/frr/daemons, restart frr, then:
vtysh <<'END'
conf t
router bgp 65010
bgp router-id 10.200.0.10
no bgp ebgp-requires-policy
neighbor 169.254.10.1 remote-as 64512
neighbor 169.254.11.1 remote-as 64512
address-family ipv4 unicast
network 10.200.0.0/24
maximum-paths 2
exit-address-family
end
write memory
END
# AWS side .1 = BGP neighbor (ASN 64512, the TGW Amazon-side ASN)
# cgw side .2 = local address on xfrm1 / xfrm2
# "network 10.200.0.0/24" works because the subnet is connected
# on ens5; maximum-paths 2 installs both tunnels for cgw → AWS
# (outbound ECMP); AWS → cgw ECMP needs VPN ECMP on the TGWNext slide: the strongSwan config that brings up the two tunnels. BGP comes up only after both IPsec tunnels are established.
One connection per tunnel; if_id binds each child SA to its xfrm interface, so BGP decides what enters which tunnel.
# /etc/swanctl/conf.d/aws.conf — tunnel 1
# (copy as t2: tunnel 2 outside IP and PSK, if_id 2)
connections {
t1 {
version = 2
local_addrs = 10.200.0.10
remote_addrs = <T1_OUTSIDE_IP>
proposals = aes256-sha256-modp2048
dpd_delay = 10s
local {
auth = psk
id = <CGW_EIP>
}
remote {
auth = psk
id = <T1_OUTSIDE_IP>
}
children {
t1 {
local_ts = 0.0.0.0/0
remote_ts = 0.0.0.0/0
esp_proposals = aes256-sha256-modp2048
if_id_in = 1
if_id_out = 1
start_action = start
dpd_action = restart
}
}
}
}secrets {
ike-t1 {
id-1 = <CGW_EIP>
id-2 = <T1_OUTSIDE_IP>
secret = "netlabTunnelOne"
}
}
# load and check
swanctl --load-all
swanctl --list-sas
# IKE_SA ESTABLISHED,
# CHILD_SA INSTALLED
ip -s link show xfrm1
# counters move with trafficlocal_addrs is the private IP; the IKE id is the Elastic IP, which AWS knows as the customer gateway.0.0.0.0/0 selectors + if_id = route-based VPN: BGP, not the IPsec policy, picks the tunnel.key = value per line: the format requires it.Prove BGP state on both sides, then steer and break the path. cgw SG: UDP 500/4500 from both tunnel IPs.
swanctl --list-sas: two ESTABLISHED IKE SAs.vtysh -c "show ip bgp summary": two neighbors Established; show ip bgp lists 10.11.0.0/16.describe-vpn-connections --query "VpnConnections[0].VgwTelemetry[].[OutsideIpAddress,Status,AcceptedRouteCount]" → UP, 1.search-transit-gateway-routes on $RT_SPOKE shows 10.200.0.0/24 via the VPN attachment.ping 10.200.0.10 works.route-map PREPEND permit 10 / set as-path prepend 65010 65010 / neighbor 169.254.10.1 route-map PREPEND out → AWS returns traffic over tunnel 2 (tcpdump -ni xfrm2, TunnelDataOut).swanctl --terminate --ike t2 → traffic moves to tunnel 1 after the BGP hold time.ip link set xfrm1 mtu 1500 → ESP packets exceed 1500 bytes on the internet path; large transfers stall or fragment while ping works (watch sizes with tcpdump -ni ens5 udp port 4500). Fix: MTU 1422 and iptables -t mangle -A FORWARD -o xfrm+ -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1382 (AWS table: AES-CBC + SHA2-256 with NAT-T; max 1446/1406 only with AES-GCM and no NAT-T).aws ec2 delete-vpn-connection \
--vpn-connection-id $VPN
aws ec2 delete-customer-gateway \
--customer-gateway-id $CGW
# a VPN bills per connection-hour:
# delete it the same day.
# keep onprem VPC + cgw for Lab 5,
# otherwise terminate cgw, release its
# EIP and delete the onprem VPC.
# Then run the Lab 3 cleanup.Goal: the VPC resolves corp.example from BIND on cgw, on-premises resolves aws.netlab. Needs Lab 4 and a 2nd prod subnet.
HZ=$(aws route53 create-hosted-zone --name aws.netlab \
--caller-reference $(date +%s) --hosted-zone-config PrivateZone=true \
--vpc VPCRegion=$AWS_REGION,VPCId=$VPC_PROD \
--query HostedZone.Id --output text)
aws route53 change-resource-record-sets --hosted-zone-id $HZ \
--change-batch '{"Changes":[{"Action":"UPSERT",
"ResourceRecordSet":{"Name":"db.aws.netlab","Type":"A","TTL":60,
"ResourceRecords":[{"Value":"10.11.0.10"}]}}]}'
ep() { aws route53resolver create-resolver-endpoint --name nl-$1 \
--direction $2 --creator-request-id $1-$(date +%s) \
--security-group-ids $SG_DNS --ip-addresses $3 $4 \
--query ResolverEndpoint.Id --output text; }
IN=$(ep in INBOUND SubnetId=$S1,Ip=10.11.0.20 SubnetId=$S2,Ip=10.11.1.20)
OUT=$(ep out OUTBOUND SubnetId=$S1 SubnetId=$S2)
RULE=$(aws route53resolver create-resolver-rule --name corp \
--rule-type FORWARD --domain-name corp.example \
--resolver-endpoint-id $OUT --target-ips Ip=10.200.0.10,Port=53 \
--creator-request-id r-$(date +%s) \
--query ResolverRule.Id --output text)
aws route53resolver associate-resolver-rule \
--resolver-rule-id $RULE --vpc-id $VPC_PROD# BIND on cgw (apt-get install -y bind9): named.conf.local
zone "corp.example" { type master; file "/etc/bind/db.corp"; };
zone "aws.netlab" { type forward; forward only;
forwarders { 10.11.0.20; 10.11.1.20; }; };
# named.conf.options: allow-query { 10.0.0.0/8; }; recursion yes;
# validate-except { "aws.netlab"; };
# /etc/bind/db.corp
$TTL 60
@ IN SOA ns.corp.example. admin.corp.example. 1 60 60 600 60
@ IN NS ns.corp.example.
ns IN A 10.200.0.10
app IN A 10.200.0.10
aws logs create-log-group --log-group-name /netlab/dns
QL=$(aws route53resolver create-resolver-query-log-config \
--name netlab --creator-request-id q-$(date +%s) \
--destination-arn \
arn:aws:logs:$AWS_REGION:$ACCT:log-group:/netlab/dns \
--query ResolverQueryLogConfig.Id --output text)
aws route53resolver associate-resolver-query-log-config \
--resolver-query-log-config-id $QL --resource-id $VPC_PROD
# DNS Firewall: create-firewall-domain-list, update-firewall-domains
# (ADD example.org), create-firewall-rule-group, create-firewall-rule
# (--action BLOCK --block-response NXDOMAIN),
# associate-firewall-rule-group (--vpc-id $VPC_PROD --priority 101)SG_DNS allows UDP and TCP 53 from 10.0.0.0/8. Endpoints need IPs in at least two subnets. Each create-* call needs a unique --creator-request-id.
Check the answer, the server that gave it, and the log line that proves the path.
dig +short app.corp.example → 10.200.0.10 (outbound endpoint → BIND).cgw: dig @10.200.0.10 +short db.aws.netlab → 10.11.0.10 (BIND → inbound endpoint → PHZ).dig example.org → NXDOMAIN (DNS Firewall)./netlab/dns: fields query_name, rcode, srcaddr | sort @timestamp desc.disassociate-resolver-rule → app.corp.example returns NXDOMAIN: the VPC asks public DNS.SG_DNS → on-premises queries time out (SERVFAIL from BIND).validate-except (Ubuntu ships dnssec-validation auto;) → db.aws.netlab returns SERVFAIL: netlab does not exist in the signed public tree.R53R="aws route53resolver"
$R53R disassociate-resolver-rule \
--resolver-rule-id $RULE \
--vpc-id $VPC_PROD
$R53R delete-resolver-rule \
--resolver-rule-id $RULE
for E in $IN $OUT; do
$R53R delete-resolver-endpoint \
--resolver-endpoint-id $E; done
$R53R \
disassociate-resolver-query-log-config \
--resolver-query-log-config-id $QL \
--resource-id $VPC_PROD
$R53R delete-resolver-query-log-config \
--resolver-query-log-config-id $QL
# DNS Firewall: disassociate the group,
# delete rule, group and domain list.
# Delete the A record (DELETE change):
aws route53 delete-hosted-zone --id $HZGoal: the same nginx targets behind each front door; compare what the target sees as the client and the time to first byte.
# VPC with public subnets in 2 AZs ($PUB1 $PUB2), nginx targets
# nginx: log_format x '$remote_addr xff="$http_x_forwarded_for"';
ALB=$(aws elbv2 create-load-balancer --name netlab-alb \
--type application --subnets $PUB1 $PUB2 \
--security-groups $SG_ALB \
--query 'LoadBalancers[0].LoadBalancerArn' --output text)
NLB=$(aws elbv2 create-load-balancer --name netlab-nlb6 \
--type network --subnets $PUB1 $PUB2 \
--query 'LoadBalancers[0].LoadBalancerArn' --output text)
# target groups: HTTP:80 (ALB) and TCP:80 (NLB), target-type
# instance; register-targets; create-listener on port 80 for each
# CloudFront: S3 origin with OAC + ALB origin for /api/*
aws cloudfront create-origin-access-control \
--origin-access-control-config '{"Name":"netlab-oac",
"SigningProtocol":"sigv4","SigningBehavior":"always",
"OriginAccessControlOriginType":"s3"}'
aws cloudfront create-distribution \
--distribution-config file://dist.json
# bucket policy: allow cloudfront.amazonaws.com s3:GetObject when
# AWS:SourceArn = the distribution ARN# Global Accelerator: the API lives in us-west-2
GA=$(aws globalaccelerator create-accelerator --region us-west-2 \
--name netlab --query Accelerator.AcceleratorArn --output text)
L=$(aws globalaccelerator create-listener --region us-west-2 \
--accelerator-arn $GA --protocol TCP \
--port-ranges FromPort=80,ToPort=80 \
--query Listener.ListenerArn --output text)
aws globalaccelerator create-endpoint-group --region us-west-2 \
--listener-arn $L --endpoint-group-region $AWS_REGION \
--endpoint-configurations \
EndpointId=$ALB,Weight=100,ClientIPPreservationEnabled=true
# compare from your laptop (run each 5 times)
for U in http://$NLB_DNS http://$ALB_DNS \
https://$CF_DNS/api/ http://$GA_DNS; do
curl -o /dev/null -s \
-w "$U connect=%{time_connect} ttfb=%{time_starttransfer}\n" $U
donedist.json: two origins (S3 with OriginAccessControlId and an empty OriginAccessIdentity; the ALB as an HTTP-only custom origin); default behavior uses managed CachingOptimized, /api/* uses CachingDisabled + AllViewer.
Read the nginx log on the target for each front door.
$remote_addr is your public IP (client IP preservation on instance targets).$remote_addr is an ALB private IP; X-Forwarded-For holds your IP.X-Forwarded-For lists your IP, then a CloudFront IP.X-Forwarded-For.x-cache: Hit from cloudfront.ttfb.SG_ALB allow only the prefix list com.amazonaws.global.cloudfront.origin-facing → direct ALB access times out, CloudFront works, GA stops: plan for that.describe-target-health.GAX="aws globalaccelerator --region us-west-2"
$GAX update-accelerator --no-enabled \
--accelerator-arn $GA
# when Status=DEPLOYED: delete endpoint
# group, listener, then the accelerator
# (fixed hourly fee: delete it today)
aws cloudfront get-distribution-config \
--id $CF > cf.json # note the ETag
# set Enabled=false; update-distribution
# --if-match <ETag>; wait until deployed
aws cloudfront delete-distribution \
--id $CF --if-match <new ETag>
aws cloudfront \
delete-origin-access-control \
--id $OAC --if-match <ETag>
for LB in $ALB $NLB; do
aws elbv2 delete-load-balancer \
--load-balancer-arn $LB; done
# target groups, instances, bucket, VPCGoal: spoke A (10.21.0.0/16, AZ 1 only) and B (10.22.0.0/16, AZ 2 only) reach the internet and each other via an inspection VPC.
NF="aws network-firewall"
RG=$($NF create-rule-group --type STATEFUL --capacity 100 \
--rule-group-name netlab-deny-domains --rule-group '{
"RulesSource":{"RulesSourceList":{"Targets":[".example.net"],
"TargetTypes":["TLS_SNI","HTTP_HOST"],
"GeneratedRulesType":"DENYLIST"}}}' \
--query RuleGroupResponse.RuleGroupArn --output text)
POL=$($NF create-firewall-policy --firewall-policy-name netlab-pol \
--firewall-policy '{
"StatelessDefaultActions":["aws:forward_to_sfe"],
"StatelessFragmentDefaultActions":["aws:forward_to_sfe"],
"StatefulRuleGroupReferences":[{"ResourceArn":"'$RG'"}]}' \
--query FirewallPolicyResponse.FirewallPolicyArn --output text)
$NF create-firewall --firewall-name netlab-fw \
--firewall-policy-arn $POL --vpc-id $VPC_INSP \
--subnet-mappings SubnetId=$FW_SUB1 SubnetId=$FW_SUB2
$NF describe-firewall --firewall-name netlab-fw --query \
'FirewallStatus.SyncStates.*.Attachment.[SubnetId,EndpointId]'
$NF update-logging-configuration --firewall-name netlab-fw \
--logging-configuration '{"LogDestinationConfigs":[{
"LogType":"ALERT","LogDestinationType":"CloudWatchLogs",
"LogDestination":{"logGroup":"/netlab/nfw-alert"}}]}'| Route table | Destination → target |
|---|---|
| Spoke A / B subnets | 0.0.0.0/0 → TGW |
| TGW RT spoke (A, B associated) | 0.0.0.0/0 → inspection attachment |
| TGW RT inspection | propagate A and B |
| Inspection TGW subnet, per AZ | 0.0.0.0/0 → FW endpoint, same AZ |
| Firewall subnet, per AZ | 0/0 → NAT same AZ; 10.0.0.0/8 → TGW |
| Public subnet, per AZ | 0/0 → IGW; 10.0.0.0/8 → FW endpoint |
| Inspection attachment | ApplianceModeSupport=enable |
Route to a firewall endpoint: create-route --destination-cidr-block 0.0.0.0/0 --vpc-endpoint-id $FWE1. Appliance mode: modify-transit-gateway-vpc-attachment --options ApplianceModeSupport=enable.
Prove that every flow crosses the firewall, then see what appliance mode is for.
curl -sI https://aws.amazon.com works; curl -sI https://www.example.net is reset or times out./netlab/nfw-alert shows the blocked SNI.curl http://<B IP> works (add FLOW logging to see it cross the firewall).ApplianceModeSupport=disable on the inspection attachment and repeat A → B. TGW keeps each direction in its source AZ: request via the AZ 1 endpoint, reply via AZ 2.# firewall endpoints bill per hour
NF="aws network-firewall"
$NF delete-firewall \
--firewall-name netlab-fw
# wait until the firewall is deleted
$NF delete-firewall-policy \
--firewall-policy-name netlab-pol
$NF delete-rule-group --type STATEFUL \
--rule-group-name netlab-deny-domains
aws logs delete-log-group \
--log-group-name /netlab/nfw-alert
# NAT gateways + EIPs, TGW attachments,
# route tables, TGW, instances, 3 VPCsGoal: fix each injected fault with evidence from a tool. Rebuild Lab 1 (lab1.sh), add nginx $WEB and a client $I.
# fault 1: default route gone
aws ec2 delete-route --route-table-id $RTPRV \
--destination-cidr-block 0.0.0.0/0
# fault 2: NACL drops return traffic (as in Lab 1)
# fault 3: the web SG no longer allows the client
aws ec2 revoke-security-group-ingress --group-id $SG_WEB \
--protocol tcp --port 80 --source-group $SG_APP
# fault 4: DHCP options point at a DNS server that does not exist
DOPT=$(aws ec2 create-dhcp-options --dhcp-configurations \
Key=domain-name-servers,Values=10.10.0.99 \
--query DhcpOptions.DhcpOptionsId --output text)
aws ec2 associate-dhcp-options --dhcp-options-id $DOPT \
--vpc-id $VPC # reboot instances to renew the lease
# fault 5: the client SG has no outbound rules
aws ec2 revoke-security-group-egress --group-id $SG_APP \
--ip-permissions 'IpProtocol=-1,IpRanges=[{CidrIp=0.0.0.0/0}]'
# Reachability Analyzer path: client ENI -> web ENI :80
aws ec2 create-network-insights-path --source $ENI_APP \
--destination $ENI_WEB --protocol tcp --destination-port 80# Traffic Mirroring: watch packets reach $WEB (Nitro types only)
TGT=$(aws ec2 create-traffic-mirror-target \
--network-interface-id $ENI_SNIFFER --output text \
--query TrafficMirrorTarget.TrafficMirrorTargetId)
F=$(aws ec2 create-traffic-mirror-filter --output text \
--query TrafficMirrorFilter.TrafficMirrorFilterId)
aws ec2 create-traffic-mirror-filter-rule \
--traffic-mirror-filter-id $F --traffic-direction ingress \
--rule-number 100 --rule-action accept --protocol 6 \
--destination-port-range FromPort=80,ToPort=80 \
--source-cidr-block 0.0.0.0/0 --destination-cidr-block 0.0.0.0/0
SES=$(aws ec2 create-traffic-mirror-session \
--network-interface-id $ENI_WEB --traffic-mirror-target-id $TGT \
--traffic-mirror-filter-id $F --session-number 1 --output text \
--query TrafficMirrorSession.TrafficMirrorSessionId)
# on the sniffer: mirrored packets arrive as VXLAN on UDP 4789
sudo tcpdump -ni ens5 -vv udp port 4789 | grep 'Flags \[S'The sniffer SG allows UDP 4789 from the web ENI. Make it realistic: a partner injects the faults in random order, and you do not read the injection script.
Write one line of evidence for each fault before you change anything.
| Fault | Symptom | Tool and evidence | Fix |
|---|---|---|---|
| 1 · No default route | Outbound IPv4 times out; no REJECT in Flow Logs | Reachability Analyzer ENI → IGW: no route at the route table | Re-add 0.0.0.0/0 → NAT |
| 2 · NACL ephemeral | Outbound connects hang | Flow Logs: inbound REJECT to ports 1024–65535 | Allow inbound TCP 1024–65535 |
| 3 · Web SG | Client → web times out | Reachability Analyzer names $SG_WEB; mirroring shows SYNs with no reply | Re-add the SG reference rule |
| 4 · DHCP DNS | "Could not resolve host"; IPs work | resolvectl status shows 10.10.0.99; dig @169.254.169.253 works | Re-associate the default options set; renew leases |
| 5 · Egress SG | Every outbound connection fails | Flow Logs: outbound REJECT from the client ENI | Restore the egress rule |
Clean up: delete the mirror session, filter and target (sessions bill per ENI-hour), the insights analyses and path; re-associate the default DHCP options and delete $DOPT; run the Lab 1 cleanup.
Goal: rebuild Lab 3 in Terraform, CIDRs from an IPAM pool (Chapter 9), and fail apply when a path regresses.
variable "ipam_pool_id" {}
data "aws_availability_zones" "az" { state = "available" }
resource "aws_vpc" "spoke" {
for_each = toset(["prod", "dev", "shared"])
ipv4_ipam_pool_id = var.ipam_pool_id
ipv4_netmask_length = 20
tags = { Name = each.key, Project = "netlab" }
}
resource "aws_subnet" "tgw" {
for_each = aws_vpc.spoke
vpc_id = each.value.id
availability_zone_id = data.aws_availability_zones.az.zone_ids[0]
cidr_block = cidrsubnet(each.value.cidr_block, 8, 0)
}
resource "aws_ec2_transit_gateway" "hub" {
default_route_table_association = "disable"
default_route_table_propagation = "disable"
}
resource "aws_ec2_transit_gateway_vpc_attachment" "a" {
for_each = aws_vpc.spoke
transit_gateway_id = aws_ec2_transit_gateway.hub.id
vpc_id = each.value.id
subnet_ids = [aws_subnet.tgw[each.key].id]
}
# + route tables, associations and propagations as in Lab 3
# (propagations: for_each resource named "p")resource "aws_ec2_network_insights_path" "prod_shared" {
source = aws_instance.t["prod"].primary_network_interface_id
destination = aws_instance.t["shared"].primary_network_interface_id
protocol = "tcp"
destination_port = 22
}
resource "terraform_data" "routing" { # changes when a propagation goes
input = [for p in aws_ec2_transit_gateway_route_table_propagation.p : p.id]
}
resource "aws_ec2_network_insights_analysis" "prod_shared" {
network_insights_path_id = aws_ec2_network_insights_path.prod_shared.id
wait_for_completion = true
lifecycle {
replace_triggered_by = [terraform_data.routing]
postcondition {
condition = self.path_found
error_message = "prod can no longer reach shared services"
}
}
}
# CI: plan on pull request, apply on merge
# terraform fmt -check && terraform validate
# terraform plan -out tfplan && terraform apply tfplan
# terraform apply -replace=aws_ec2_network_insights_analysis.prod_shared
# (re-checks intent after the routing change is live)The point of the lab: a bad change fails in apply, not in production.
terraform apply completes; the analysis reports the expected result. Add a prod → dev path asserting !self.path_found to encode isolation.aws ec2 describe-vpcs --filters Name=tag:Project,Values=netlab --query "Vpcs[].CidrBlock": three non-overlapping /20s from the pool.terraform plan right after apply shows no changes.replace_triggered_by re-runs the analysis, but Terraform may run it before the propagation is deleted, so run the CI re-check terraform apply -replace=aws_ec2_network_insights_analysis.prod_shared → the postcondition fails with "prod can no longer reach shared services".ipv4_netmask_length to 21 → plan says the VPCs must be replaced (terraform show -json tfplan | jq on resource_changes lists the deletes). Do not apply.terraform plan -refresh-only reports it.terraform destroy (analyses and paths go first automatically).aws ec2 get-ipam-pool-allocations --ipam-pool-id $POOL.Goal: watch NAT processing vanish behind a gateway endpoint, find it in Cost Explorer, and measure the single-flow limit.
# Part A: Lab 1 VPC without the S3 endpoint
aws ce update-cost-allocation-tags-status \
--cost-allocation-tags-status TagKey=Project,Status=Active
# on the private instance, three times:
aws s3 cp s3://netlab-$ACCT/1g.bin /dev/null
# NAT bytes over the last hour (GNU date: Linux or CloudShell)
aws cloudwatch get-metric-statistics --namespace AWS/NATGateway \
--metric-name BytesInFromDestination --statistics Sum \
--dimensions Name=NatGatewayId,Value=$NAT --period 300 \
--start-time $(date -u -d '-1 hour' +%FT%TZ) \
--end-time $(date -u +%FT%TZ)
# add the S3 gateway endpoint (Lab 2), repeat: NAT bytes stay flat
# next day; every Cost Explorer API call is billed
aws ce get-cost-and-usage --granularity DAILY \
--time-period Start=2026-10-01,End=2026-10-31 \
--metrics UnblendedCost UsageQuantity \
--filter '{"Tags":{"Key":"Project","Values":["netlab"]}}' \
--group-by Type=DIMENSION,Key=USAGE_TYPE# Part B: two c6in.8xlarge in the same AZ, no placement group
# (expensive: finish within 30 minutes)
# on B: iperf3 -s
# on A:
iperf3 -c $B_IP -t 20 -P 1 # one flow: about 5 Gbps
iperf3 -c $B_IP -t 20 -P 8 # eight flows: near the instance limit
# enable ENA Express on BOTH primary ENIs
for E in $ENI_A $ENI_B; do
aws ec2 modify-network-interface-attribute \
--network-interface-id $E \
--ena-srd-specification 'EnaSrdEnabled=true'
done
iperf3 -c $B_IP -t 20 -P 1 # one flow again: higher
ethtool -S ens5 | grep ena_srd # SRD counters increaseWrite each measured value next to the documented limit.
BytesInFromDestination grows by about 3 GB without the endpoint and stays flat with it.APN1-NatGateway-Bytes and APN1-NatGateway-Hours for the tag.ena_srd counters rise.aws s3 cp fails with AccessDenied while NAT stays idle: the endpoint route still wins.Network resources bill per hour whether or not traffic flows. A forgotten lab can cost more than the exam.
aws resourcegroupstaggingapi get-resources --tag-filters Key=Project,Values=netlab.describe-addresses, describe-nat-gateways, describe-vpc-endpoints, describe-transit-gateway-attachments.Short anchors; explain each in your own words.
| Term | Short meaning |
|---|---|
| Longest prefix match | The most specific matching route wins |
| Propagation | Routes learned automatically from an attachment or VGW |
| Association | Which route table a subnet or TGW attachment uses |
| Appliance mode | TGW keeps both directions of a flow in one AZ |
| ECMP | Equal-cost paths used in parallel (TGW VPN, Connect) |
| ASN | Autonomous system number used by BGP |
| AS_PATH prepend | Make a path look longer so peers prefer another |
| Term | Short meaning |
|---|---|
| Local preference | BGP attribute: higher wins, set by communities on DX |
| MED | BGP hint to a neighbor about the preferred entry point |
| VIF | Direct Connect virtual interface: private, public or transit |
| LOA-CFA | Letter authorizing the cross connect at a DX location |
| LAG | Bundle of DX connections acting as one |
| MACsec | Layer 2 encryption on dedicated DX connections |
| Transitive routing | Traffic passing through a middle network; peering and VGW do not allow it |
Short anchors; explain each in your own words.
| Term | Short meaning |
|---|---|
| ENI | Virtual network interface: IPs, SGs, MAC |
| PrivateLink | Private, one-way access to a service through interface endpoints |
| GENEVE | Encapsulation GWLB uses to send traffic to appliances (UDP 6081) |
| Prefix list | Named set of CIDRs referenced by SGs and routes |
| IPAM pool | Address space from which VPC CIDRs are allocated |
| NAU | Network Address Usage: VPC size metric and quota |
| MSS clamping | Rewriting TCP MSS so segments fit a smaller MTU |
| Term | Short meaning |
|---|---|
| PHZ | Private hosted zone, answered only inside associated VPCs |
| Inbound endpoint | Resolver IPs that on-premises DNS forwards to |
| Outbound endpoint | Resolver IPs that forward VPC queries to other DNS servers |
| Alias record | Route 53 record pointing to an AWS resource, allowed at the apex |
| DNSSEC | Signed DNS answers that resolvers can validate |
| Reachability Analyzer | Configuration-based path analysis between two endpoints |
| Traffic Mirroring | Copy of ENI packets sent to a target over VXLAN |
The exam tests current AWS behavior; when a slide and the docs disagree, the docs win.
docs.aws.amazon.com/aws-certification/latest/advanced-networking-specialty-01/aws.github.io/aws-networking-best-practices/docs.aws.amazon.com/whitepapers/docs.aws.amazon.com/whitepapers/latest/hybrid-connectivity/aws.amazon.com/blogs/networking-and-content-delivery/docs.aws.amazon.com/vpc/latest/userguide/amazon-vpc-limits.htmldocs.aws.amazon.com/vpc/latest/tgw/transit-gateway-quotas.htmldocs.aws.amazon.com/directconnect/latest/UserGuide/docs.aws.amazon.com/vpn/latest/s2svpn/docs.aws.amazon.com/AWSEC2/latest/UserGuide/network_mtu.htmldocs.aws.amazon.com/Route53/latest/DeveloperGuide/DNSLimitations.html