Architecture, compute, networking, identity, security, storage, scaling, observability, upgrades, cost — and the best practices for each.
Use ← → / Space to navigate · I contents · O overview · T theme · F fullscreen
Click any line to jump there · press I to return to this page
What EKS adds to Kubernetes, how it is built, and how to create a cluster.
Kubernetes in one slide, and what EKS adds on top.
Kubernetes is a declarative control loop: you describe desired state, controllers reconcile it.
Pod — smallest unit, 1+ containersDeployment → ReplicaSet — statelessStatefulSet — stable identity + volumesDaemonSet — one per nodeJob / CronJob — batchService — ClusterIP / NodePort / LoadBalancerIngress / Gateway API — L7 routingNetworkPolicy — pod firewallsvc.ns.svc.cluster.localConfigMap, SecretPersistentVolumeClaim, StorageClassServiceAccount, Role, RoleBindingNamespace, ResourceQuota, LimitRangeUpstream, CNCF-conformant Kubernetes with a managed control plane and deep AWS integration.
Pricing basics: $0.10 per cluster-hour in standard support, $0.60 per cluster-hour in extended support — plus the compute, storage, and data transfer you use. Auto Mode adds a per-instance management fee; Provisioned Control Plane tiers (for very large clusters) and EKS Capabilities are billed on top.
kubectl apply to a running podKnow this path by heart — almost every troubleshooting session maps to one of these steps.
aws eks get-tokenPending → Karpenter / Auto Mode launches a nodeReadyThe more managed the compute option, the more AWS takes on.
| Area | Self-managed nodes | Managed node groups | Fargate | Auto Mode |
|---|---|---|---|---|
| Control plane, etcd | AWS | AWS | AWS | AWS |
| Node AMI / OS patching | You | You trigger, AWS provides AMI | AWS | AWS (nodes recycled, max 21-day lifetime) |
| Node provisioning & scaling | You | You (ASG + autoscaler) | AWS (per pod) | AWS (Karpenter-based) |
| Core add-ons (CNI, CoreDNS, kube-proxy, EBS CSI, LB) | You | You | Partially AWS | AWS (built in) |
| Version upgrades | You | You | You (control plane) | You (control plane); nodes follow |
| IAM, RBAC, network policy, app security, data | You | You | You | You |
Clusters are cattle too — always build them from code.
terraform-aws-modules/eks — most common in productionaws-auth ConfigMapmodule "eks" {
source = "terraform-aws-modules/eks/aws"
version = "~> 21.0"
name = "prod-apne1"
kubernetes_version = "1.36"
vpc_id = module.vpc.vpc_id
subnet_ids = module.vpc.private_subnets
endpoint_public_access = false
authentication_mode = "API"
enable_cluster_creator_admin_permissions = false
enabled_log_types = ["api", "audit", "authenticator"]
addons = {
vpc-cni = { before_compute = true }
coredns = {}
kube-proxy = {}
eks-pod-identity-agent = {}
}
}
14 months from the version's EKS release. $0.10/cluster-hour.
Additional 12 months. $0.60/cluster-hour. Enabled by default — set the upgrade policy to STANDARD to opt out.
After extended support ends, EKS auto-upgrades the control plane to the oldest supported version.
# 0. Tools: AWS CLI v2, kubectl, eksctl, helm
aws sts get-caller-identity
# 1. Create a cluster with Auto Mode (~15 min)
eksctl create cluster --name lab \
--region ap-northeast-1 --enable-auto-mode
# 2. Connect and deploy something
aws eks update-kubeconfig --name lab --region ap-northeast-1
kubectl get nodes # empty until pods need capacity
kubectl create deployment web \
--image=public.ecr.aws/nginx/nginx --replicas=2
kubectl get pods -o wide -w # watch a node appear
kubectl get nodeclaims
# 3. Expose it
kubectl expose deployment web --port=80 --type=ClusterIP
kubectl port-forward svc/web 8080:80
# 4. ALWAYS clean up — clusters bill every hour
eksctl delete cluster --name lab --region ap-northeast-1
Where pods run and how they get IP addresses and traffic.
| Option | What it is | Use when | Watch out |
|---|---|---|---|
| EKS Auto Mode | AWS manages nodes (Karpenter-based), CNI, CoreDNS, kube-proxy, EBS CSI, LB controller; Bottlerocket-based, immutable nodes | New clusters, small platform teams, want least ops | Per-instance fee; less node-level customization (no SSH/SSM) |
| Karpenter (self-run) | Open-source node autoscaler; provisions right-sized EC2 directly | Diverse workloads, Spot, cost focus, full control | You run and upgrade Karpenter itself (on MNG or Fargate) |
| Managed node groups | EKS-managed ASGs with EKS-optimized AMIs, managed rolling updates | Stable, homogeneous capacity; system/add-on pools | Scaling via Cluster Autoscaler is slower and less bin-packing |
| Self-managed nodes | Your own ASG + launch template | Custom OS / AMI requirements | Everything is on you |
| Fargate | Serverless pod — one microVM per pod | Isolation, bursty or low-ops jobs | No DaemonSets, no GPUs, no EBS volumes, no privileged pods; slower start |
| Hybrid Nodes | On-prem/edge machines joined to an EKS control plane | Data residency, existing hardware | Needs private connectivity (VPN / Direct Connect) |
1 unless pods truly need IMDSkube-reserved/system-reserved (EKS AMIs set sane defaults)general-purpose and systemNodePool + NodeClass (eks.amazonaws.com) for GPU, Spot, or isolated poolsStorageClass (ebs.csi.eks.amazonaws.com) and IngressClassVPC CNI, IP planning, load balancing, DNS.
aws-node DaemonSet attaches ENIs and secondary IPs to each nodeWARM_ENI_TARGET, WARM_IP_TARGET)ENIConfigkubectl set env daemonset aws-node -n kube-system ENABLE_PREFIX_DELEGATION=true WARM_PREFIX_TARGET=1
# Prefer configuring this through the managed add-on configuration values instead of kubectl in production
ENIs × (IPs per ENI − 1) + 2ClusterIP: virtual IP; kube-proxy (iptables/IPVS/nftables) DNATs to pod IPsNodePort: port on every nodeLoadBalancer: NLB via LB Controller; IP mode targets pods directlyexternalTrafficPolicy: Local keeps the client IP and avoids an extra hopkubernetes.io/role/elb and kubernetes.io/role/internal-elbIngress → ALB (L7, path/host routing, WAF, Cognito/OIDC)Service type: LoadBalancer → NLB (L4, static IPs, PrivateLink)IngressGroup shares one ALB across many Ingressestarget-type: ip — traffic goes straight to podspreStop sleep + tune deregistration delay to avoid 502s on rolloutapiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: web
annotations:
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/target-type: ip
alb.ingress.kubernetes.io/group.name: public
alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:...
spec:
ingressClassName: alb
rules:
- host: app.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service: { name: web, port: { number: 80 } }
ndots or use FQDNs with trailing dot for heavy external lookupsWho can reach the cluster, what pods may call in AWS, and how to harden it.
Humans → cluster, and pods → AWS.
aws-authaws eks get-tokenCONFIG_MAP → API_AND_CONFIG_MAP → API (one-way migration)AmazonEKSClusterAdminPolicy (cluster scope), AmazonEKSAdminPolicy, AmazonEKSEditPolicy, AmazonEKSViewPolicyAPI mode; manage entries in IaCaws eks create-access-entry \
--cluster-name prod \
--principal-arn arn:aws:iam::111122223333:role/TeamA
POLICY=arn:aws:eks::aws:cluster-access-policy
aws eks associate-access-policy \
--cluster-name prod \
--principal-arn arn:aws:iam::111122223333:role/TeamA \
--policy-arn $POLICY/AmazonEKSEditPolicy \
--access-scope \
type=namespace,namespaces=team-a
# Or bind to RBAC groups instead of access policies:
aws eks create-access-entry ... \
--kubernetes-groups team-a-devs
| IRSA (IAM Roles for Service Accounts) | EKS Pod Identity | |
|---|---|---|
| Mechanism | OIDC provider + projected SA token → AssumeRoleWithWebIdentity | eks-pod-identity-agent DaemonSet + association via EKS API |
| Trust policy | Per cluster OIDC issuer, per SA — must be edited for each cluster | One trust to pods.eks.amazonaws.com — reusable across clusters |
| Setup | Create OIDC provider, annotate SA with role ARN | aws eks create-pod-identity-association |
| Session tags / ABAC | No | Yes (cluster, namespace, SA tags) |
| Works on | EC2, Fargate, any cluster incl. self-managed K8s | Linux EC2 nodes incl. Auto Mode (not Fargate, not Windows) |
Default to Pod Identity for new EKS workloads; one role per app; never use node instance role permissions for apps.
Set IMDSv2 hop limit to 1 so pods can't steal the node role credentials.
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata: { name: deployer, namespace: team-a }
rules:
- apiGroups: ["apps"]
resources: ["deployments"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
- apiGroups: [""]
resources: ["pods", "pods/log", "services", "configmaps"]
verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata: { name: team-a-deployers, namespace: team-a }
subjects:
- kind: Group
name: team-a-devs # from access entry --kubernetes-groups
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: Role
name: deployer
apiGroup: rbac.authorization.k8s.io
Role = namespaced; ClusterRole = cluster-wide or reusableUser, Group, ServiceAccountcluster-admin for humans* verbs/resourcessecrets get/list, pods/exec, escalate, bind, impersonate as privilegedkubectl auth can-i --list --as=...# 1. IAM role trusted by EKS Pod Identity
cat > trust.json <<'EOF'
{ "Version": "2012-10-17",
"Statement": [{ "Effect": "Allow",
"Principal": { "Service": "pods.eks.amazonaws.com" },
"Action": ["sts:AssumeRole", "sts:TagSession"] }] }
EOF
aws iam create-role --role-name orders-app \
--assume-role-policy-document file://trust.json
aws iam put-role-policy --role-name orders-app \
--policy-name s3-read --policy-document file://s3-read.json
# 2. Agent (built into Auto Mode)
aws eks create-addon --cluster-name prod \
--addon-name eks-pod-identity-agent
# 3. Bind role ↔ namespace + service account
aws eks create-pod-identity-association \
--cluster-name prod --namespace orders \
--service-account orders \
--role-arn arn:aws:iam::111122223333:role/orders-app
# 4. Verify from inside the pod
kubectl exec -n orders deploy/orders -- \
aws sts get-caller-identity
eks-cluster-name, kubernetes-namespace, kubernetes-service-accounthostPath pod → escapes to the node, then to other podsrestricted by default, baseline where neededkubectl label ns payments pod-security.kubernetes.io/enforce=restricted pod-security.kubernetes.io/warn=restricted
spec:
serviceAccountName: orders
automountServiceAccountToken: false
securityContext:
runAsNonRoot: true
runAsUser: 10001
fsGroup: 10001
seccompProfile: { type: RuntimeDefault }
containers:
- name: app
image: <ecr-repo>/orders@sha256:<digest>
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities: { drop: ["ALL"] }
resources:
requests: { cpu: 250m, memory: 256Mi }
limits: { memory: 256Mi }
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: { name: default-deny, namespace: orders }
spec:
podSelector: {}
policyTypes: [Ingress, Egress]
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: { name: allow-dns, namespace: orders }
spec:
podSelector: {}
policyTypes: [Egress]
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
ports:
- { protocol: UDP, port: 53 }
- { protocol: TCP, port: 53 }
Set automountServiceAccountToken: true only for pods that call the Kubernetes API (IRSA / Pod Identity inject their own token volumes). With default-deny egress, also allow the Pod Identity agent at 169.254.170.23.
| Level | What it allows / blocks | Use for |
|---|---|---|
| privileged | No restrictions | System namespaces only (CNI, CSI, monitoring agents) |
| baseline | Blocks privileged containers, hostNetwork/hostPID/hostIPC, hostPath, hostPorts, dangerous capabilities | Legacy apps during migration |
| restricted | Baseline + must run as non-root, drop ALL capabilities, no privilege escalation, seccomp RuntimeDefault, limited volume types | Default for all application namespaces |
apiVersion: v1
kind: Namespace
metadata:
name: orders
labels:
pod-security.kubernetes.io/enforce: restricted
pod-security.kubernetes.io/enforce-version: latest
pod-security.kubernetes.io/warn: restricted
pod-security.kubernetes.io/audit: restricted
enforce rejects pods · warn shows client warnings · audit annotates audit logswarn + audit first, fix workloads, then enforceapiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata: { name: disallow-latest-tag }
spec:
background: true
rules:
- name: require-pinned-image
match:
any:
- resources: { kinds: [Pod] }
validate:
failureAction: Enforce
message: "Pin an image tag or digest, not :latest"
pattern:
spec:
containers:
- image: "!*:latest"
:latest; signed images (verifyImages)hostPath, no LoadBalancer without approvalAudit, read policy reports, then Enforce| Approach | How | Trade-off |
|---|---|---|
| Kubernetes Secret | Base64 in etcd, envelope-encrypted with KMS | Simple; anyone with get secrets in the namespace can read it |
| External Secrets Operator | Syncs Secrets Manager / Parameter Store into K8s Secrets | Source of truth in AWS, rotation via refresh; still a K8s Secret |
| Secrets Store CSI + AWS provider | Mounts secrets as files at pod start | No K8s Secret unless you enable sync |
| SDK call with Pod Identity | App reads Secrets Manager directly | Nothing in the cluster; app code depends on AWS |
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata: { name: db, namespace: orders }
spec:
refreshInterval: 1h
secretStoreRef: { kind: ClusterSecretStore, name: aws-sm }
target: { name: db-credentials }
data:
- secretKey: password
remoteRef: { key: prod/orders/db, property: password }
secrets; mount as files rather than env varsorders pods reach RDS (needs ENABLE_POD_ENI=true on the VPC CNI)# Turn on native network policy in the VPC CNI
aws eks update-addon --cluster-name prod \
--addon-name vpc-cni \
--configuration-values '{"enableNetworkPolicy":"true"}'
apiVersion: vpcresources.k8s.aws/v1beta1
kind: SecurityGroupPolicy
metadata: { name: orders-db, namespace: orders }
spec:
podSelector: { matchLabels: { app: orders } }
securityGroups:
groupIds: [sg-0123456789abcdef0]
USERInspector re-scans images continuously when new CVEs appear; Renovate/Dependabot bump dependencies; rebuild images regularly, not only on code changes.
CI authenticates to AWS with OIDC (no static keys), can push only to its own repos, and cannot touch the cluster directly when GitOps does the deploy.
expireAfter, AMI updates)AmazonEKSWorkerNodePolicyAmazonEC2ContainerRegistryPullOnlyAmazonEKS_CNI_Policy to the aws-node service account (Pod Identity / IRSA)AmazonSSMManagedInstanceCore only if you use SSM# Logs Insights on /aws/eks/<cluster>/cluster
# Who deleted what?
fields @timestamp, user.username, objectRef.resource,
objectRef.namespace, objectRef.name
| filter @logStream like /kube-apiserver-audit/
| filter verb = "delete"
| sort @timestamp desc | limit 50
# Who reads secrets?
fields @timestamp, user.username, objectRef.name
| filter @logStream like /kube-apiserver-audit/
| filter objectRef.resource = "secrets"
and verb in ["get", "list"]
| stats count(*) by user.username
# RBAC denials
fields user.username, verb, objectRef.resource
| filter @logStream like /kube-apiserver-audit/
| filter responseStatus.code = 403
| stats count(*) as denials by user.username, verb
| sort denials desc
describe output# Detach from Service & ReplicaSet (a clean replica starts)
kubectl label pod web-7d9f -n orders app=quarantine --overwrite
# Cut all traffic to/from it
kubectl apply -f - <<'EOF'
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: { name: quarantine, namespace: orders }
spec:
podSelector: { matchLabels: { app: quarantine } }
policyTypes: [Ingress, Egress]
EOF
# Stop new pods landing on the node
kubectl cordon ip-10-0-12-34.ap-northeast-1.compute.internal
# Preserve evidence
aws ec2 create-snapshot --volume-id vol-0abc123 \
--description "IR case 123 - node root volume"
Persistent data, capacity that follows demand, and the platform add-ons around them.
| Driver | Access | Best for | Notes |
|---|---|---|---|
| EBS CSI | RWO, single AZ | Databases, StatefulSets | Use gp3; volume is AZ-bound → use WaitForFirstConsumer |
| EFS CSI | RWX, multi-AZ | Shared files, CMS, ML datasets | Dynamic provisioning via access points; Fargate supports static provisioning only |
| FSx (Lustre/ONTAP/OpenZFS) | RWX | HPC/ML throughput, enterprise NAS | Higher performance, higher cost |
| Mountpoint for S3 CSI | RWX (read-heavy) | Large datasets, ML training | Not POSIX-complete; sequential writes only |
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata: { name: gp3 }
provisioner: ebs.csi.aws.com
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
parameters: { type: gp3, encrypted: "true" }
Pods scale first, nodes follow.
Scales replicas on CPU/memory or custom/external metrics. Needs metrics-server.
Event-driven scaling (SQS, Kafka, Prometheus, cron), incl. scale-to-zero.
Recommends/sets requests. Use in recommendation mode alongside HPA on CPU.
Karpenter (or Auto Mode) provisions nodes for pending pods; Cluster Autoscaler scales ASGs.
| Cluster Autoscaler | Karpenter | |
|---|---|---|
| Unit | Node groups (ASGs) with fixed instance types | Individual instances, chosen per pending pod batch |
| Speed | Minutes (ASG round-trip) | Faster — calls EC2 Fleet directly |
| Cost features | Limited | Consolidation, Spot/On-Demand mix, Graviton, drift replacement |
apiVersion: karpenter.sh/v1
kind: NodePool
metadata: { name: default }
spec:
template:
spec:
nodeClassRef: { group: karpenter.k8s.aws,
kind: EC2NodeClass, name: default }
requirements:
- key: karpenter.sh/capacity-type
operator: In
values: ["spot", "on-demand"]
- key: kubernetes.io/arch
operator: In
values: ["arm64", "amd64"]
- key: karpenter.k8s.aws/instance-category
operator: In
values: ["c", "m", "r"]
expireAfter: 720h
limits: { cpu: "1000" }
disruption:
consolidationPolicy: WhenEmptyOrUnderutilized
consolidateAfter: 1m
budgets: [{ nodes: "10%" }]
limits to cap runaway spendbudgets + PDBs to control churnexpireAfter forces regular node refresh (patching)The platform components every cluster needs.
vpc-cni, coredns, kube-proxyeks-pod-identity-agent, aws-ebs-csi-driver, aws-efs-csi-driver, aws-mountpoint-s3-csi-driveramazon-cloudwatch-observability, adot, aws-guardduty-agentmetrics-server) & Marketplace add-onsaws eks describe-addon-versionsPrefer managed add-ons where available · pin versions in IaC · upgrade add-ons with every cluster upgrade · avoid manual kubectl edit on managed add-ons (use configuration values) · give each add-on its own Pod Identity role.
See what the cluster is doing and keep workloads available.
Control plane, nodes, workloads, logs, traces, alerts, SLOs.
api, audit, authenticator, controllerManager, scheduler → CloudWatch LogsPending pods & unschedulable reasons · node NotReady · pod restarts/OOMKilled · API server 5xx & latency · CoreDNS errors · free IPs per subnet · PVC usage · certificate expiry · SLO-based app alerts.
| Layer | Key signals | Source |
|---|---|---|
| Control plane | API latency & 5xx, request rate, throttling (APF), etcd size, webhook latency, pending pods | CloudWatch AWS/EKS, API /metrics |
| Nodes | CPU, memory, disk, network, NotReady, pressure conditions, pods per node | node-exporter, Container Insights |
| K8s objects | Available vs desired replicas, restarts, OOMKills, failed Jobs, HPA at max, PVC usage | kube-state-metrics, kubelet |
| Applications | RED: rate, errors, duration; saturation; business KPIs | App metrics, OpenTelemetry, ALB metrics |
| Add-ons | CoreDNS errors & latency, CNI IP allocation, Karpenter launch errors, LB controller errors | Add-on /metrics endpoints |
| Network & cost | Free IPs per subnet, NAT bytes, cross-AZ traffic, requests vs usage | CloudWatch, VPC Flow Logs, Kubecost |
Methods: USE (utilization, saturation, errors) for resources · RED (rate, errors, duration) for services · Golden signals (latency, traffic, errors, saturation) for user-facing SLOs.
AWS/EKS metrics (new clusters on K8s 1.28+, no extra cost): API request totals & 5xx, storage size, scheduler attempts, pending pods/aws/eks/<cluster>/clusterapiserver_request_duration_seconds p99 risingapiserver_flowcontrol_rejected_requests_total > 0 (throttling)etcd_mvcc_db_total_size_in_use_in_bytes near the 8 GiB etcd limit# API server metrics
kubectl get --raw /metrics | grep apiserver_request_total
# Scheduler metrics (EKS-proxied endpoint)
kubectl get --raw \
"/apis/metrics.eks.amazonaws.com/v1/ksh/container/metrics"
# Controller manager metrics
kubectl get --raw \
"/apis/metrics.eks.amazonaws.com/v1/kcm/container/metrics"
# Enable control plane logs
aws eks update-cluster-config --name prod \
--logging '{"clusterLogging":[{"types":
["api","audit","authenticator",
"controllerManager","scheduler"],
"enabled":true}]}'
# Audit logs can be large: set retention
aws logs put-retention-policy \
--log-group-name /aws/eks/prod/cluster \
--retention-in-days 90
# IAM for the CloudWatch agent via Pod Identity
# (role with CloudWatchAgentServerPolicy)
aws eks create-pod-identity-association \
--cluster-name prod \
--namespace amazon-cloudwatch \
--service-account cloudwatch-agent \
--role-arn arn:aws:iam::111122223333:role/cw-agent
# Install the add-on: CloudWatch agent + Fluent Bit
aws eks create-addon --cluster-name prod \
--addon-name amazon-cloudwatch-observability
# Log groups created
/aws/containerinsights/prod/application
/aws/containerinsights/prod/host
/aws/containerinsights/prod/dataplane
/aws/containerinsights/prod/performance
remote_writekube-prometheus-stack: Prometheus Operator, Alertmanager, Grafana, node-exporter, kube-state-metrics# Tell Prometheus Operator to scrape your app
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: orders
namespace: orders
# must match Prometheus serviceMonitorSelector
labels: { release: kube-prometheus-stack }
spec:
selector:
matchLabels: { app: orders }
endpoints:
- port: metrics # named Service port
path: /metrics
interval: 30s
| Alert | Expression |
|---|---|
| Pod crash-looping | increase(kube_pod_container_status_restarts_total[15m]) > 3 |
Pods pending (with for: 10m) | sum by (namespace) (kube_pod_status_phase{phase="Pending"}) > 0 |
| Node not ready | kube_node_status_condition{condition="Ready",status="true"} == 0 |
| Deployment degraded | kube_deployment_status_replicas_available < kube_deployment_spec_replicas |
| OOMKilled | kube_pod_container_status_last_terminated_reason{reason="OOMKilled"} == 1 |
| CPU throttling | rate(container_cpu_cfs_throttled_periods_total[5m]) / rate(container_cpu_cfs_periods_total[5m]) > 0.25 |
| PVC almost full | kubelet_volume_stats_used_bytes / kubelet_volume_stats_capacity_bytes > 0.85 |
| API server 5xx | sum(rate(apiserver_request_total{code=~"5.."}[5m])) / sum(rate(apiserver_request_total[5m])) > 0.01 |
| HPA maxed out | kube_horizontalpodautoscaler_status_current_replicas == kube_horizontalpodautoscaler_spec_max_replicas |
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata: { name: workloads, namespace: monitoring }
spec:
groups:
- name: workloads
rules:
- alert: PodCrashLooping
expr: |
increase(kube_pod_container_status_restarts_total[15m]) > 3
for: 5m
labels:
severity: warning
team: "{{ $labels.namespace }}"
annotations:
summary: "{{ $labels.pod }} is restarting"
runbook_url: https://wiki.example.com/runbooks/crashloop
for: to avoid flappingWrites structured JSON to stdout/stderr
containerd writes to /var/log/containers
DaemonSet tails files, adds pod / namespace / labels metadata
CloudWatch Logs, OpenSearch, or S3 via Firehose
aws-observability namespace ConfigMap# Logs Insights: top error sources in the last hour
fields @timestamp, kubernetes.namespace_name,
kubernetes.pod_name, log
| filter log like /(?i)error|exception/
| stats count(*) as errors
by kubernetes.namespace_name,
kubernetes.container_name
| sort errors desc
| limit 20
traceparent across all servicesapiVersion: opentelemetry.io/v1alpha1
kind: Instrumentation
metadata: { name: default, namespace: orders }
spec:
exporter:
endpoint: http://otel-collector.observability:4318 # OTLP/HTTP
propagators: [tracecontext, baggage]
sampler:
type: parentbased_traceidratio
argument: "0.1"
---
# Opt a workload in (pod template annotation)
metadata:
annotations:
instrumentation.opentelemetry.io/inject-java: "true"
Fleet overview → cluster health → namespace / team → workload detail → pod. Each level links to the next, and every alert links to the dashboard that explains it.
topologySpreadConstraints across zones and nodesPodDisruptionBudget for every multi-replica servicePriorityClass for critical workloadsSIGTERM; preStop sleep for LB deregistrationterminationGracePeriodSecondsmaxUnavailable: 0, maxSurge: 25%topologySpreadConstraints:
- { maxSkew: 1, topologyKey: topology.kubernetes.io/zone, whenUnsatisfiable: ScheduleAnyway,
labelSelector: { matchLabels: { app: web } } }
Upgrades, cost control, and running EKS over time.
pluto, kubent)kubelet may be up to 3 minor versions older than the API server (K8s 1.28+), but never newer.
For big jumps or risky changes: build a new cluster, shift traffic via DNS/ALB weights, then retire the old one. Requires stateless clusters + GitOps.
ResourceQuota & LimitRange per namespacerestrictedCluster strategy: separate prod and non-prod clusters (ideally separate AWS accounts); prefer fewer, larger shared clusters over many small ones unless isolation or blast radius requires it.
| Symptom | Likely cause | Check |
|---|---|---|
Pods Pending | No capacity, taints, affinity, PVC AZ mismatch | kubectl describe pod events; Karpenter logs |
Pods stuck ContainerCreating | Subnet out of IPs, CNI errors, volume attach | Subnet free IPs; aws-node logs; ipamd |
ImagePullBackOff | Wrong tag, no ECR perms, no NAT/endpoints | Node role ECR policy; VPC endpoints |
CrashLoopBackOff / OOMKilled | App error, low memory limit, bad probes | kubectl logs --previous; describe |
Node NotReady | kubelet/CNI failure, disk/memory pressure | Node conditions; SSM to node; log collector script |
Unauthorized / Forbidden | Missing access entry or RBAC | aws sts get-caller-identity; kubectl auth can-i |
| ALB 502/504 on deploy | No preStop delay, readiness, SG rules | Target health; LB controller logs |
| DNS timeouts | CoreDNS overload, VPC resolver limits | CoreDNS metrics; NodeLocal DNSCache |
# What just happened?
kubectl get events -A --sort-by=.lastTimestamp | tail -30
kubectl describe pod <pod> -n <ns>
kubectl logs <pod> -n <ns> -c <container> --previous
# Resources
kubectl top nodes
kubectl top pods -A --sort-by=memory
kubectl describe node <node> | grep -A8 Allocated
# Get a shell next to a broken container
kubectl debug -it pod/<pod> -n <ns> \
--image=busybox --target=<container>
kubectl debug node/<node> -it --image=ubuntu
# Networking
kubectl get endpointslices -n <ns>
kubectl run tmp --rm -it --image=busybox -- \
nslookup web.orders.svc.cluster.local
# Access
aws sts get-caller-identity
kubectl auth can-i --list -n <ns>
aws eks list-access-entries --cluster-name prod
# Cluster health & upgrades
aws eks describe-cluster --name prod \
--query 'cluster.{status:status,version:version}'
aws eks list-insights --cluster-name prod
aws eks describe-addon --cluster-name prod \
--addon-name vpc-cni --query 'addon.health'
# Karpenter / Auto Mode
kubectl get nodepools,nodeclaims
kubectl describe nodeclaim <name> # events, conditions
# self-managed Karpenter only (namespace may differ):
kubectl logs -n kube-system -l app.kubernetes.io/name=karpenter
# CNI / IPs
kubectl logs -n kube-system -l k8s-app=aws-node -c aws-node
aws ec2 describe-subnets --subnet-ids subnet-0abc \
--query 'Subnets[].AvailableIpAddressCount'
Build, secure, scale, observe, upgrade, break, and tear down real EKS clusters — one lab per part of this deck.
Use a sandbox account. Every lab assumes Region ap-northeast-1 and a cluster named lab.
kubectl (±1 minor of the cluster), eksctl, helm.k9s, jq, hey or busybox for load.aws sts get-caller-identity before anything else.| Item | Price driver |
|---|---|
| Control plane | US$0.10 per cluster-hour (standard support); US$0.60 in extended support |
| Auto Mode | Per-instance management fee on top of EC2 |
| Nodes | EC2 instance-hours (Spot is cheaper) |
| NAT gateway | Hourly + per-GB processed |
| Load balancers | Hourly + LCU for each ALB/NLB |
| EBS, logs | GB-month; CloudWatch Logs ingestion |
kubectl sessions.Goal: replace the one-line eksctl create cluster with a reviewable file you can rebuild from.
# cluster.yaml
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata: { name: lab, region: ap-northeast-1, version: "1.35" }
autoModeConfig: { enabled: true } # general-purpose + system pools
accessConfig: { authenticationMode: API }
vpc:
clusterEndpoints: { publicAccess: true, privateAccess: true }
publicAccessCIDRs: ["203.0.113.10/32"] # your IP only
cloudWatch:
clusterLogging: { enableTypes: ["api", "audit", "authenticator"] }
eksctl create cluster -f cluster.yaml # ~15 min
aws eks describe-cluster --name lab --query \
'cluster.{v:version,auth:accessConfig,ep:resourcesVpcConfig}'kubectl get nodes is empty until a pod needs capacity — that is Auto Mode working.publicAccessCIDRs (eksctl utils update-cluster-vpc-config) and watch kubectl time out; restore it.aws logs describe-log-groups --log-group-name-prefix /aws/eks/lab shows the control-plane log group.terraform-aws-modules/eks — same settings, one PR per change.Goal: give a read-only team role access to one namespace — no aws-auth ConfigMap.
ACCT=$(aws sts get-caller-identity --query Account --output text)
ROLE=arn:aws:iam::$ACCT:role/eks-viewer # role you can assume
kubectl create namespace demo
aws eks create-access-entry --cluster-name lab \
--principal-arn $ROLE --type STANDARD
aws eks associate-access-policy --cluster-name lab \
--principal-arn $ROLE \
--policy-arn arn:aws:eks::aws:cluster-access-policy/AmazonEKSViewPolicy \
--access-scope type=namespace,namespaces=demo
aws eks update-kubeconfig --name lab --region ap-northeast-1 \
--role-arn $ROLE --alias lab-viewer
kubectl --context lab-viewer get pods -n demo # allowed
kubectl --context lab-viewer get pods -n kube-system # Forbidden
kubectl --context lab-viewer auth can-i delete pods -n demo # noaws eks list-access-entries --cluster-name lab shows your creator entry, the insights entry, and the new one.AmazonEKSEditPolicy and re-run auth can-i.Unauthorized (authentication), not Forbidden (authorization). Know the difference.CreateAccessEntry event in CloudTrail.Goal: one pod can list S3; every other pod cannot. Auto Mode has the Pod Identity agent built in.
cat > trust.json <<'EOF'
{"Version":"2012-10-17","Statement":[{"Effect":"Allow",
"Principal":{"Service":"pods.eks.amazonaws.com"},
"Action":["sts:AssumeRole","sts:TagSession"]}]}
EOF
aws iam create-role --role-name lab-s3-reader \
--assume-role-policy-document file://trust.json
aws iam attach-role-policy --role-name lab-s3-reader \
--policy-arn arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess
kubectl create sa s3-reader -n demo
aws eks create-pod-identity-association --cluster-name lab \
--namespace demo --service-account s3-reader \
--role-arn arn:aws:iam::$ACCT:role/lab-s3-reader
kubectl run cli -n demo --image=public.ecr.aws/aws-cli/aws-cli \
--overrides='{"spec":{"serviceAccountName":"s3-reader"}}' \
--command -- sleep 3600
kubectl exec -n demo cli -- aws s3 lskubectl exec … -- env | grep AWS_CONTAINER shows the agent endpoint, not keys.aws s3 ls fails with no credentials.eks-pod-identity-agent add-on first.Goal: see which Kubernetes object creates which AWS load balancer. Auto Mode runs the controller for you.
# Service → NLB (default loadBalancerClass: eks.amazonaws.com/nlb)
apiVersion: v1
kind: Service
metadata:
name: web-nlb
annotations:
service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
spec:
type: LoadBalancer
selector: { app: web }
ports: [{ port: 80, targetPort: 80 }]
---
apiVersion: eks.amazonaws.com/v1
kind: IngressClassParams
metadata: { name: alb }
spec: { scheme: internet-facing }
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata: { name: alb }
spec:
controller: eks.amazonaws.com/alb
parameters:
{ apiGroup: eks.amazonaws.com, kind: IngressClassParams, name: alb }ingressClassName: alb, path /, backend web port 80 (needs a ClusterIP or NodePort Service).kubectl get svc,ingress -w until an address appears (2–4 min); curl both.ip — targets are pod IPs.kubernetes.io/role/elb tag from public subnets in a test VPC — the load balancer stays pending; kubectl describe shows why.elbv2.k8s.aws APIs instead.Goal: prove isolation works and learn why DNS is the first thing a deny-all breaks.
# Auto Mode: turn on the network policy controller once
kubectl apply -f - <<'EOF'
apiVersion: v1
kind: ConfigMap
metadata: { name: amazon-vpc-cni, namespace: kube-system }
data: { enable-network-policy-controller: "true" }
EOF
# deny all ingress + egress in demo
kubectl apply -n demo -f - <<'EOF'
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: { name: default-deny }
spec: { podSelector: {}, policyTypes: [Ingress, Egress] }
EOF
kubectl exec -n demo cli -- aws s3 ls # now times out
aws eks describe-cluster --name lab --query \
cluster.kubernetesNetworkConfig.serviceIpv4Cidr # e.g. 10.100.0.0/16.10 address of the service CIDR (e.g. 10.100.0.10/32), so use an ipBlock, not a pod selector.web only from pods labelled role=frontend; test with two curl pods.vpc-cni add-on (enableNetworkPolicy: "true") and select CoreDNS with k8s-app: kube-dns.Goal: block unsafe pods at admission and see vulnerabilities before deploy.
# 1. Pod Security Admission: enforce restricted on demo
kubectl label ns demo pod-security.kubernetes.io/enforce=restricted
kubectl run bad -n demo --image=nginx
# → Error: violates PodSecurity "restricted:latest" (runAsNonRoot …)
kubectl label ns demo pod-security.kubernetes.io/warn=restricted \
pod-security.kubernetes.io/enforce=baseline --overwrite
# 2. ECR: scan on push, immutable tags
aws ecr create-repository --repository-name lab/web \
--image-scanning-configuration scanOnPush=true \
--image-tag-mutability IMMUTABLE
# push an image, then:
aws ecr describe-image-scan-findings --repository-name lab/web \
--image-id imageTag=1.0 \
--query 'imageScanFindings.findingSeverityCounts'bad pod with the hardened spec from the Security part (non-root, drop ALL, seccomp RuntimeDefault) until it is admitted.describe-cluster --query cluster.encryptionConfig shows your own KMS key only if you set one.kubectl run with a privileged pod in a non-restricted namespace.Goal: watch an EBS volume follow a pod, and learn why volumes are AZ-bound.
kubectl apply -f - <<'EOF'
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata: { name: auto-ebs-sc }
provisioner: ebs.csi.eks.amazonaws.com # Auto Mode block storage
volumeBindingMode: WaitForFirstConsumer
allowedTopologies:
- matchLabelExpressions:
- { key: eks.amazonaws.com/compute-type, values: [auto] }
parameters: { type: gp3, encrypted: "true" }
EOF
# StatefulSet "db" with volumeClaimTemplates → storageClassName: auto-ebs-sc
kubectl get pvc,pv -n demo
kubectl exec -n demo db-0 -- sh -c 'date >> /data/log; cat /data/log'
kubectl delete pod -n demo db-0 # comes back, same dataaws ec2 describe-volumes --filters Name=tag-key,Values=kubernetes.io/created-for/pvc/name shows the gp3 volume and its AZ.reclaimPolicy: Delete the EBS volume goes too.aws-ebs-csi-driver add-on and provisioner ebs.csi.aws.com.Goal: drive load, watch replicas then nodes grow, and see consolidation shrink them.
aws eks create-addon --cluster-name lab \
--addon-name metrics-server
kubectl set resources deploy web -n demo \
--requests=cpu=200m,memory=128Mi
kubectl apply -n demo -f - <<'EOF'
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata: { name: web }
spec:
scaleTargetRef: { apiVersion: apps/v1, kind: Deployment, name: web }
minReplicas: 2
maxReplicas: 30
metrics:
- type: Resource
resource:
name: cpu
target: { type: Utilization, averageUtilization: 50 }
EOF
kubectl run load -n demo --image=busybox:1.36 --restart=Never -- \
sh -c 'while true; do wget -q -O- http://web; done'
kubectl get hpa,pods -n demo -w ; kubectl get nodeclaims -wkubectl top pods -n demo works only after metrics-server is ready.load pod: HPA scales down after its stabilization window, then empty or underused nodes are consolidated.<unknown> — utilization needs requests.demo still enforces restricted PSA from Lab 6, the busybox load pod is rejected — use baseline or harden it.Goal: run stateless work on Spot, cap spend, and see how a workload chooses a pool.
kubectl apply -f - <<'EOF'
apiVersion: karpenter.sh/v1
kind: NodePool
metadata: { name: spot }
spec:
template:
spec:
nodeClassRef:
{ group: eks.amazonaws.com, kind: NodeClass, name: default }
requirements:
- key: karpenter.sh/capacity-type
operator: In
values: [spot]
- key: eks.amazonaws.com/instance-category
operator: In
values: [c, m, r]
limits: { cpu: "32" }
disruption:
consolidationPolicy: WhenEmptyOrUnderutilized
consolidateAfter: 1m
EOF
kubectl patch deploy web -n demo --type=json -p '[{"op":"add",
"path":"/spec/template/spec/nodeSelector",
"value":{"karpenter.sh/capacity-type":"spot"}}]'
kubectl get nodes -L karpenter.sh/capacity-typelimits.cpu: "2" and scale up: pods stay Pending once the cap is reached — a budget guardrail.kubectl top pods with requests; lower requests and watch consolidation pack nodes.Goal: build the non-Auto Mode path once, so you know what Auto Mode hides. Use a second cluster, lab-k.
# Follow karpenter.sh Getting Started (current release)
export CLUSTER_NAME=lab-k KARPENTER_NAMESPACE=kube-system
export KARPENTER_VERSION="<current, e.g. 1.15.0>"
# 1. IAM via the release's CloudFormation template
aws cloudformation deploy --stack-name "Karpenter-${CLUSTER_NAME}" \
--template-file cloudformation.yaml \
--capabilities CAPABILITY_NAMED_IAM \
--parameter-overrides "ClusterName=${CLUSTER_NAME}"
# 2. eksctl config: managedNodeGroups (2 × m5.large AL2023),
# podIdentityAssociations for the karpenter SA, eks-pod-identity-agent
eksctl create cluster -f lab-k.yaml
# 3. Controller
helm upgrade --install karpenter \
oci://public.ecr.aws/karpenter/karpenter \
--version "${KARPENTER_VERSION}" --namespace "${KARPENTER_NAMESPACE}" \
--set settings.clusterName=${CLUSTER_NAME} \
--set settings.interruptionQueue=${CLUSTER_NAME} --wait
# 4. NodePool + EC2NodeClass (karpenter.k8s.aws/v1,
# amiSelectorTerms: alias al2023@latest)| Need | Pick |
|---|---|
| Least ops, small platform team | Auto Mode |
| Custom AMIs, DaemonSet-heavy, SSH/SSM to nodes | MNG + Karpenter |
| Steady baseline, simple scaling | MNG (+ Cluster Autoscaler) |
| Karpenter controller placement | On the MNG (or Fargate), never on nodes it manages |
| Core add-ons | Auto Mode: built in · MNG: you upgrade them |
Goal: manage the platform layer on lab-k the way you would in production.
aws eks list-addons --cluster-name lab-k
aws eks describe-addon-versions --addon-name vpc-cni \
--kubernetes-version 1.35 \
--query 'addons[].addonVersions[].addonVersion'
aws eks update-addon --cluster-name lab-k --addon-name vpc-cni \
--addon-version <version> --resolve-conflicts PRESERVE
aws eks describe-addon --cluster-name lab-k --addon-name vpc-cni \
--query 'addon.{status:status,v:addonVersion,health:health}'
# configuration values (JSON schema per add-on)
aws eks describe-addon-configuration --addon-name vpc-cni \
--addon-version <version> --query configurationSchema \
--output text | jq .aws-node DaemonSet by hand, then update with --resolve-conflicts NONE: the update fails on the conflict. OVERWRITE wins; PRESERVE keeps your change.ENABLE_PREFIX_DELEGATION) and compare max pods on a new node.lab) the CNI, CoreDNS, kube-proxy, and EBS CSI do not appear as add-ons — AWS runs them.Goal: answer "who deleted my deployment?" and "why is this pod restarting?" from data, not guesses.
# Logs Insights on /aws/eks/lab/cluster
fields @timestamp, user.username, verb, objectRef.name
| filter @logStream like /kube-apiserver-audit/
| filter verb = "delete" and objectRef.resource = "deployments"
| sort @timestamp desc | limit 20
# From the shell
kubectl delete deploy web -n demo # create the evidence first
kubectl get events -n demo --sort-by=.lastTimestamp | tail
kubectl top nodes ; kubectl top pods -A --sort-by=memory | head
aws cloudwatch list-metrics --namespace AWS/EKS \
--dimensions Name=ClusterName,Value=lab --query 'Metrics[].MetricName'--limits=memory=32Mi on a memory-hungry image) and find OOMKilled in describe and events.amazon-cloudwatch-observability add-on (Pod Identity for its agent; check describe-addon-versions for your cluster) and open Container Insights.Goal: keep a service up through node maintenance and a bad deploy.
kubectl create deploy web -n demo --replicas=4 \
--image=public.ecr.aws/nginx/nginx
kubectl create pdb web -n demo --selector=app=web --min-available=3
# spread across zones (add to the pod template)
# topologySpreadConstraints:
# - maxSkew: 1
# topologyKey: topology.kubernetes.io/zone
# whenUnsatisfiable: ScheduleAnyway
# labelSelector: { matchLabels: { app: web } }
kubectl get pods -n demo -o wide -L topology.kubernetes.io/zone
NODE=$(kubectl get pods -n demo -l app=web \
-o jsonpath='{.items[0].spec.nodeName}')
kubectl drain $NODE --ignore-daemonsets --delete-emptydir-data
# → evictions wait whenever they would break minAvailable
# broken rollout: an image tag that does not exist
kubectl set image deploy/web -n demo \
nginx=public.ecr.aws/nginx/nginx:does-not-exist
kubectl rollout status deploy/web -n demo --timeout=60s
kubectl rollout undo deploy/web -n demokubectl get pdb -n demo shows ALLOWED DISRUPTIONS hitting 0.--min-available=4 with 4 replicas: the drain blocks forever — a classic upgrade blocker.ImagePullBackOff./healthz that returns 404: pods run but never become Ready, and the rollout stalls the same way.Goal: run the upgrade runbook from the Upgrades part on lab (built at 1.35).
aws eks start-insights-refresh --cluster-name lab
aws eks list-insights --cluster-name lab \
--filter '{"categories":["UPGRADE_READINESS"],
"statuses":["ERROR","WARNING"]}'
aws eks update-cluster-version --name lab --kubernetes-version 1.36
# or: eksctl upgrade cluster --name lab --version 1.36 --approve
aws eks describe-update --name lab --update-id <id> # → Successful
kubectl version ; kubectl get nodes -o wide # kubelet versions
# Auto Mode replaces nodes; MNG: update-nodegroup-version
# within 7 days you may roll back one minor version
aws eks list-insights --cluster-name lab \
--filter '{"categories":["ROLLBACK_READINESS"]}'
aws eks update-cluster-version --name lab --kubernetes-version 1.35pluto, kubent).Have a partner cause one of these without telling you which; fix it using only the cluster and AWS CLI.
| Inject | Symptom | Where to look | Fix |
|---|---|---|---|
NodePool limits.cpu too low | Pods Pending, no new nodes | describe pod events, kubectl get nodepools, NodeClaim events | Raise limits or rightsize requests |
| Small subnet / no free IPs (lab-k) | Pods stuck ContainerCreating | aws-node / ipamd logs, subnet free IP count | Prefix delegation, bigger or secondary CIDR |
| Pod Identity association removed | App gets AccessDenied / no credentials | list-pod-identity-associations, CloudTrail AssumeRoleForPodIdentity | Recreate association; check trust and SA name |
| Public subnets missing ELB tag | Service EXTERNAL-IP stays <pending> | kubectl describe svc events | Tag subnets kubernetes.io/role/elb=1 |
| Wrong image tag | ImagePullBackOff | describe pod; ECR repo; node pull permissions | Fix tag / digest; ECR pull policy |
| Default-deny without DNS allow | Timeouts, no such host | kubectl get networkpolicy; test with nslookup | Allow port 53 to CoreDNS |
Goal: zero orphaned load balancers, volumes, or IAM roles after the labs.
# 1. Delete what creates AWS resources, while controllers still run
kubectl delete ingress,svc --all -n demo
kubectl delete statefulset,pvc --all -n demo
kubectl delete nodepool spot
# 2. Clusters (and Karpenter's CloudFormation stack)
eksctl delete cluster --name lab --region ap-northeast-1 --wait
eksctl delete cluster --name lab-k --region ap-northeast-1 --wait
aws cloudformation delete-stack --stack-name Karpenter-lab-k
# 3. Prove nothing is left
aws elbv2 describe-load-balancers --query 'LoadBalancers[].LoadBalancerName'
aws ec2 describe-volumes --filters Name=status,Values=available \
--query 'Volumes[].[VolumeId,Size,CreateTime]'
aws iam delete-role --role-name lab-s3-reader # after detaching its policy
aws ecr delete-repository --repository-name lab/web --forceTick each one without looking at the slides.
lab from cluster.yaml alone.API auth mode + access entries| MNG | Managed node group — EKS-managed Auto Scaling group | IRSA | IAM Roles for Service Accounts (OIDC-based) |
| Auto Mode | AWS-managed nodes and core add-ons | Pod Identity | EKS-native pod → IAM role mapping |
| Karpenter | Node autoscaler that launches right-sized EC2 | Access entry | IAM principal → cluster access mapping |
| NodePool / NodeClaim | Karpenter's node template / a requested node | PSA / PSS | Pod Security Admission / Standards |
| VPC CNI | Plugin giving pods VPC IPs | PDB | PodDisruptionBudget — limits voluntary evictions |
| ENI / prefix delegation | Network interface / /28 blocks per slot | HPA / VPA / KEDA | Horizontal, vertical, event-driven pod autoscaling |
| CSI | Container Storage Interface drivers | AMP / AMG | Amazon Managed Prometheus / Grafana |
| Add-on | EKS-managed cluster component | ADOT | AWS Distro for OpenTelemetry |
| Extended support | Extra 12 months per version at $0.60/h | SLO | Service level objective with an error budget |
docs.aws.amazon.com/eks/latest/best-practices/docs.aws.amazon.com/eks/latest/userguide/eksworkshop.comkarpenter.shkubernetes-sigs.github.io/aws-load-balancer-controllerregistry.terraform.iokubernetes.io/docsPrices and version support windows change — verify against the AWS pages before making decisions.