Provision two dedicated Direct Connect connections at two different Direct Connect locations.
Why: Using two different locations protects against location-level failures (fiber cuts, power outages), providing maximum resiliency. A single location, even with multiple connections, is a single point of failure.
Bidirectional DNS resolution between on-premises and AWS private hosted zones.
Use Route 53 Resolver. Create Inbound Endpoints for on-prem to query AWS. Create Outbound Endpoints with forwarding rules for AWS to query on-prem.
Why: Inbound endpoints provide reachable IPs for on-prem DNS forwarders. Outbound endpoints enable conditional forwarding from within the VPC. The VPC default resolver (VPC+2) is not reachable from on-prem.
Provide service-level access between two VPCs without creating network-layer routes.
Use AWS PrivateLink. Create a VPC Endpoint Service (backed by an NLB) in the provider VPC and an Interface VPC Endpoint in the consumer VPC.
Why: PrivateLink provides unidirectional, service-specific connectivity using ENIs in the consumer's VPC, completely avoiding network-level routing and CIDR overlap issues.
Provide outbound-only internet access for IPv6-enabled instances in private subnets.
Create an Egress-Only Internet Gateway (EIGW) and add a route for `::/0` to the private subnet's route table pointing to the EIGW.
Why: An EIGW is stateful for outbound IPv6 connections, allowing return traffic but preventing unsolicited inbound connections, analogous to a NAT Gateway but for IPv6.
Inspect all inter-VPC traffic using AWS Network Firewall in a centralized model with Transit Gateway.
Create a dedicated inspection VPC with Network Firewall. Configure TGW route tables to send all inter-VPC traffic to the inspection VPC. Within the inspection VPC, route tables must steer traffic through the NFW endpoints for symmetric routing.
Why: This architecture requires careful routing: TGW sends traffic to the inspection VPC; VPC route tables send it to the firewall endpoint; the firewall sends it back to the TGW attachment ENI; TGW routes it to the final destination.
Segmenting VPCs (e.g., prod vs. dev) using Transit Gateway, while allowing both to access a shared services VPC.
Use multiple Transit Gateway route tables. Create a route table for each segment (prod, dev, shared). Associate VPCs with their respective tables. Propagate routes to create a hub-spoke topology where spokes can only see the hub.
Why: TGW route table associations and propagations are the primary mechanism for network segmentation and traffic isolation at the network layer.
Reduce latency for a dynamic, non-cacheable global application (e.g., API, gaming) hosted in a single region.
Use AWS Global Accelerator. It provides anycast IPs that route users to the nearest AWS edge location, then traffic traverses the optimized AWS backbone to the origin.
Why: Global Accelerator optimizes the "first mile" and "middle mile" over the AWS network, reducing latency and jitter for TCP/UDP traffic. CloudFront is better for cacheable content.
Provide private access from a VPC to S3 and DynamoDB without traversing the internet.
Create Gateway VPC Endpoints for S3 and DynamoDB. This adds prefix list entries to the specified subnet route tables.
Why: Gateway endpoints are the specific, highly-performant, and no-cost mechanism for S3 and DynamoDB private access. Other services use Interface Endpoints (PrivateLink).
Access VPCs in multiple AWS Regions from a single on-premises Direct Connect connection.
Use a Direct Connect Gateway with a Transit Virtual Interface (T-VIF). Associate the DX Gateway with Transit Gateways in each required Region.
Why: A Transit VIF with a DX Gateway is the scalable solution for connecting to multiple Transit Gateways across regions. A Private VIF with a DX Gateway has lower limits.
Integrate a third-party virtual firewall appliance for transparent traffic inspection.
Use a Gateway Load Balancer (GWLB). It operates at Layer 3 and uses the GENEVE protocol to encapsulate traffic, preserving the original source/destination IP.
Why: GWLB's GENEVE encapsulation makes it a "bump-in-the-wire," transparently inserting appliances into the network path without requiring source NAT, which is critical for security appliances.
Manage IP address allocation for hundreds of VPCs across a multi-account organization to prevent overlaps and track usage.
Use Amazon VPC IP Address Manager (IPAM). Create a top-level pool and delegate Regional pools to automate VPC CIDR allocation.
Why: VPC IPAM is the purpose-built, scalable AWS service for centralized IP address management, replacing error-prone manual methods.
Integrate a third-party SD-WAN appliance with Transit Gateway using GRE tunnels and dynamic BGP routing.
Use a Transit Gateway Connect attachment.
Why: TGW Connect is specifically designed for SD-WAN integration. It supports GRE for higher bandwidth (up to 5 Gbps per peer) and BGP for dynamic routing.
An IPv6-only VPC needs to communicate with IPv4-only resources on the internet.
Enable DNS64 on the VPC's Route 53 Resolver settings and configure a NAT Gateway in a public subnet. Route `64:ff9b::/96` to the NAT Gateway.
Why: DNS64 synthesizes AAAA records for IPv4 destinations. The NAT Gateway performs the NAT64 protocol translation from the synthesized IPv6 address to the real IPv4 address.
Connect hundreds of VPCs across many regions with strict segmentation requirements (prod, dev, shared services).
Use AWS Cloud WAN. Define segments and segment actions in a single core network policy to control inter-segment routing globally.
Why: Cloud WAN provides a centralized, declarative global network policy, which is more scalable and less complex than managing a full mesh of Transit Gateway peerings and route tables in each region.
Network Implementation
Aggregating multiple Direct Connect connections at a single location for increased bandwidth and link redundancy.
Configure a Link Aggregation Group (LAG). All connections must have the same bandwidth and terminate on the same AWS device.
Why: LAGs bundle physical links into one logical link. This provides link-level failover but does not protect against device or location failure. Use `minimum links` to define the failover threshold.
Achieve sub-60-second DNS failover for a multi-Region application.
Use Route 53 failover routing with health checks. Configure fast interval (10s) health checks with a low failure threshold (1). Use Alias records or very low TTLs (e.g., 10-60s).
Why: Fast failover requires rapid detection (fast health checks) and rapid client-side updates (low TTLs or Alias records which have dynamic TTLs).
Increase aggregate VPN throughput by using both tunnels of a Site-to-Site VPN connection simultaneously.
Attach the VPN to a Transit Gateway. Enable ECMP support on the Transit Gateway VPN attachment.
Why: By default, VPN to TGW may use only one tunnel. Enabling ECMP on the TGW attachment distributes traffic across both tunnels if BGP routes are equal-cost.
Ensure backend TCP services behind a Network Load Balancer see the original client IP address.
Register targets by instance ID. If targets are registered by IP, enable Proxy Protocol v2 on the target group.
Why: When targets are registered by instance ID, the NLB preserves the client IP by default. If registered by IP, the NLB's IP becomes the source, and Proxy Protocol v2 is needed to pass the original IP.
Influence how AWS routes traffic back to an on-premises network when multiple Direct Connect paths exist.
Use BGP AS path prepending on the less-preferred path from the on-premises router.
Why: For outbound traffic from AWS, the primary customer-controlled mechanism is AS path length. AWS prefers the path with the shortest AS path. You cannot configure local preference on the AWS side.
Enable spoke accounts in an AWS Organization to attach their VPCs to a Transit Gateway owned by a central networking account.
Use AWS Resource Access Manager (RAM). The networking account shares the Transit Gateway with the Organization or specific OUs.
Why: RAM is the specific AWS service designed for sharing resources like Transit Gateways across accounts. This allows centralized management while enabling self-service attachment for spoke accounts.
Aggregate throughput beyond the 1.25 Gbps limit of a single VPN tunnel.
Create multiple Site-to-Site VPN connections to a Transit Gateway with ECMP enabled.
Why: Each VPN tunnel is limited to ~1.25 Gbps. To scale, you must use multiple tunnels/connections and leverage ECMP on a Transit Gateway to load-balance traffic across them.
Configure Route 53 health checks for an internal, non-internet-facing resource like an internal ALB.
Create a CloudWatch alarm that monitors a metric for the internal resource (e.g., `HealthyHostCount` for an ALB). Configure the Route 53 health check to monitor the state of the CloudWatch alarm.
Why: Route 53 health checkers are external. To monitor internal resources, they must monitor a proxy signal like a CloudWatch alarm state, which can be triggered by internal metrics.
Automatically failover CloudFront traffic to a secondary origin (e.g., a static S3 site) when the primary origin (e.g., an ALB) returns 5xx errors.
Create a CloudFront Origin Group with the ALB as primary and S3 as secondary. Configure it to failover on specified status codes (e.g., 500, 502, 503, 504).
Why: Origin Groups are the native CloudFront mechanism for high availability, providing seamless failover at the edge without requiring DNS changes.
Achieve sub-second failover for a Direct Connect connection to a backup VPN or secondary DX path.
Enable Bidirectional Forwarding Detection (BFD) on the Direct Connect virtual interface. Configure BFD on the on-premise router.
Why: BFD provides much faster link failure detection (as low as 300ms) compared to BGP keepalive timers (default 90s), enabling rapid traffic reconvergence.
Enable connectivity between Transit Gateways in two different regions.
Establish a Transit Gateway peering connection. Manually add static routes in each TGW route table pointing to the remote region's CIDRs via the peering attachment.
Why: Crucially, Transit Gateway inter-region peering does not support dynamic route propagation. All cross-region routes must be configured statically.
Network Management and Operation
Troubleshoot connectivity issues within AWS by identifying the specific blocking component (e.g., route, NACL, SG).
Use VPC Reachability Analyzer. Specify a source and destination, and it performs a static analysis of the network path configuration.
Why: Reachability Analyzer provides a definitive, hop-by-hop analysis of AWS network constructs, which is more effective than traceroute (which may not work) or manually checking every component.
Long-term storage and ad-hoc querying of detailed VPC Flow Logs for compliance.
Publish flow logs directly to S3 in Parquet format with a custom field layout. Use Amazon Athena for SQL-based ad-hoc queries.
Why: S3 is the most cost-effective storage. Parquet format is highly efficient for Athena queries, reducing scan costs and improving performance. This is the serverless, scalable pattern for flow log analysis.
Centrally enforce mandatory security group rules across all VPCs in an AWS Organization and auto-remediate non-compliance.
Use AWS Firewall Manager with a security group audit policy. Define the required rules and configure the policy to automatically remediate non-compliant groups.
Why: Firewall Manager is the centralized governance tool for security policies (WAF, SG, NFW) across an Organization. Its audit policy with auto-remediation provides enforcement.
Visualize and monitor a global network topology including Transit Gateways, VPNs, and Direct Connect across regions and accounts.
Use AWS Network Manager. Register Transit Gateways into a single global network to get a centralized dashboard, topology map, and health monitoring.
Why: Network Manager is purpose-built to provide a single pane of glass for complex, global AWS networks, consolidating monitoring and management.
Diagnose intermittent packet loss or errors on a Direct Connect connection.
Check Direct Connect CloudWatch metrics (`ConnectionErrorCount`). On the customer router, check optical signal levels (Tx/Rx light levels) and interface error counters (CRC errors, input errors).
Why: Packet loss can be a physical layer issue. Both AWS-side metrics and customer-side router diagnostics are needed to isolate problems like a degrading fiber optic cable or transceiver.
Automatically detect and remediate non-compliant network configurations, such as a security group allowing public SSH access.
Use AWS Config with a managed rule (e.g., `restricted-ssh`) and configure an automatic remediation action using an SSM Automation document.
Why: This provides a closed-loop compliance system. AWS Config detects the violation, and its remediation action triggers an SSM document to automatically fix the configuration.
Proactively identify unintended network access paths to sensitive resources from the internet or other untrusted networks.
Use VPC Network Access Analyzer. Define an access scope and run an analysis to get a list of all possible network paths that match.
Why: This tool performs a formal network verification, analyzing all components (SGs, NACLs, TGW, IGW) to find potential paths, which is more comprehensive than manual checks or reactive monitoring.
Collect VPC Flow Logs, DNS Query Logs, and Network Firewall logs from many member accounts into a central logging account.
Configure services in member accounts to publish logs directly to a centralized S3 bucket (for Flow Logs/NFW) or CloudWatch Log Group (for DNS logs) in the logging account, using cross-account bucket policies and IAM roles.
Why: Direct cross-account log publishing is the most efficient and scalable pattern, leveraging native AWS capabilities without requiring agents or complex data pipelines.
Optimize network costs for high-volume traffic between specific pairs of VPCs in a Transit Gateway hub-and-spoke architecture.
For high-traffic VPC pairs, create a direct VPC peering connection to bypass the Transit Gateway. Keep the TGW for all other hub-spoke traffic.
Why: VPC peering has no per-GB data processing charge (only standard data transfer), while Transit Gateway does. Moving high-volume point-to-point traffic to peering significantly reduces costs.
Network Security, Compliance, and Governance
Encrypt Direct Connect traffic at Layer 2 for line-rate performance.
Enable MACsec (IEEE 802.1AE). Requires a dedicated 10Gbps or 100Gbps connection at a MACsec-capable location.
Why: MACsec provides hop-by-hop encryption between the customer router and the AWS device, securing the physical link with minimal performance overhead.
Protect a web application from common attacks (SQLi, XSS) and restrict access by country.
Use AWS WAF. Attach a web ACL to the ALB/CloudFront. Use AWS Managed Rule groups (e.g., `AWSManagedRulesSQLiRuleSet`, `AWSManagedRulesCommonRuleSet`) and create a geo-match rule.
Why: AWS Managed Rules provide out-of-the-box protection for common threats, while geo-match rules provide geographic control. This is the standard WAF implementation pattern.
Implement multi-tier application isolation (e.g., web -> app -> db) within a VPC.
Create a security group for each tier. Use security group ID references in the rules (e.g., the app-sg allows ingress from the web-sg).
Why: Referencing security groups is more dynamic and secure than using CIDR ranges. It automatically adapts as instances are added or removed from a tier.
Monitor and detect DNS-based threats like DNS tunneling and communication with C2 servers.
Enable Route 53 Resolver query logging. Enable Amazon GuardDuty, which analyzes the DNS query logs as a data source.
Why: GuardDuty has built-in threat intelligence that analyzes DNS logs to detect known malicious domains, DGAs, and anomalous query patterns indicative of data exfiltration.
Ensure S3 content can only be accessed via CloudFront, not directly via the S3 URL, while still allowing other IAM principals access.
Use Origin Access Control (OAC). Update the S3 bucket policy to allow access from the OAC service principal and any other required IAM roles/users.
Why: OAC is the modern replacement for OAI. It creates a service principal that can be referenced in bucket policies, providing more granular and flexible access control.
Prevent users from bypassing CloudFront and accessing an ALB origin directly.
Configure CloudFront to add a custom HTTP header with a secret value to origin requests. Create an ALB listener rule that checks for this header and value, blocking requests that don't have it.
Why: This provides stronger protection than IP-based restrictions (using the managed prefix list), as it verifies the request came from your specific distribution. Use both for defense-in-depth.
Capture full network packet data from a specific EC2 instance for forensic analysis without installing agents.
Use VPC Traffic Mirroring. Configure a mirror session on the instance's ENI to copy traffic to a target (e.g., an NLB fronting analysis tools).
Why: Traffic Mirroring provides agentless, full packet capture, which is essential for deep forensic analysis. Flow logs only provide metadata.
Provide comprehensive DDoS protection for public-facing applications.
Subscribe to AWS Shield Advanced. Associate protection with critical resources (CloudFront, ALB, EIPs, Route 53). Use AWS WAF for Layer 7 mitigation. Engage the Shield Response Team (SRT) during attacks.
Why: Shield Advanced provides enhanced detection, cost protection against DDoS-induced scaling, and access to the SRT for expert assistance, which is critical for business-critical applications.
Issue and automatically rotate TLS certificates for internal microservices from a private Certificate Authority.
Use AWS Private Certificate Authority (Private CA) to create the CA. Use AWS Certificate Manager (ACM) to issue and manage the lifecycle (including auto-renewal) of private certificates from that CA.
Why: This combination provides a fully managed private PKI solution, automating the complex lifecycle of internal certificates without public exposure.
Prevent any user in any member account of an AWS Organization from creating or attaching an internet gateway.
Apply a Service Control Policy (SCP) at the Organization root that denies the `ec2:CreateInternetGateway` and `ec2:AttachInternetGateway` actions.
Why: SCPs provide preventative guardrails that cannot be overridden by IAM policies within member accounts, making them the definitive tool for enforcing organization-wide security policies.
Decrypt and inspect HTTPS traffic for threats using AWS Network Firewall, then re-encrypt it.
Create or import a CA certificate in ACM. Create a TLS inspection configuration in the firewall policy that references this CA. Distribute the CA certificate to client systems as a trusted root CA.
Why: Network Firewall performs TLS inspection via a man-in-the-middle approach, using a CA you provide to re-sign certificates on the fly. Clients must trust this CA to avoid certificate errors.
Block DNS queries to known malicious domains across all VPCs in an AWS Organization.
Create DNS Firewall rule groups in a central account using AWS managed domain lists. Share the rule groups via RAM and associate them with VPCs in member accounts.
Why: DNS Firewall provides managed, centrally-updatable threat intelligence to block malicious domains at the DNS resolution layer, a critical security control.
Encrypt all network traffic between EC2 instances in the same VPC without application changes.
Use AWS Nitro System-based instance types.
Why: Nitro instances automatically encrypt all traffic between instances at the hardware level, providing transparent, line-rate encryption with no configuration required.