AWS VPC Connectivity: Peering vs Transit Gateway vs PrivateLink vs VPN

AWS gives you six fundamentally different ways to connect VPCs, accounts, and on-premises networks — and picking the wrong one costs you real money, creates operational headaches, or creates security gaps that are painful to unwind. VPC Peering is cheap but doesn't scale. Transit Gateway scales beautifully but adds cost and complexity. PrivateLink solves a completely different problem. VPN is the right tool for on-premises hybrid connectivity, until your bandwidth needs push you toward Direct Connect.

This guide is the only reference you need. It covers every option with CLI commands, Terraform HCL, route table configurations, BGP examples, and a decision matrix you can take straight into your architecture review. We close with a complete real-world multi-account hub-and-spoke design that ties everything together.

VPC Connectivity Overview & Decision Flowchart

Before diving into configuration details, it helps to see all six options on one map. Each option solves a different connectivity problem. Choosing the wrong tool often means refactoring your entire network topology months later — a painful and expensive exercise.

AWS VPC CONNECTIVITY — DECISION FLOWCHART
==========================================

START: What do you need to connect?
  |
  +-- Two VPCs (same account, same region, <5 pairs)?
  |     --> VPC PEERING  (cheapest, simplest)
  |
  +-- Many VPCs (5+) OR need transitive routing?
  |     --> TRANSIT GATEWAY  (hub-and-spoke, centralized)
  |
  +-- Expose a service privately to consumers in OTHER VPCs/accounts?
  |     --> AWS PRIVATELINK  (consumer never sees your VPC CIDR)
  |
  +-- Connect on-premises network to AWS?
  |     |
  |     +-- Dev/test, <1.25 Gbps, IPsec OK?
  |     |     --> SITE-TO-SITE VPN  (hours to set up)
  |     |
  |     +-- Production, consistent latency, >1 Gbps?
  |           --> DIRECT CONNECT  (private fiber, SLA-backed)
  |
  +-- Remote users need private access to AWS resources?
        --> AWS CLIENT VPN  (OpenVPN-compatible, mutual TLS or AD auth)

Here is a high-level cost summary to anchor the decision before going deeper:

VPC Peering — Setup, Limits & When to Use

VPC Peering creates a direct, private network connection between two VPCs. Traffic stays on the AWS backbone — it never traverses the public internet. Peering works within a region, across regions, and across AWS accounts. The connection is point-to-point and non-transitive — this single fact is the most common source of VPC Peering regret at scale.

Non-Transitive Peering — The Core Limitation

VPC A <---peering---> VPC B <---peering---> VPC C

A CANNOT reach C through B.
You need a direct A-C peering connection.

For N VPCs: N*(N-1)/2 peering connections required
  5 VPCs  = 10 connections
  10 VPCs = 45 connections
  20 VPCs = 190 connections  <-- maintenance nightmare

Setting Up VPC Peering with the AWS CLI

# Step 1: Request peering connection (from requester account/VPC)
PEERING_ID=$(aws ec2 create-vpc-peering-connection \
  --vpc-id vpc-aaa111 \
  --peer-vpc-id vpc-bbb222 \
  --peer-region us-west-2 \
  --peer-owner-id 123456789012 \
  --query VpcPeeringConnection.VpcPeeringConnectionId \
  --output text)

echo "Peering connection: $PEERING_ID"

# Step 2: Accept the peering connection (run in accepter account/region)
aws ec2 accept-vpc-peering-connection \
  --vpc-peering-connection-id $PEERING_ID \
  --region us-west-2

# Step 3: Add route in VPC A's private route table pointing to VPC B's CIDR
aws ec2 create-route \
  --route-table-id rtb-aaaa1111 \
  --destination-cidr-block 10.1.0.0/16 \
  --vpc-peering-connection-id $PEERING_ID

# Step 4: Add return route in VPC B's private route table
aws ec2 create-route \
  --route-table-id rtb-bbbb2222 \
  --destination-cidr-block 10.0.0.0/16 \
  --vpc-peering-connection-id $PEERING_ID

# Step 5: (Cross-account only) Update DNS resolution to allow private hostnames
aws ec2 modify-vpc-peering-connection-options \
  --vpc-peering-connection-id $PEERING_ID \
  --requester-peering-connection-options AllowDnsResolutionFromRemoteVpc=true \
  --accepter-peering-connection-options AllowDnsResolutionFromRemoteVpc=true

VPC Peering in Terraform

# providers.tf
provider "aws" {
  alias  = "requester"
  region = "us-east-1"
}

provider "aws" {
  alias  = "accepter"
  region = "us-west-2"
}

# VPC Peering Connection
resource "aws_vpc_peering_connection" "peer" {
  provider    = aws.requester
  vpc_id      = aws_vpc.requester.id
  peer_vpc_id = aws_vpc.accepter.id
  peer_region = "us-west-2"
  auto_accept = false

  tags = {
    Name = "requester-to-accepter-peering"
    Side = "Requester"
  }
}

# Accept in the peer region
resource "aws_vpc_peering_connection_accepter" "peer" {
  provider                  = aws.accepter
  vpc_peering_connection_id = aws_vpc_peering_connection.peer.id
  auto_accept               = true

  tags = {
    Name = "requester-to-accepter-peering"
    Side = "Accepter"
  }
}

# Route in requester VPC
resource "aws_route" "requester_to_accepter" {
  provider                  = aws.requester
  route_table_id            = aws_route_table.requester_private.id
  destination_cidr_block    = "10.1.0.0/16"
  vpc_peering_connection_id = aws_vpc_peering_connection.peer.id
}

# Route in accepter VPC
resource "aws_route" "accepter_to_requester" {
  provider                  = aws.accepter
  route_table_id            = aws_route_table.accepter_private.id
  destination_cidr_block    = "10.0.0.0/16"
  vpc_peering_connection_id = aws_vpc_peering_connection.peer.id
}

Transit Gateway — Hub-and-Spoke at Scale

Transit Gateway (TGW) is a managed, regional network hub. Every VPC attaches once to TGW; TGW handles routing between all attachments. This hub-and-spoke model reduces the number of connections from O(N²) to O(N) and, critically, supports transitive routing — spoke A can reach spoke B through the hub without a direct peering connection.

TGW also connects to on-premises networks via Direct Connect Gateway and Site-to-Site VPN, making it the single control plane for your entire AWS network topology.

TRANSIT GATEWAY HUB-AND-SPOKE ARCHITECTURE

        Production VPC (10.0.0.0/16)
              |
Staging VPC --+-- [TRANSIT GATEWAY] --+-- Shared Services VPC (10.3.0.0/16)
(10.1.0.0/16) |                       |
              |               On-Premises (192.168.0.0/16)
        Dev VPC (10.2.0.0/16)         via Site-to-Site VPN or DX

Each VPC has ONE attachment to TGW.
TGW route tables control which spokes can reach which other spokes.

Creating Transit Gateway with AWS CLI

# Create the Transit Gateway
TGW_ID=$(aws ec2 create-transit-gateway \
  --description "Central network hub" \
  --options "AmazonSideAsn=64512,AutoAcceptSharedAttachments=disable,\
DefaultRouteTableAssociation=enable,DefaultRouteTablePropagation=enable,\
VpnEcmpSupport=enable,DnsSupport=enable,\
MulticastSupport=disable" \
  --tag-specifications 'ResourceType=transit-gateway,Tags=[{Key=Name,Value=main-tgw}]' \
  --query TransitGateway.TransitGatewayId \
  --output text)

aws ec2 wait transit-gateway-available --transit-gateway-ids $TGW_ID

# Attach Production VPC (subnets must be in different AZs for HA)
PROD_ATTACH=$(aws ec2 create-transit-gateway-vpc-attachment \
  --transit-gateway-id $TGW_ID \
  --vpc-id vpc-prod-aaa \
  --subnet-ids subnet-prod-az-a subnet-prod-az-b subnet-prod-az-c \
  --options "DnsSupport=enable,Ipv6Support=disable,ApplianceModeSupport=disable" \
  --tag-specifications 'ResourceType=transit-gateway-attachment,Tags=[{Key=Name,Value=prod-attach}]' \
  --query TransitGatewayVpcAttachment.TransitGatewayAttachmentId \
  --output text)

# Attach Shared Services VPC
SHARED_ATTACH=$(aws ec2 create-transit-gateway-vpc-attachment \
  --transit-gateway-id $TGW_ID \
  --vpc-id vpc-shared-ccc \
  --subnet-ids subnet-shared-az-a subnet-shared-az-b \
  --tag-specifications 'ResourceType=transit-gateway-attachment,Tags=[{Key=Name,Value=shared-attach}]' \
  --query TransitGatewayVpcAttachment.TransitGatewayAttachmentId \
  --output text)

# Add static route in VPC private route table pointing to TGW
# (or use VPC CIDR propagation — TGW auto-propagates attached VPC CIDRs)
aws ec2 create-route \
  --route-table-id rtb-prod-private \
  --destination-cidr-block 10.3.0.0/16 \
  --transit-gateway-id $TGW_ID

Segmented TGW Route Tables — Isolating Environments

By default, TGW uses one route table and all attachments can reach all others. For production environments, you want isolation: Dev should not reach Production. Use separate TGW route tables with selective propagation.

# Create separate route tables for prod and non-prod
PROD_RT=$(aws ec2 create-transit-gateway-route-table \
  --transit-gateway-id $TGW_ID \
  --tag-specifications 'ResourceType=transit-gateway-route-table,Tags=[{Key=Name,Value=prod-rt}]' \
  --query TransitGatewayRouteTable.TransitGatewayRouteTableId \
  --output text)

NONPROD_RT=$(aws ec2 create-transit-gateway-route-table \
  --transit-gateway-id $TGW_ID \
  --tag-specifications 'ResourceType=transit-gateway-route-table,Tags=[{Key=Name,Value=nonprod-rt}]' \
  --query TransitGatewayRouteTable.TransitGatewayRouteTableId \
  --output text)

# Associate production attachment with prod route table
aws ec2 associate-transit-gateway-route-table \
  --transit-gateway-route-table-id $PROD_RT \
  --transit-gateway-attachment-id $PROD_ATTACH

# Propagate shared services routes into prod route table
aws ec2 enable-transit-gateway-route-table-propagation \
  --transit-gateway-route-table-id $PROD_RT \
  --transit-gateway-attachment-id $SHARED_ATTACH

# DO NOT propagate prod routes into non-prod (isolation)
# Associate dev/staging attachments with nonprod route table only

Transit Gateway in Terraform

resource "aws_ec2_transit_gateway" "main" {
  description                     = "Central network hub"
  amazon_side_asn                 = 64512
  auto_accept_shared_attachments  = "disable"
  default_route_table_association = "disable"  # we manage route tables manually
  default_route_table_propagation = "disable"
  vpn_ecmp_support                = "enable"
  dns_support                     = "enable"

  tags = { Name = "main-tgw" }
}

resource "aws_ec2_transit_gateway_vpc_attachment" "production" {
  transit_gateway_id = aws_ec2_transit_gateway.main.id
  vpc_id             = aws_vpc.production.id
  subnet_ids         = aws_subnet.production_private[*].id

  transit_gateway_default_route_table_association = false
  transit_gateway_default_route_table_propagation = false

  tags = { Name = "prod-tgw-attachment" }
}

resource "aws_ec2_transit_gateway_route_table" "production" {
  transit_gateway_id = aws_ec2_transit_gateway.main.id
  tags               = { Name = "prod-tgw-rt" }
}

resource "aws_ec2_transit_gateway_route_table_association" "production" {
  transit_gateway_attachment_id  = aws_ec2_transit_gateway_vpc_attachment.production.id
  transit_gateway_route_table_id = aws_ec2_transit_gateway_route_table.production.id
}

resource "aws_ec2_transit_gateway_route_table_propagation" "shared_to_prod" {
  transit_gateway_attachment_id  = aws_ec2_transit_gateway_vpc_attachment.shared.id
  transit_gateway_route_table_id = aws_ec2_transit_gateway_route_table.production.id
}

# Static route in spoke VPC pointing to TGW
resource "aws_route" "prod_to_shared" {
  route_table_id         = aws_route_table.production_private.id
  destination_cidr_block = "10.3.0.0/16"
  transit_gateway_id     = aws_ec2_transit_gateway.main.id
}

AWS PrivateLink — Interface vs Gateway Endpoints

AWS PrivateLink is conceptually different from peering and Transit Gateway. Instead of routing entire VPC CIDRs to each other, PrivateLink exposes a specific service from a provider VPC to consumer VPCs. The consumer never sees the provider's CIDR block, and the provider never sees the consumer's CIDR block — communication goes through an elastic network interface (ENI) deployed in the consumer's subnet.

This makes PrivateLink ideal for SaaS-style internal services: a security team exposing a secrets vault, a platform team exposing a logging pipeline, or AWS exposing managed services (SSM, ECR, Secrets Manager) without requiring internet access.

Endpoint Types

Creating a Custom PrivateLink Service (NLB-Backed)

# ---- PROVIDER SIDE ----
# Step 1: Create an NLB in the provider VPC pointing to your service
NLB_ARN=$(aws elbv2 create-load-balancer \
  --name my-internal-service-nlb \
  --type network \
  --scheme internal \
  --subnets subnet-provider-az-a subnet-provider-az-b \
  --query LoadBalancers[0].LoadBalancerArn \
  --output text)

# Step 2: Create the VPC Endpoint Service backed by the NLB
SERVICE_ID=$(aws ec2 create-vpc-endpoint-service-configuration \
  --network-load-balancer-arns $NLB_ARN \
  --acceptance-required \
  --private-dns-name my-service.techoral.internal \
  --query ServiceConfiguration.ServiceId \
  --output text)

# Allow specific consumer accounts to create endpoints
aws ec2 modify-vpc-endpoint-service-permissions \
  --service-id $SERVICE_ID \
  --add-allowed-principals arn:aws:iam::CONSUMER_ACCOUNT_ID:root

# ---- CONSUMER SIDE ----
# Step 3: Create the interface endpoint in the consumer VPC
ENDPOINT_ID=$(aws ec2 create-vpc-endpoint \
  --vpc-id vpc-consumer \
  --service-name com.amazonaws.vpce.us-east-1.$SERVICE_ID \
  --vpc-endpoint-type Interface \
  --subnet-ids subnet-consumer-az-a subnet-consumer-az-b \
  --security-group-ids sg-consumer-endpoint \
  --private-dns-enabled \
  --query VpcEndpoint.VpcEndpointId \
  --output text)

# Step 4: Provider accepts the connection request
aws ec2 accept-vpc-endpoint-connections \
  --service-id $SERVICE_ID \
  --vpc-endpoint-ids $ENDPOINT_ID

PrivateLink for AWS Services (Interface Endpoints)

# Create interface endpoints for private EKS/ECS clusters
# (eliminates NAT Gateway cost for ECR image pulls)

ENDPOINT_SG=$(aws ec2 create-security-group \
  --group-name vpc-endpoints-sg \
  --description "Allow HTTPS from private subnets" \
  --vpc-id $VPC_ID \
  --query GroupId --output text)

aws ec2 authorize-security-group-ingress \
  --group-id $ENDPOINT_SG \
  --protocol tcp --port 443 \
  --cidr 10.0.0.0/16

# ECR API endpoint
aws ec2 create-vpc-endpoint \
  --vpc-id $VPC_ID \
  --service-name com.amazonaws.us-east-1.ecr.api \
  --vpc-endpoint-type Interface \
  --subnet-ids $PRIV_SUBNET_A $PRIV_SUBNET_B $PRIV_SUBNET_C \
  --security-group-ids $ENDPOINT_SG \
  --private-dns-enabled

# ECR Docker Registry endpoint
aws ec2 create-vpc-endpoint \
  --vpc-id $VPC_ID \
  --service-name com.amazonaws.us-east-1.ecr.dkr \
  --vpc-endpoint-type Interface \
  --subnet-ids $PRIV_SUBNET_A $PRIV_SUBNET_B $PRIV_SUBNET_C \
  --security-group-ids $ENDPOINT_SG \
  --private-dns-enabled

# Secrets Manager endpoint
aws ec2 create-vpc-endpoint \
  --vpc-id $VPC_ID \
  --service-name com.amazonaws.us-east-1.secretsmanager \
  --vpc-endpoint-type Interface \
  --subnet-ids $PRIV_SUBNET_A $PRIV_SUBNET_B $PRIV_SUBNET_C \
  --security-group-ids $ENDPOINT_SG \
  --private-dns-enabled

# S3 Gateway endpoint (free — no ENI, just a route)
aws ec2 create-vpc-endpoint \
  --vpc-id $VPC_ID \
  --service-name com.amazonaws.us-east-1.s3 \
  --vpc-endpoint-type Gateway \
  --route-table-ids $PRIV_RT_A $PRIV_RT_B $PRIV_RT_C

Site-to-Site VPN & Client VPN

Site-to-Site VPN

AWS Site-to-Site VPN creates an encrypted IPsec tunnel between your on-premises network and your AWS VPC. Each VPN connection consists of two tunnels for redundancy — always configure both tunnels and enable BGP so that if one tunnel fails, traffic automatically fails over to the second.

# Step 1: Create a Customer Gateway (represents your on-prem VPN device)
CGW_ID=$(aws ec2 create-customer-gateway \
  --type ipsec.1 \
  --public-ip 203.0.113.1 \
  --bgp-asn 65000 \
  --tag-specifications 'ResourceType=customer-gateway,Tags=[{Key=Name,Value=onprem-cgw}]' \
  --query CustomerGateway.CustomerGatewayId \
  --output text)

# Step 2a: Attach to a Transit Gateway (recommended for multi-VPC access)
VPN_TGW=$(aws ec2 create-vpn-connection \
  --type ipsec.1 \
  --customer-gateway-id $CGW_ID \
  --transit-gateway-id $TGW_ID \
  --options "TunnelOptions=[{TunnelInsideCidr=169.254.10.0/30,\
PreSharedKey=CHANGE_ME_USE_STRONG_KEY,DPDTimeoutAction=restart,\
IKEVersions=[{Value=ikev2}]},{TunnelInsideCidr=169.254.10.4/30,\
PreSharedKey=CHANGE_ME_USE_STRONG_KEY_2,DPDTimeoutAction=restart,\
IKEVersions=[{Value=ikev2}]}]" \
  --query VpnConnection.VpnConnectionId \
  --output text)

# Step 2b: OR attach to Virtual Private Gateway (single-VPC access)
VGW_ID=$(aws ec2 create-vpn-gateway \
  --type ipsec.1 \
  --amazon-side-asn 64512 \
  --query VpnGateway.VpnGatewayId \
  --output text)

aws ec2 attach-vpn-gateway \
  --vpc-id $VPC_ID \
  --vpn-gateway-id $VGW_ID

# Enable route propagation so VGW auto-populates VPC route tables
aws ec2 enable-vgw-route-propagation \
  --gateway-id $VGW_ID \
  --route-table-id $PRIV_RT

BGP Configuration Example (Cisco IOS)

! On-premises Cisco router BGP configuration
! AWS Tunnel 1: 169.254.10.1 (AWS side), 169.254.10.2 (your side)

crypto isakmp policy 10
  encryption aes 256
  hash sha256
  authentication pre-share
  group 14
  lifetime 28800

crypto isakmp key CHANGE_ME_USE_STRONG_KEY address 52.x.x.x  ! AWS tunnel 1 public IP

crypto ipsec transform-set AWS-TRANSFORM esp-aes 256 esp-sha256-hmac
  mode tunnel

crypto map AWS-VPN 10 ipsec-isakmp
  set peer 52.x.x.x
  set transform-set AWS-TRANSFORM
  match address AWS-VPN-ACL
  set pfs group14

interface Tunnel0
  ip address 169.254.10.2 255.255.255.252
  tunnel source GigabitEthernet0/0
  tunnel destination 52.x.x.x
  ip tcp adjust-mss 1379

router bgp 65000
  neighbor 169.254.10.1 remote-as 64512
  neighbor 169.254.10.1 timers 10 30
  network 192.168.0.0 mask 255.255.0.0  ! advertise on-prem CIDR to AWS

AWS Client VPN

Client VPN provides OpenVPN-compatible remote access for users connecting to AWS resources. It supports two authentication methods: mutual TLS (certificate-based) and Active Directory (via AWS Managed Microsoft AD or self-managed AD).

# Step 1: Generate server and client certificates (using easy-rsa)
git clone https://github.com/OpenVPN/easy-rsa.git
cd easy-rsa/easyrsa3
./easyrsa init-pki
./easyrsa build-ca nopass
./easyrsa build-server-full server nopass
./easyrsa build-client-full client1.techoral.internal nopass

# Import certificates to ACM
aws acm import-certificate \
  --certificate fileb://pki/issued/server.crt \
  --private-key fileb://pki/private/server.key \
  --certificate-chain fileb://pki/ca.crt \
  --query CertificateArn --output text

# Step 2: Create Client VPN Endpoint
aws ec2 create-client-vpn-endpoint \
  --client-cidr-block 172.16.0.0/22 \
  --server-certificate-arn arn:aws:acm:us-east-1:123456789012:certificate/SERVER_CERT_ID \
  --authentication-options "Type=certificate-authentication,\
MutualAuthentication={ClientRootCertificateChainArn=arn:aws:acm:...:CLIENT_CA_ARN}" \
  --connection-log-options "Enabled=true,CloudwatchLogGroup=/aws/client-vpn,CloudwatchLogStream=connections" \
  --split-tunnel \
  --transport-protocol udp \
  --vpn-port 443 \
  --tag-specifications 'ResourceType=client-vpn-endpoint,Tags=[{Key=Name,Value=techoral-client-vpn}]'

# Step 3: Associate with a VPC subnet
aws ec2 associate-client-vpn-target-network \
  --client-vpn-endpoint-id cvpn-endpoint-xxxxx \
  --subnet-id $PRIV_SUBNET_A

# Step 4: Add authorization rule (allow all users to reach VPC CIDR)
aws ec2 authorize-client-vpn-ingress \
  --client-vpn-endpoint-id cvpn-endpoint-xxxxx \
  --target-network-cidr 10.0.0.0/16 \
  --authorize-all-groups

Direct Connect + Transit Gateway — Hybrid at Line Rate

AWS Direct Connect provides a dedicated private network connection between your on-premises data center and AWS. Unlike VPN, which runs over the public internet and caps out at 1.25 Gbps per tunnel, Direct Connect provides consistent, low-latency connectivity at 1 Gbps to 100 Gbps with an SLA.

Hosted vs Dedicated Connections

Direct Connect Gateway + Transit Gateway Architecture

ON-PREMISES DATA CENTER
  Physical cross-connect at DX Location
        |
  [AWS Direct Connect (1G/10G/100G)]
        |
  [Virtual Interface (Private VIF or Transit VIF)]
        |
  [Direct Connect Gateway (DXGW)]  <-- global, connects to TGW in any region
        |
  [Transit Gateway (TGW)]           <-- regional, routes to all attached VPCs
        |
  +-----+-----+-----+
  |     |     |     |
Prod  Stage  Dev  Shared
VPC   VPC   VPC  Services

# Create Direct Connect Gateway
DXGW_ID=$(aws directconnect create-direct-connect-gateway \
  --direct-connect-gateway-name techoral-dxgw \
  --amazon-side-asn 64512 \
  --query directConnectGateway.directConnectGatewayId \
  --output text)

# Create Transit VIF on your Direct Connect connection
aws directconnect create-transit-virtual-interface \
  --connection-id dxcon-xxxxx \
  --new-transit-virtual-interface "{
    \"virtualInterfaceName\": \"techoral-transit-vif\",
    \"vlan\": 100,
    \"asn\": 65000,
    \"mtu\": 8500,
    \"authKey\": \"BGP_MD5_AUTH_KEY\",
    \"amazonAddress\": \"169.254.100.1/30\",
    \"customerAddress\": \"169.254.100.2/30\",
    \"addressFamily\": \"ipv4\",
    \"directConnectGatewayId\": \"$DXGW_ID\"
  }"

# Associate DXGW with Transit Gateway
aws directconnect create-direct-connect-gateway-association \
  --direct-connect-gateway-id $DXGW_ID \
  --gateway-id $TGW_ID \
  --add-allowed-prefixes-to-direct-connect-gateway \
    '[{"cidr":"10.0.0.0/16"},{"cidr":"10.1.0.0/16"},{"cidr":"10.2.0.0/16"}]'

Failover Architecture: Direct Connect + VPN Backup

RECOMMENDED HYBRID FAILOVER SETUP:

Primary path:  On-premises --[DX 1G]--> DXGW --> TGW  (fast, reliable)
Backup path:   On-premises --[VPN]---> TGW              (auto-failover via BGP)

BGP Priority Configuration:
- DX advertises routes with higher LOCAL_PREF (primary)
- VPN advertises same routes with lower LOCAL_PREF (backup)
- When DX fails, BGP withdraws those routes; VPN routes take over
- Failover typically <60 seconds with BGP BFD enabled on DX

On-premises router BGP:
  DX peer (169.254.100.1): route-map SET-HIGH-PREF in  -- set local-preference 200
  VPN peer (169.254.10.1):  route-map SET-LOW-PREF in   -- set local-preference 100

AWS Resource Access Manager — Cross-Account Sharing

AWS Resource Access Manager (RAM) lets you share AWS resources across accounts within your AWS Organization without complex cross-account IAM role chains. For networking, two resources are commonly shared: Transit Gateway attachments and VPC subnets.

Sharing Transit Gateway with RAM

# Share Transit Gateway with entire AWS Organization
aws ram create-resource-share \
  --name tgw-org-share \
  --resource-arns arn:aws:ec2:us-east-1:NETWORK_ACCOUNT_ID:transit-gateway/$TGW_ID \
  --principals arn:aws:organizations::MANAGEMENT_ACCOUNT_ID:organization/o-xxxxx \
  --permission-arns arn:aws:ram::aws:permission/AWSRAMDefaultPermissionTransitGateway \
  --allow-external-principals false

# From a spoke account — create attachment to the shared TGW
aws ec2 create-transit-gateway-vpc-attachment \
  --transit-gateway-id $TGW_ID \      # same TGW ID, different account
  --vpc-id vpc-spoke-account \
  --subnet-ids subnet-spoke-az-a subnet-spoke-az-b \
  --tag-specifications 'ResourceType=transit-gateway-attachment,Tags=[{Key=Name,Value=spoke-attach}]'

# Network account must accept the attachment (if auto-accept is disabled)
aws ec2 accept-transit-gateway-vpc-attachment \
  --transit-gateway-attachment-id tgw-attach-xxxxx

Sharing Subnets with RAM (Shared VPC)

# Share subnets from a central network account to application accounts
# Application accounts deploy resources into shared subnets
aws ram create-resource-share \
  --name shared-vpc-subnets \
  --resource-arns \
    arn:aws:ec2:us-east-1:NETWORK_ACCOUNT:subnet/$PRIV_SUBNET_A \
    arn:aws:ec2:us-east-1:NETWORK_ACCOUNT:subnet/$PRIV_SUBNET_B \
  --principals \
    arn:aws:organizations::MGMT:ou/o-xxx/ou-xxx-appteam \
  --permission-arns arn:aws:ram::aws:permission/AWSRAMDefaultPermissionSubnet

Decision Matrix: Which Option to Use When

Use this matrix in your architecture review. Map your requirements to the right tool before you start building.

Cost Comparison for 10 VPCs with Full Mesh Connectivity

SCENARIO: 10 VPCs, each needing to communicate with all others
  - 1TB data transferred between VPCs per month
  - us-east-1

VPC PEERING APPROACH:
  Connections needed: 10*(10-1)/2 = 45 peering connections
  Hourly cost: $0 (peering is free)
  Data transfer (cross-AZ): 1TB × $0.01/GB = $10.24/month
  Operational overhead: Managing 45 route table entries × 10 VPCs = 450 routes
  Total: ~$10/month + significant operational cost

TRANSIT GATEWAY APPROACH:
  Connections needed: 10 VPC attachments
  Attachment fee: 10 × $0.05 × 730 hrs = $365/month
  Data processing: 1TB × $0.02/GB = $20.48/month
  Operational overhead: 1 TGW route table (or segmented tables)
  Total: ~$385/month — dramatically simpler operations

VERDICT: For 2-4 VPCs with infrequent changes, peering wins on cost.
For 5+ VPCs, the operational savings of TGW far outweigh the $365/month.
At 50 VPCs, the peering complexity (1,225 connections) makes TGW
essentially mandatory regardless of cost.

Real-World Multi-Account Architecture

Here is the architecture used in a typical enterprise AWS landing zone with 20–50 accounts. It combines all the connectivity options discussed in this article into a coherent, secure, and cost-effective design.

TECHORAL ENTERPRISE AWS NETWORK ARCHITECTURE
=============================================

AWS ORGANIZATION
├── Management Account (billing, org-level SCPs)
│
├── Network Account  [CENTRALIZED NETWORKING]
│   ├── Transit Gateway (TGW)  -- hub for all VPCs
│   ├── Egress VPC (10.255.0.0/16)
│   │   ├── NAT Gateways (centralized outbound internet)
│   │   └── AWS Network Firewall (IDS/IPS for all egress)
│   ├── Ingress VPC (10.254.0.0/16)
│   │   └── ALB + WAF for inbound public traffic
│   └── Direct Connect (1G hosted connection to on-prem DC)
│       └── DXGW → TGW
│
├── Shared Services Account  [PLATFORM SERVICES]
│   └── Shared Services VPC (10.3.0.0/16)
│       ├── Active Directory (for Client VPN auth)
│       ├── Internal PKI / Certificate Authority
│       ├── Centralized logging (OpenSearch + S3)
│       └── PrivateLink Endpoint Services (exposed to all spokes)
│           ├── Logging service  → NLB → PrivateLink
│           └── Secrets service  → NLB → PrivateLink
│
├── Production Account
│   └── Prod VPC (10.0.0.0/16)
│       ├── Attached to TGW (prod route table only)
│       ├── Interface Endpoints: ECR, SSM, Secrets Manager
│       └── Gateway Endpoint: S3
│
├── Staging Account
│   └── Staging VPC (10.1.0.0/16) -- isolated TGW route table
│
├── Dev Account
│   └── Dev VPC (10.2.0.0/16) -- isolated TGW route table
│
└── Security Account  [AUDIT & VISIBILITY]
    └── Guard Duty, SecurityHub, CloudTrail aggregation

DATA FLOWS:
  Spoke VPC → TGW → Egress VPC → NAT GW → Internet
  On-prem → DX → DXGW → TGW → Prod/Shared VPCs
  Remote user → Client VPN → Shared Services VPC → TGW → Resources
  Spoke VPC → PrivateLink ENI → Shared Services (no TGW hop for services)

Terraform for the Core Network Account

# network-account/main.tf

module "transit_gateway" {
  source = "./modules/transit-gateway"

  name            = "techoral-tgw"
  amazon_side_asn = 64512

  # One route table per environment (isolation)
  route_tables = {
    production    = { name = "prod-rt" }
    nonproduction = { name = "nonprod-rt" }
    shared        = { name = "shared-rt" }
    egress        = { name = "egress-rt" }
  }
}

module "egress_vpc" {
  source = "./modules/egress-vpc"

  vpc_cidr            = "10.255.0.0/16"
  transit_gateway_id  = module.transit_gateway.id
  tgw_route_table_id  = module.transit_gateway.route_tables["egress"].id

  # Centralized NAT Gateways (one per AZ)
  nat_gateway_count   = 3

  # AWS Network Firewall
  enable_network_firewall = true
  firewall_policy_arn     = aws_networkfirewall_firewall_policy.egress.arn
}

# Share TGW with entire organization
resource "aws_ram_resource_share" "tgw_share" {
  name                      = "tgw-org-share"
  allow_external_principals = false

  tags = { Name = "tgw-org-share" }
}

resource "aws_ram_resource_association" "tgw" {
  resource_arn       = module.transit_gateway.arn
  resource_share_arn = aws_ram_resource_share.tgw_share.arn
}

resource "aws_ram_principal_association" "org" {
  principal          = var.organization_arn
  resource_share_arn = aws_ram_resource_share.tgw_share.arn
}

# Propagate routes: shared services routes go into all route tables
resource "aws_ec2_transit_gateway_route_table_propagation" "shared_to_prod" {
  transit_gateway_attachment_id  = module.shared_services_attachment.id
  transit_gateway_route_table_id = module.transit_gateway.route_tables["production"].id
}

# Default route in prod/nonprod tables → egress VPC (for internet access)
resource "aws_ec2_transit_gateway_route" "default_to_egress" {
  for_each = {
    prod    = module.transit_gateway.route_tables["production"].id
    nonprod = module.transit_gateway.route_tables["nonproduction"].id
  }

  destination_cidr_block         = "0.0.0.0/0"
  transit_gateway_attachment_id  = module.egress_attachment.id
  transit_gateway_route_table_id = each.value
}

Frequently Asked Questions

Can I combine VPC Peering and Transit Gateway?

Yes. A common pattern is to use TGW for the main hub-and-spoke topology and VPC Peering for specific low-latency, high-bandwidth connections where the TGW processing fee would be significant. For example, a database VPC that serves one application VPC might use peering instead of TGW to avoid the $0.02/GB processing fee on heavy database traffic.

Does VPC Peering work across AWS Organizations?

Yes. VPC Peering works across accounts (including accounts in different organizations) and across regions. The accepter simply accepts the peering connection request. The only hard requirements are non-overlapping CIDRs and that both VPCs are in supported regions. For cross-organization sharing, you must specify the peer account ID and optionally the peer region when creating the connection.

What is the maximum number of Transit Gateway attachments?

Each Transit Gateway supports up to 5,000 attachments by default. Each attachment can be a VPC, a VPN connection, a Direct Connect Gateway, a TGW Connect (for SD-WAN), or a TGW peering connection with another TGW. Route tables support up to 10,000 routes. These limits are generally sufficient for even very large enterprises — AWS has customers running thousands of VPCs through a single TGW.

Can PrivateLink endpoints be accessed from on-premises?

Yes, if you have Direct Connect or VPN connectivity. When a consumer creates an Interface Endpoint, it gets a private IP in the consumer's subnet. That IP is reachable from on-premises via the Direct Connect/VPN path to that VPC. Enable private DNS on the endpoint, and on-premises clients can resolve the AWS service hostname to the private endpoint IP — making the service appear local. This is the recommended pattern for accessing AWS services (SSM, Secrets Manager, etc.) from on-premises without internet access.

How do I monitor VPC connectivity and troubleshoot routing?

Use VPC Reachability Analyzer (aws ec2 start-network-insights-analysis) to trace the network path between any two endpoints and identify configuration gaps. Use VPC Flow Logs to capture accepted/rejected traffic. For Transit Gateway, enable Transit Gateway Flow Logs to see which attachments and routes are handling traffic. CloudWatch metrics for TGW show BytesIn/BytesOut, PacketsIn/PacketsOut, and PacketDropCountBlackhole (routes with no destination — always investigate these).

Is Direct Connect always better than VPN?

Not always. VPN is faster to provision (hours vs weeks/months for DX), cheaper for low-bandwidth use cases, and already encrypted. Direct Connect provides consistent sub-10ms latency, dedicated bandwidth with no jitter, and is more cost-effective at high data volumes (DX data transfer rates are lower than VPN internet egress rates). Use VPN for dev/test, disaster recovery, and initial connectivity while DX is being provisioned. Use Direct Connect as the primary path for production workloads requiring consistent performance.