A URL shortener is one of the most commonly asked system design interview questions — deceptively simple at first, but with rich depth in hash generation, collision handling, caching strategies, and analytics scale. This walkthrough covers the full design as you'd present it in a 45-minute FAANG-style interview.
short.ly/abc123)short.ly/my-link)Out of scope: user accounts, link management dashboard, A/B testing links, QR code generation (can be added later without architectural change).
# Write traffic: 100M URLs/day ÷ 86,400 sec = ~1,157 writes/sec → round to 1,200 QPS # Read traffic (100:1 ratio): 1,200 × 100 = 120,000 reads/sec → ~120K RPS # Storage (5 years): Each URL record ≈ 500 bytes (URL up to 2048 chars + metadata) 100M URLs/day × 365 × 5 = 182.5B records → NOT realistic, cap at: If we store 1M new URLs/day → 1M × 365 × 5 × 500B ≈ 912 GB ≈ ~1 TB # Short code length — how many unique codes? Base62 = 62 characters (a-z A-Z 0-9) 7 chars = 62^7 = 3.5 trillion unique codes → enough for any foreseeable scale # Cache (top 20% URLs = 80% traffic): Keep top 20% of daily active URLs in Redis If 10M URLs are active, 20% = 2M URLs × 500B = 1 GB → tiny # Bandwidth: Read: 120K RPS × 500B ≈ 60 MB/s → trivial for modern infra Write: 1.2K RPS × 500B ≈ 0.6 MB/s → negligible
Two separate flows — write path (shorten URL) and read path (redirect):
POST /api/v1/urls
Authorization: Bearer <token>
Content-Type: application/json
Body:
{
"original_url": "https://www.example.com/very/long/article/path?with=params",
"custom_alias": "my-article", // optional
"expire_at": "2027-01-01T00:00:00Z" // optional
}
Response 201 Created:
{
"short_url": "https://short.ly/abc1234",
"short_code": "abc1234",
"original_url": "https://www.example.com/...",
"created_at": "2026-06-24T12:00:00Z",
"expire_at": "2027-01-01T00:00:00Z"
}
---
GET /{short_code}
→ 302 Found, Location: https://www.example.com/... // trackable (always hits server)
→ 301 Moved Permanently, Location: ... // browser-cached (faster, can't track)
→ 404 Not Found (code doesn't exist or expired)
→ 410 Gone (deliberately deleted)
---
GET /api/v1/urls/{short_code}/stats
Response 200:
{
"short_code": "abc1234",
"total_clicks": 14231,
"last_24h": 892,
"top_countries": [{"country": "US", "clicks": 8102}, ...],
"top_referrers": [{"referrer": "twitter.com", "clicks": 5234}, ...]
}
---
DELETE /api/v1/urls/{short_code}
→ 204 No Content
This is the most interesting part of URL shortener design. You need to generate unique, short, URL-safe codes. Three main approaches:
| Approach | How It Works | Pros | Cons |
|---|---|---|---|
| MD5 + truncate | MD5(original_url) → take first 7 chars → Base62 encode | Same URL always gets same code (natural dedup) | Collision risk — 7 chars from 16-byte hash collide at ~1B URLs. Retry on collision is complex. |
| Auto-increment ID + Base62 | DB auto-increment → convert integer to Base62 | Zero collisions. Simple. Predictable length. | Sequential — enumerable by attackers. Single DB is bottleneck at high write QPS. |
| Distributed ID (Snowflake) + Base62 | 64-bit Snowflake ID → Base62 encode → ~11 chars (trim to 7 by using smaller ID range) | Zero collisions. Distributed. No DB needed for ID gen. High throughput (~4K IDs/ms/node). | Slightly more infrastructure. Slightly longer codes (8–11 chars). |
| Pre-generated token pool | Batch-generate random Base62 codes. Store unused in a "token pool" table. Assign one per URL request. | Guaranteed unique. Fast assignment. Simple reads. | Pool management complexity. Race condition on concurrent assignment (needs atomic DB pop). |
# Snowflake ID structure (64-bit):
# [1 sign bit][41 timestamp ms][10 datacenter+machine][12 sequence]
# Max: 4096 IDs/ms per machine. 1000 machines → 4M IDs/ms = 4B IDs/sec
# Base62 alphabet (URL-safe, case-sensitive):
CHARS = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
def encode_base62(num: int) -> str:
if num == 0:
return CHARS[0]
result = []
while num:
result.append(CHARS[num % 62])
num //= 62
return ''.join(reversed(result))
# Snowflake ID ≈ 1.7 × 10^18 at max → Base62 → ~10 chars
# For shorter codes: use 40-bit IDs → max 1.1T → Base62 → 7 chars
# 40-bit counter: epoch-based, 5 bits datacenter, 5 bits machine, 12 bits sequence
# → 4096 IDs/ms per machine → enough for our 1200 QPS requirement
encode_base62(1234567890) # → "1LY7VK" (6 chars)
encode_base62(549755813888) # → "9qUDkA" (7 chars — ~550B range)
# Custom aliases compete with generated codes — need to avoid conflicts
# Strategy:
# 1. Store all codes (custom + generated) in same urls table with UNIQUE(short_code)
# 2. Pre-reserve aliases starting with digits for auto-generated codes (safe assumption)
# 3. Validate custom alias: alphanumeric + hyphen only, 3–30 chars, not a reserved word
RESERVED_WORDS = {'api', 'admin', 'health', 'stats', 'login', 'about', 'help'}
def validate_alias(alias: str) -> bool:
if alias in RESERVED_WORDS:
return False
if not re.match(r'^[a-zA-Z0-9\-]{3,30}$', alias):
return False
return True
CREATE TABLE urls ( id BIGINT PRIMARY KEY, -- Snowflake ID short_code VARCHAR(16) NOT NULL UNIQUE, -- "abc1234" or custom alias original_url TEXT NOT NULL, -- up to 2048 chars user_id BIGINT, -- nullable: anonymous users allowed is_custom BOOLEAN DEFAULT FALSE, created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), expire_at TIMESTAMPTZ, -- NULL = never expires is_deleted BOOLEAN DEFAULT FALSE -- soft delete ); -- Heavily used indexes: CREATE INDEX idx_short_code ON urls(short_code); -- redirect lookup (most critical) CREATE INDEX idx_user_id ON urls(user_id) WHERE user_id IS NOT NULL; -- "my links" queries CREATE INDEX idx_expire_at ON urls(expire_at) WHERE expire_at IS NOT NULL AND is_deleted = FALSE; -- expiry cleanup job
-- ClickHouse: columnar store, optimised for aggregate queries over billions of rows
CREATE TABLE click_events (
event_id UInt64,
short_code String,
clicked_at DateTime,
ip_country FixedString(2), -- ISO country code: "US", "IN"
device_type Enum('desktop','mobile','tablet','bot'),
referrer String,
user_agent String
) ENGINE = MergeTree()
PARTITION BY toYYYYMM(clicked_at) -- partition by month for fast range queries
ORDER BY (short_code, clicked_at); -- primary sort key for per-link queries
The redirect path hits Redis first. Cache hit → return in <1ms. Cache miss → hit PostgreSQL read replica → write back to Redis.
# Redis key: "url:{short_code}" → original_url
# TTL: 24h for popular links; no TTL for custom permanent links
# Cache size: 10M hot links × 500B avg = 5GB → one r6g.large Redis node ($0.15/hr)
# Write path — cache-aside with write-through on creation:
def shorten_url(original_url, alias=None):
short_code = alias or encode_base62(generate_snowflake_id())
db.execute("INSERT INTO urls (id, short_code, original_url) VALUES (...)")
redis.setex(f"url:{short_code}", 86400, original_url) # warm cache immediately
return short_code
# Read path — cache-aside:
def redirect(short_code):
original_url = redis.get(f"url:{short_code}")
if original_url:
publish_click_event(short_code) # async — don't await
return redirect_302(original_url)
# Cache miss:
row = db.query("SELECT original_url, expire_at FROM urls WHERE short_code = %s", short_code)
if not row or row.expire_at < now():
return 404
redis.setex(f"url:{short_code}", 86400, row.original_url)
publish_click_event(short_code)
return redirect_302(row.original_url)
# Cache invalidation on delete:
def delete_url(short_code):
db.execute("UPDATE urls SET is_deleted=TRUE WHERE short_code=%s", short_code)
redis.delete(f"url:{short_code}") # immediately evict from cache
# Problem: viral link gets 10K concurrent requests; all miss cache simultaneously
# Solution 1: probabilistic early expiry (re-cache before TTL expires)
def get_with_early_expiry(key, ttl):
value, remaining_ttl = redis.get_with_ttl(key)
if value is None or (remaining_ttl < 60 and random.random() < 0.01):
# 1% chance per request to refresh when <60s remaining
value = db_lookup(key)
redis.setex(key, ttl, value)
return value
# Solution 2: Redis SET NX mutex (only one thread fetches from DB)
def get_with_mutex(key):
value = redis.get(key)
if value: return value
lock = redis.set(f"lock:{key}", 1, nx=True, ex=5) # 5s lock TTL
if lock:
value = db_lookup(key)
redis.setex(key, 86400, value)
redis.delete(f"lock:{key}")
return value
else:
time.sleep(0.05) # wait for winner to populate cache
return redis.get(key)
short_code hash. Gives ~1M ops/sec across the cluster.short_code prefix (first character → 62 shards, but in practice use 8 shards). Snowflake ID generation is already distributed, so no coordination needed.# Cron job runs nightly: # 1. Query urls WHERE expire_at < NOW() AND is_deleted = FALSE # 2. Soft-delete (set is_deleted = TRUE) and evict from Redis # Don't delete rows immediately — keep for analytics history # Hard delete after 30-day grace period # At scale: use DB partition pruning — partition urls table by created_at month # Drop old partitions instead of row-by-row deletes (instant, no I/O spike)
Global Load Balancer (CloudFront / Cloudflare)
↓
US-East-1 (primary write region)
├── URL Service (writes)
├── PostgreSQL Primary
└── Redis Primary
EU-West-1 (read replica region)
├── Redirect Service (reads only)
├── PostgreSQL Read Replica (async replication, <50ms lag acceptable for redirects)
└── Redis (replicated from US-East)
AP-Southeast-1 (read replica region)
└── same as EU-West-1
# Redirect reads go to nearest region — p99 latency <10ms within region
# Writes (URL creation) always go to primary US region — single-region writes avoids conflicts
# Acceptable eventual consistency for redirects: 50ms replication lag is fine
| Failure | Impact | Mitigation |
|---|---|---|
| Redis node failure | Cache misses → all reads hit DB | Redis Sentinel / Cluster auto-failover in <30s. Read replicas absorb the load. |
| PostgreSQL primary failure | Writes fail; reads still work via replica | Automated failover with Patroni/RDS Multi-AZ. RPO <1s, RTO <30s. |
| Snowflake ID generator failure | New URL creation fails (reads unaffected) | Run 3+ ID generator instances; round-robin assignment. Each has unique machine ID. |
| Kafka failure | Click events not recorded | Analytics only — non-critical. Redirect still works. Kafka cluster with 3 brokers + replication factor 3. |
| Entire region failure | Traffic served by other regions | Global LB health checks + failover. Read-only mode in replica regions until primary recovered. |