System Design: URL Shortener (bit.ly at Scale)

Published data: 15 April 2026

bit.ly processes over
600 million redirects per
day. At that volume, every
architectural decision
has measurable cost.

Every system design interview eventually asks you to build one. A URL shortener seems trivial — until you think about billions of redirects, collision-free code generation across distributed servers, and sub-10ms latency requirements.

This post works through the problem from first principles: a naïve single-server implementation all the way to a production-grade distributed system. Each level introduces a real failure mode from the previous one.

1. Requirements

Before touching any code, pin down what you’re actually building.

Functional requirements:

• Given a long URL, return a unique short code (e.g. sho.rt/aB3k9z)
• Given a short code, redirect to the original URL
• Support custom aliases (sho.rt/my-company)
• Support optional expiry dates
• Track click analytics per short code

Non-functional requirements:

Back-of-envelope math:

• 100M writes/day ÷ 86,400 sec/day ≈ 1,157 writes/sec
• 10:1 read/write ratio → 11,570 redirects/sec
• Average URL length: 200 bytes. Short code + metadata: ~500 bytes per record
• 100M × 365 × 5 years × 500 bytes ≈ 91 TB total storage over 5 years
• Hot URLs: 20% of URLs drive 80% of traffic → cache the top ~200M entries

2. Level 1 — The Naïve Approach: Random ID

The simplest possible approach: generate a random 6-character string, check if it already exists in the database, store it if not.

function generateRandomCode(length = 6) {
  const chars = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789';
  let code = '';
  for (let i = 0; i < length; i++) {
    code += chars[Math.floor(Math.random() * chars.length)];
  }
  return code;
}

async function shortenUrl(longUrl) {
  let code;
  do {
    code = generateRandomCode();
  } while (await db.exists(code));  // retry on collision

  await db.insert({ code, url: longUrl, createdAt: Date.now() });
  return 'https://sho.rt/' + code;
}

With 6 characters from a 62-character alphabet, you have 62⁶ = 56.8 billion possible codes. Sounds safe. But here’s the trap.

3. Level 2 — Hash + Truncate

Instead of pure randomness, hash the input URL. Same input always produces the same output (useful for deduplication), and hashes are well-distributed.

const crypto = require('crypto');

function hashUrl(longUrl) {
  // MD5 produces 128 bits = 32 hex chars. Take first 7.
  const hash = crypto.createHash('md5').update(longUrl).digest('hex');
  return hash.slice(0, 7);
}

// "https://example.com/very-long-path"
// → md5 → "a3f8b2c..."
// → take 7 → "a3f8b2c"

4. Level 3 — Base62 Encoding

Base62 with 7 characters
gives you 3.5 trillion
unique codes. At 100M/day
that’s 95 years of URLs
before you run out.

The insight: use a monotonically increasing integer counter as the primary key, then encode that integer in base62. No collision is possible — each integer is unique by definition.

Base62 alphabet: 0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ

62⁷ = 3,521,614,606,208 — over 3.5 trillion unique codes from 7 characters.

ALPHABET = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"
BASE     = len(ALPHABET)  # 62

def to_base62(n: int) -> str:
    if n == 0:
        return ALPHABET[0]
    result = []
    while n > 0:
        result.append(ALPHABET[n % BASE])
        n //= BASE
    return ''.join(reversed(result))

def from_base62(s: str) -> int:
    result = 0
    for char in s:
        result = result * BASE + ALPHABET.index(char)
    return result

# ID 1          → "1"
# ID 62         → "a0"
# ID 1_000_000  → "4c92"
# ID 3.5T       → 7-char code

Interactive Base62 Encoder

5. Level 4 — The Distributed Counter Problem

The counter pre-allocation
trick is elegant: each app
server grabs a range of
1,000 IDs at startup.
Only talk to the counter
service once per 1,000 URLs.

A single auto-increment database counter is a single point of failure and a write bottleneck. Three API servers competing for the same sequence creates lock contention. Solutions:

Strategy A: Range-Based Allocation

Each server claims a pre-allocated range from a central coordinator.

# Counter service (runs once on startup per app server)
class RangeAllocator:
    RANGE_SIZE = 1_000

    def __init__(self, redis_client):
        self.redis = redis_client
        self.current = 0
        self.max_id  = -1

    def next_id(self) -> int:
        if self.current > self.max_id:
            # Grab the next range atomically from Redis
            end = self.redis.incrby("global_counter", self.RANGE_SIZE)
            self.current = end - self.RANGE_SIZE
            self.max_id  = end - 1

        self.current += 1
        return self.current

Why this works: Each server only contacts Redis once every 1,000 URL creations. Redis INCRBY is atomic — no two servers get overlapping ranges. If a server crashes, at most 1,000 IDs are wasted (gaps in the sequence are fine for URL shorteners).

Strategy B: Redis INCR

Simpler: just call Redis INCR url_counter for every URL. Redis single-threaded operations are atomic. At 1,157 writes/sec this is well within Redis capacity (~100K ops/sec). Add Redis Sentinel for HA.

Strategy C: Snowflake-style ID

A 64-bit ID composed of timestamp + datacenter ID + machine ID + sequence. Twitter/Sonyflake approach. No coordination needed between machines.

// Snowflake ID layout (64 bits)
// [41 bits timestamp ms] [5 bits datacenter] [5 bits machine] [12 bits sequence]

type Snowflake struct {
    datacenterID int64
    machineID    int64
    sequence     int64
    lastTS       int64
    mu           sync.Mutex
}

func (s *Snowflake) Next() int64 {
    s.mu.Lock()
    defer s.mu.Unlock()

    ts := nowMs()
    if ts == s.lastTS {
        s.sequence = (s.sequence + 1) & 0xFFF
        if s.sequence == 0 { ts = waitNextMs(ts) }
    } else {
        s.sequence = 0
    }
    s.lastTS = ts

    return (ts << 22) | (s.datacenterID << 17) | (s.machineID << 12) | s.sequence
}

6. Level 5 — Full Architecture

Click any component to learn about its role, failure modes, and scaling strategy.

7. Level 6 — Redirect Optimization

301 vs 302 is a trap question
— most interviewers know
to ask it. 301 breaks click
analytics; 302 adds server
load. The right answer is
“it depends on your SLA.”

The redirect itself is a single HTTP response. The status code choice has major system consequences.

Redis Cache Strategy (80/20 Rule)

20% of your URLs receive 80% of the traffic. Cache the hot 20% in Redis. With 100M active URLs, the hot set is ~20M records. Each entry: 7-byte key + 200-byte URL = ~207 bytes. Total: ~4 GB — fits comfortably in a single Redis instance.

async def redirect(short_code: str):
    # 1. Try Redis first (hot path)
    url = await redis.get("url:" + short_code)

    if not url:
        # 2. Cache miss → query MySQL read replica
        record = await db.query_one(
            "SELECT long_url, expires_at FROM urls WHERE code = %s",
            short_code
        )
        if not record:
            return HTTP_404()

        if record.expires_at and record.expires_at < now():
            return HTTP_410()  # Gone

        url = record.long_url
        # 3. Populate cache with TTL
        await redis.setex("url:" + short_code, 3600, url)

    # 4. Fire-and-forget analytics (non-blocking)
    asyncio.create_task(record_click(short_code))

    return HTTP_302(location=url)

8. Level 7 — Custom Aliases & Expiry

Custom Aliases

sho.rt/my-company requires a uniqueness check before write. Two concerns:

1. Namespace collision: A user wants my-company, but it was already taken (or reserved).
2. Bloom filter optimization: Before hitting the DB, use a Bloom filter to check “definitely not exists” in O(1). Only proceed to DB on probable matches.

async def create_custom(alias: str, long_url: str):
    # Validate format: 3-50 chars, alphanumeric + hyphens only
    if not re.match(r'^[a-zA-Z0-9-]{3,50}$', alias):
        raise ValueError("Invalid alias format")

    # Bloom filter fast-path (no false negatives)
    if bloom.might_contain(alias):
        # Might exist — confirm with DB
        if await db.exists("SELECT 1 FROM urls WHERE code = %s", alias):
            raise ConflictError("Alias already taken")

    # INSERT with unique constraint as safety net
    try:
        await db.insert({
            "code": alias,
            "long_url": long_url,
            "is_custom": True,
        })
        bloom.add(alias)
    except UniqueConstraintViolation:
        raise ConflictError("Race condition: alias taken")

URL Expiry: Two Approaches

9. Analytics Architecture

Tracking every click naïvely destroys your write throughput. Each redirect triggering an INSERT is 11,570 writes/sec to the database — far exceeding comfortable MySQL territory.

Evolution of the analytics pipeline:

# Every redirect does this — disaster at 11,570 req/sec
async def record_click_naive(code, req):
    await db.execute(
        "INSERT INTO clicks (code, ts, ip, referer) VALUES (%s,%s,%s,%s)",
        code, now(), req.ip, req.headers["Referer"]
    )

# On redirect: non-blocking publish to Kafka
async def record_click(code, req):
    await kafka_producer.send(
        topic = "url.clicks",
        value = {
            "code": code, "ts": now(),
            "ip": req.ip, "referer": req.headers.get("Referer"),
        }
    )  # Returns immediately, ~0.1ms

# Separate consumer: batch flush every 5 seconds
class ClickConsumer:
    async def run(self):
        batch = []
        async for msg in kafka_consumer:
            batch.append(msg.value)
            if len(batch) >= 1000 or elapsed() > 5:
                await timescaledb.bulk_insert(batch)
                batch.clear()

The Kafka stream also enables real-time dashboards: a separate consumer aggregates counts per code into Redis sorted sets (ZINCRBY clicks:daily {code} 1), giving you a live top-N leaderboard.

10. Interactive: Short Code Generator

11. Capacity Estimator

12. Interview Checklist

What interviewers are actually checking. Walk through each point deliberately.

• ✓
  Scope the problem first. Ask: custom aliases? expiry? analytics? read-heavy or write-heavy? International users? Clarifying questions signal seniority.
• ✓
  Do envelope math before designing. State: 1,157 writes/sec, 11,570 reads/sec, 91 TB over 5 years. Then justify your component choices against these numbers.
• ✓
  Explain why random IDs fail at scale. Birthday paradox. Collision retries. Show the formula: P ≈ 1 − e^(−n²/2N).
• ✓
  Choose base62 counter encoding. Explain 62⁷ = 3.5T unique codes. Zero collisions. Then immediately address the distributed counter problem.
• ✓
  Address distributed counter explicitly. Name range-based allocation, Redis INCR, or Snowflake. Explain tradeoffs. Range allocation is the easiest to explain clearly.
• ✓
  Distinguish 301 vs 302. This is a trap. 301 caches at browser, breaks analytics. 302 always hits server, enables full tracking. Know which to use and why.
• ✓
  Propose Redis caching for reads. 80/20 rule. ~4 GB for 20M hot URLs. Show you know cache-aside pattern and TTL management.
• ✓
  Decouple analytics writes. Kafka queue + batch flush to time-series DB. Never INSERT per click on the critical path.
• ✓
  Handle expiry two ways. Lazy deletion (check on read, return 410) + background sweep. Explain why both are needed.
• ✓
  Name failure modes. Counter service down → pre-cached ranges. Redis down → fallback to DB. DB primary down → failover to replica (accept brief read-only mode).

The counter pre-allocation
trick is elegant: each app
server grabs a range of
1,000 IDs at startup.
Only talk to the counter
service once per 1,000 URLs.

This is post #2 in the System Design Interview Prep series. The next post covers the design of a distributed rate limiter — another interview staple where the tricky part isn’t the algorithm but the failure semantics of your counter store.