Split a massive dataset into smaller pieces spread across servers.

Data Flow (hash-based sharding)

key  ->  hash(key) % N  -> shard_id
user:42 -> hash('user:42') % 4 = 2  -> shard 2

+-------+   +-------+   +-------+   +-------+
| shard0|   | shard1|   | shard2|   | shard3|
+-------+   +-------+   +-------+   +-------+

Strategies

• Hash-based: hash(key) mod N. Even distribution; reshuffles when N changes.
• Range-based: id in [0, 1B) -> shard 0; [1B, 2B) -> shard 1. Good for range queries; bad if data is skewed.
• Directory-based: a lookup service maps keys to shards. Most flexible, most complex.

Pitfalls

• Hot shard: a viral post lands all reads on shard 2. Mitigate with cache + replica reads + virtual buckets.
• Cross-shard joins: SQL JOINs across shards are painful. Denormalise or move to a wide-column store.
• Resharding pain: doubling N reshuffles 50% of keys in hash-based. Use consistent hashing for steady migration.

Analogy

Dividing a phonebook alphabetically across 26 cabinets: range-based gives locality; hash-based by middle name gives even fills.

Interview tip: Always mention hot shard mitigation (replicas, virtual buckets, cache).