Why use etcd or ZooKeeper for leader election?

Category: System Design

Answer

A leader election protocol needs to elect exactly one leader even across network partitions. Doing it yourself in DB rows is fragile (split brain, two leaders, lost lock on failure). etcd/ZK provide distributed locks with TTL + health-check. Most schedulers (Kubernetes, Nomad) build on this primitive.

Concrete examples from the fca project context

Example 1

Lease with TTL: whoever holds the lease is the leader.

Example 2

Watch event: clients wake up when the leader changes.

Example 3

Etcd/Consul/ZK for the primitive; bespoke implementations are risky.

Data flow / flow chart

candidates attempt lease
  one wins -> leader
  heartbeat keeps lease alive
  on heartbeat failure -> lease expires -> other candidate wins

Takeaway

Use a distributed primitive (etcd) for leader election. Don’t invent it on top of row locks.