Why does read-replica lag matter for write-then-read?

Category: System Design

Answer

You write to the primary, immediately read from a replica, and get stale data because the replica hasn’t replicated yet. The fix: read-your-writes consistency — after a write, route reads to the primary for some TTL, OR use a synchronous cluster. The bug is rare but it bites at 3 a.m.

Concrete examples from the fca project context

Example 1

“Stickiness” key in cache: after write, store user_id -> “read primary for 5 sec”.

Example 2

Or run on a strongly consistent cluster (spanner, etcd) and pay the latency.

Example 3

Diagnose: compare write timestamp to replica replication lag (Prometheus metric).

Data flow / flow chart

client writes -> primary
  client reads -> replica (stale!)
  fix: stickiness / strong consistency

Takeaway

Read-your-writes is non-negotiable for UX. Default replicas to follow after a TTL, not forever.