Chapter 5: Replication

(Enough detail for real work, but still short enough to skim before an interview)

High availability – keep serving data while a machine or an entire datacenter is down.
Read scaling – let many machines answer read-only queries so one node is not a bottleneck.
Geo-latency – put a copy close to every user so round-trip time is low.
Disconnected work – keep accepting writes even when the WAN or a rack switch breaks.

Every replication scheme is a compromise between those goals, data-loss risk, and operational complexity.

Concept	What it really means	Design consequences
Leader / follower roles	Exactly one node accepts all writes. Followers pull the leader’s log and replay it.	Simple mental model; no conflicts to resolve. All write throughput is limited by that one leader.
Synchronous replication	Leader waits for an ACK from at least one follower before confirming the write to the client.	No data loss if the leader dies, but a slow follower stalls every write.
Asynchronous replication	Leader returns as soon as it has written locally.	Fast and tolerant of slow followers, but the last few acknowledged writes disappear if the leader crashes.
Semi-sync (“2-safe”)	Leader waits until one follower has the log in RAM (not necessarily on disk).	<-50 ms extra latency, yet RPO = 0 (bytes aren’t lost). Good middle ground.
Catch-up recovery	A lagging follower asks the leader for log entries starting at its last LSN/binlog position.	Adding or rebooting a follower requires no cluster downtime.
Fail-over	1️⃣ Detect leader timeout 2️⃣ Elect best follower 3️⃣ Fence the old leader (STONITH) and redirect clients.	Wrong timeout ⇒ either long outage (too high) or split-brain (too low).

Log formats you should be able to name

Statement (old MySQL) – replays SQL text; breaks on NOW() or triggers.
WAL shipping (Postgres, Oracle) – streams raw disk pages; fast but version-coupled.
Logical / row-based – emits which row changed; storage-agnostic, perfect for Kafka CDC.
Trigger-based – custom tables capture changes; flexible but slows writes.

Replication lag introduces three user-visible anomalies

Read-after-write gap – user cannot see what they just posted.
Monotonic-read violation – page refresh shows older data.
Causal flip – answer appears before the question (cross-partition).

Mitigations: route recent readers to the leader, sticky sessions, attach “last-seen LSN” token with every request.