Clocks In Distributed Systems

Imagine you’re running a relay race – but every runner has a slightly different wrist-watch, and the track marshal’s stopwatch sometimes jumps backward.

That’s basically what computers face when they try to agree on time across a network.

1. Why time matters at all

Distributed programs constantly ask questions like:

“Has the request timed-out yet?”
“Which write happened first?”
“Do we need to refresh this cache entry?”

To answer those, you need a clock – but there isn’t one single clock. Each machine owns its own little quartz oscillator that:

Ticks at a slightly different speed (called drift)
Might be adjusted by software such as NTP (Network Time Protocol)
Can even jump forward or backward if that adjustment is big

2. Two kinds of clocks you’ll meet

Clock type	What it tells you	Safe use-case	Hidden trap
Time-of-day clock (“wall-clock”, `System.currentTimeMillis()`)	A calendar date & time	Show a timestamp to users	Can go backwards (NTP correction, leap-second, VM resume) – never measure durations with it
Monotonic clock (`System.nanoTime()` / `CLOCK_MONOTONIC`)	An ever-increasing counter with no real-world meaning	Measure elapsed time (timeouts, latency)	Values from two machines are unrelated – can’t order events across nodes

Rule of thumb:

Use monotonic for how long, wall-clock for when.

3. Synchronizing wall-clock time

Programs often run ntpd or chronyd, which: