Skip to main content
data systems mechanics invariants in distributed architectures

The Trouble with Time: Consistency Models

4 min read Chapter 9 of 28
Summary

This section deconstructs the illusion of simultaneity in...

This section deconstructs the illusion of simultaneity in distributed systems by rigorously defining consistency models. It introduces the CAP theorem (the impossibility of simultaneously guaranteeing Consistency, Availability, and Partition tolerance) and its extension PACELC (which adds the Latency vs. Consistency trade-off during normal operation). The core content presents a spectrum of consistency models from strongest (linearizable) to weakest (eventual), explaining their guarantees and trade-offs. Key code artifacts include a pseudocode example demonstrating how a database client might tune consistency levels (Linearizable, Eventual, Session) to navigate the latency-consistency trade-off. The section also provides classification tables that map real-world systems (Cassandra, HBase, DynamoDB, Spanner, CockroachDB) to their CAP/PACELC choices and rationales. The narrative establishes that consistency model selection is a fundamental design decision with immutable trade-offs between consistency, availability, and performance.

The Trouble with Time: Consistency Models

Distributed systems must reconcile the physical reality of networked computation with the logical requirement for coherent data. The assumption of global simultaneity is invalid; clocks drift, messages delay, and failures are inevitable. This creates a fundamental tension: data replicas across nodes cannot always reflect the same state at the same instant. Consistency models resolve this by defining precise invariants about when updates become visible, thereby enforcing predictable behavior despite temporal divergence.

Invariant: Linearizability – The Strongest Guarantee

Linearizability requires that every operation appears to take effect instantaneously between its invocation and response, as if executed on a single copy of the data. This is not a configuration option but a fundamental constraint demanding strict coordination.

Achieving linearizability necessitates mechanisms such as consensus protocols (e.g., Paxos, Raft) or quorum-based synchronization, where read and write operations must contact a majority of replicas to ensure visibility of prior writes. This coordination introduces high latency and reduces availability during network partitions, as progress halts if a quorum cannot be reached.

# Example: Quorum Read/Write for Linearizability (Python-like pseudocode)
from typing import Set

def quorum_read(key: str, replicas: Set[Replica]) -> Value:
    # Must collect responses from strict majority to detect latest version
    responses = [r.read(key) for r in replicas]
    majority = len(replicas) // 2 + 1
    valid_responses = [r for r in responses if r.acknowledged]
    if len(valid_responses) < majority:
        raise UnavailableError("Quorum not reached")
    return max(valid_responses, key=lambda r: r.timestamp)  # Choose most recent


def quorum_write(key: str, value: Value, replicas: Set[Replica]) -> None:
    # Write must persist to majority before acknowledgment
    acks = [r.write(key, value) for r in replicas]
    if sum(1 for a in acks if a.success) < len(replicas) // 2 + 1:
        raise ConsistencyFailure("Write quorum failed")

Invariant: Eventual Consistency – Availability Over Ordering

Eventual consistency guarantees that, in the absence of new updates, all replicas will converge to the same value after a finite period. There is no requirement for immediate agreement, making it highly available and low-latency.

This model operates under the assumption that divergence is expected and temporary. Updates propagate asynchronously, often via anti-entropy protocols or gossip. However, during convergence, clients may observe stale, inconsistent, or conflicting states. Conflict resolution is deferred, typically handled through application logic, vector clocks, or last-write-wins policies.

The tradeoff is explicit: availability and performance are prioritized at the expense of predictable ordering and immediate correctness.

Invariant: Causal Consistency – Preserving Dependency Order

Causal consistency preserves the apparent order of causally related operations. If operation A causally precedes B, all processes observe A before B. Concurrent, unrelated operations may be observed in different orders.

This is enforced by tracking dependencies using logical clocks or version vectors. Each update carries metadata encoding its causal history. Replicas apply updates only when their dependencies are satisfied, ensuring that effects follow causes.

Unlike linearizability, causal consistency does not impose global ordering, allowing greater availability and lower latency while still preventing paradoxical outcomes (e.g., replying to a message before it is sent).

Invariant: Session Consistency – Client-Centric Guarantees

Session consistency provides monotonic reads, read-your-writes, and monotonic writes within a single client session. It ensures that a client never sees time move backward in its own context.

This is achieved by binding a session to a specific replica or maintaining client-side sequence numbers. Reads may be served locally, but the system tracks the client’s latest observed state to enforce ordering. Writes are routed through a coordinator that sequences them relative to prior session activity.

The mechanism balances usability and performance: clients experience coherent behavior without requiring system-wide synchronization.

Synthesis: The Immutable Trade-Offs

The selection of a consistency model is not an implementation detail but a foundational architectural decision governed by immutable tradeoffs:

  • Consistency vs. Availability: During a network partition, the system must choose between blocking operations (CP) or proceeding with potentially inconsistent state (AP). This is not negotiable; it is enforced by the laws of distributed computing.
  • Consistency vs. Latency: Stronger consistency requires coordination, which increases latency due to message round-trips and quorum waits. Weaker models reduce coordination, enabling faster responses at the cost of staleness.
  • Availability vs. Correctness: High availability assumes partial failure is the norm. Systems designed for uptime accept temporary inconsistency, resolving it during recovery rather than preventing it.

These tradeoffs are not mitigated by better engineering—they are inherent. Real-world systems like Spanner (CP, PC/EC) and DynamoDB (AP, PA/EL) do not escape these constraints; they explicitly embrace them based on workload requirements.

Designing distributed systems requires acknowledging that failure, latency, and inconsistency are not exceptions but the default conditions. The role of consistency models is to define what correctness means under those conditions—and what must be sacrificed to achieve it.