Wise Hustlers — Digital Product & App Development Studio Logo
Get Consultation
By Wise Hustler Admin•10/26/2026•24 min read

Distributed Locks in Production: Redis Redlock vs PostgreSQL Advisory Locks vs etcd Raft

Distributed Locks in Production: Redis Redlock vs PostgreSQL Advisory Locks vs etcd Raft

# Distributed Locks Redis vs Postgres Advisory vs etcd in Production

Summary: Distributed locks coordinate shared resources across microservices, but selecting between Redis, PostgreSQL advisory locks, and etcd requires balancing linearizability, infrastructure overhead, and failure modes. Redis provides low-latency mutual exclusion but risks split-brain during failover and clock drift. PostgreSQL advisory locks offer zero-infrastructure transactional locking but degrade under connection pooling limits. etcd provides provable linearizable leases via Raft consensus at higher operational complexity. For safety-critical workloads, monotonic fencing tokens are essential to eliminate stale zombie writes.

---

The Anatomy of Distributed Locking: Efficiency vs Correctness

Distributed locking is one of the most deceptively complex coordination primitives in backend systems engineering. Software engineers frequently treat distributed locks as drop-in network replacements for in-memory mutexes like sync.Mutex in Go or pthread_mutex_t in C. This mental model breaks down in production because network-based systems introduce asynchronous execution delays, non-monotonic physical clocks, partial network partitions, and unpredictable process pauses.

When evaluating distributed locks redis vs postgres advisory vs etcd, the initial architectural decision requires categorizing the workload into one of two fundamental operational categories:

1. Locking for Efficiency: The goal is avoiding duplicate expensive computational work. If the lock fails and two nodes execute the critical section simultaneously, the system incurs minor resource overhead without compromising data integrity. Examples include warming a shared cache, processing an idempotent video transcoding task, or calculating non-critical analytics aggregates.

2. Locking for Correctness: The goal is preventing data corruption or inconsistent business state. If two nodes acquire the lock concurrently or execute writes out of order, the system risks split-brain anomalies and inconsistent application state. Examples include balance adjustments in ledger systems, assigning duplicate inventory items in an e-commerce order pipeline, or issuing conflicting hardware actuator commands.

In-memory primitives assume a shared memory space and an authoritative operating system kernel capable of preempting threads cleanly. Distributed systems, by contrast, operate under Fischer-Lynch-Paterson (FLP) constraints and asynchronous network realities. Messages between the lock manager and client nodes can be delayed, duplicated, reordered, or lost entirely.

To determine the appropriate locking mechanism, engineers must inspect how Redis (SET NX and Redlock), PostgreSQL (session and transaction advisory locks), and etcd (Raft-backed leader leases) handle failover, physical time drift, process pauses, and connection limits.

---

Redis: Single-Node SET NX, Redlock, and the Martin Kleppmann Critique

Redis is frequently the default choice for distributed locking due to its omnipresence in modern application stacks and sub-millisecond execution speeds. However, Redis was architected primarily as an in-memory data store with asynchronous replication, creating subtle failure modes when used for distributed mutual exclusion.

Single-Node Locking and Asynchronous Replication Hazards

The standard single-instance Redis locking pattern relies on the atomic SET command with conditional parameters:

SET resource_lock_key "random_client_identifier_uuid" NX PX 30000

This command sets the key only if it does not already exist (NX), with a physical millisecond time-to-live (PX 30000). To prevent an unauthorized client from deleting another worker's lock after an execution delay, release operations require an atomic Lua script:

if redis.call("get", KEYS[1]) == ARGV[1] then
    return redis.call("del", KEYS[1])
else
    return 0
end

While clean on a solitary Redis node, production deployments run Redis in primary-replica configurations managed by Redis Sentinel or Redis Cluster. Redis replication is fundamentally asynchronous:

1. Client A sends SET lock_key client_A NX PX 30000 to the Redis Primary.

2. The Primary writes the key to its internal dictionary and returns OK to Client A.

3. Before the primary streams the write command to the replication buffer, the Primary hardware suffers an immediate power failure or kernel panic.

4. Redis Sentinel detects the heartbeat timeout and promotes a Replica to Primary.

5. Client B attempts to acquire the lock: SET lock_key client_B NX PX 30000.

6. Because the newly promoted Primary never received the write from the old master, the key does not exist. The new Primary grants the lock to Client B.

7. Split-Brain: Client A and Client B now operate inside the protected critical section simultaneously.

If a workload requires mutual exclusion for correctness, standard Redis primary-replica topologies fail to provide strict safety guarantees due to asynchronous replication lag and failover data loss.

The Redlock Algorithm

To bypass single-point-of-failure and asynchronous replication vulnerabilities, Redis creator Salvatore Sanfilippo proposed the Redlock algorithm. Redlock provisions $N$ independent, un-replicated Redis master nodes (typically $N = 5$) hosted on distinct physical infrastructure or failure domains.

To acquire a lock under Redlock:

1. The client captures the current timestamp $T_1$.

2. It attempts to acquire the lock sequentially across all $N$ instances using identical key names and random value tokens, applying an acquisition timeout strictly smaller than the total lock TTL (e.g., a 5–50 ms timeout for a 10,000 ms TTL).

3. The client records the timestamp $T_2$ after polling all nodes.

4. The lock is considered acquired if and only if:

- The client acquired the lock from a strict majority of nodes: $Q \ge \lfloor N/2 \rfloor + 1$ (at least 3 out of 5).

- The elapsed acquisition time $(T_2 - T_1)$ is strictly less than the total lock TTL.

5. The remaining valid lock validity time is computed as:

$$\text{Validity Time} = \text{TTL} - (T_2 - T_1) - \text{Clock Drift}$$

The Martin Kleppmann Critique of Redlock

In 2016, distributed systems researcher Martin Kleppmann published a seminal analysis titled "How to do distributed locking". The redis redlock martin kleppmann critique demonstrated that Redlock fails to provide safety for correctness-critical applications. Kleppmann's analysis centers on two systemic assumptions that do not hold in real-world asynchronous computing environments:

1. Synchronized Physical Clocks: Redlock assumes that the drift across independent Redis nodes bounded by physical wall clocks does not exceed a small delta. In practice, physical hardware clocks drift due to quartz oscillator temperature variations, virtualization hypervisor jitter, and Network Time Protocol (NTP) adjustments. If NTP performs a backward time step or an aggressive slew on a Redis node, keys expire prematurely, violating majority lease assumptions.

2. Process Pauses and Network Delays: A client can experience arbitrary execution pauses at any point after acquiring the majority quorum. Common causes include:

- Garbage collection Stop-the-World (STW) pauses in runtimes like Java, Go, or V8 (Node.js).

- Operating system memory paging and page faults under high memory pressure.

- Hypervisor context switching, CPU throttling, or live virtual machine migrations.

- Dynamic network socket buffering delays.

When a client pauses longer than the computed lock validity time, its lease expires on the Redis instances. Another client acquires the lock. When the stalled client resumes execution, it remains unaware that its lock expired and proceeds to execute destructive writes.

---

Clock Drift, Asynchronous Pauses, and Split-Brain Dual Lock Holders

The failure mode identified by Kleppmann demonstrates why no lock service based on client-side time tracking can guarantee mutual exclusion without server-side verification. The following sequence diagram illustrates this exact race condition:

+----------+             +----------+             +--------------+             +----------------+
| Client 1 |             | Client 2 |             | Redis Cluster|             | Shared Storage |
+----------+             +----------+             +--------------+             +----------------+
     |                        |                          |                             |
     | 1. Acquire Lock (TTL 10s)                         |                             |
     |-------------------------------------------------->|                             |
     | 2. Lock Granted (Validity = 10s)                  |                             |
     |<--------------------------------------------------|                             |
     |                        |                          |                             |
     | *** LONG GC PAUSE ***  |                          |                             |
     | (Process stalls 15s)   |                          |                             |
     |                        |                          |                             |
     |        ~ ~ ~ ~ ~ TIME PASSES (10s) ~ ~ ~ ~ ~      |                             |
     |                        |                          |                             |
     |                        |                          |                             |
     |                        | 3. Acquire Lock (TTL 10s)|                             |
     |                        |------------------------->|                             |
     |                        | 4. Lock Granted          |                             |
     |                        |<-------------------------|                             |
     |                        |                          |                             |
     |                        | 5. Mutate Resource       |                             |
     |                        |------------------------------------------------------->|
     |                        | 6. Resource Updated (v1) |                             |
     |                        |<-------------------------------------------------------|
     |                        |                          |                             |
     | *** RESUMES EXECUTION ***                         |                             |
     | (Unaware of elapsed time)                         |                             |
     |                        |                          |                             |
     | 7. Mutate Resource (STALE WRITE / ZOMBIE WRITE)   |                             |
     |-------------------------------------------------------------------------------->|
     |                                                   |                             |
     | !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! |
     | !!! SPLIT-BRAIN: Stale Client 1 overwrites valid write from Client 2        !!! |
     | !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! |
     |                                                   |                             |

In this failure scenario, both clients were entirely rational from their own perspective:

  • Client 1 verified that it held a valid lease at step 2.
  • Client 2 verified that no other node held the lease at step 3.
  • The failure occurred at the boundary between the lock coordinator and the downstream data storage layer. Because the storage engine possessed no mechanism to validate the authorization epoch of incoming mutations, it accepted the stale zombie write from Client 1.

---

PostgreSQL Advisory Locks: Session vs Transaction-Level Semantics

For engineering teams operating relational databases, PostgreSQL provides application-level distributed locks natively via advisory locks. Advisory locks consume no disk space, generate no write-ahead log (WAL) traffic, and require zero additional infrastructure components.

PostgreSQL allocates advisory locks in the database server's shared memory (LockMethodData). They are indexed either by a single 64-bit signed integer (bigint) or two 32-bit signed integers ((classid, objid)).

-- 64-bit lock identifier
SELECT pg_advisory_lock(8472948291048);

-- Two 32-bit identifiers (e.g., Tenant ID and Resource ID)
SELECT pg_advisory_lock(1001, 45892);

Transaction-Level Advisory Locks (pg_advisory_xact_lock)

Transaction-level advisory locks are bound strictly to the lifetime of the active SQL transaction block:

BEGIN;
SELECT pg_advisory_xact_lock(42);
-- Execute critical transactional business logic
UPDATE accounts SET balance = balance - 100 WHERE id = 10;
COMMIT; -- The advisory lock is automatically and instantly released here

If the client application crashes, network connectivity drops, or the database server encounters an unhandled exception triggering a ROLLBACK, the lock releases automatically. This eliminates abandoned or orphaned locks caused by crashed workers.

Session-Level Advisory Locks (pg_advisory_lock)

Session-level advisory locks outlive individual transaction blocks and persist across the lifetime of the PostgreSQL backend process connection:

-- Client opens connection
SELECT pg_advisory_lock(42);
-- Transaction 1
BEGIN; UPDATE ledger SET state = 'PROCESSING' WHERE id = 10; COMMIT;
-- Long external API call...
-- Transaction 2
BEGIN; UPDATE ledger SET state = 'COMPLETED' WHERE id = 10; COMMIT;
-- Explicit release required
SELECT pg_advisory_unlock(42);

Postgres Advisory Locks Connection Pooling Limits

While transaction-level advisory locks are elegant, understanding postgres advisory locks connection pooling limits is essential when operating alongside proxy connection poolers such as PgBouncer, Supavisor, or Odyssey.

Modern high-throughput architectures configure PgBouncer in Transaction Pooling Mode (pool_mode = transaction) to multiplex thousands of microservice clients over a small pool of physical database connections (e.g., 50 backend connections).

This creates three critical production failure modes:

1. Session-Level Lock Leaks in Transaction Pooling: In transaction pooling mode, PgBouncer reassigns the underlying PostgreSQL connection to a completely different application client the instant a COMMIT or ROLLBACK executes. If an engineer executes SELECT pg_advisory_lock(42) inside an application flow, the lock remains bound to the physical PostgreSQL backend process. When PgBouncer reallocates that backend connection to an unrelated HTTP request, that new request unexpectedly inherits the lock, while the original client can no longer release it. This causes silent lock starvation across the entire cluster.

2. Connection Starvation in Session Pooling: To safely use session-level advisory locks, teams must switch connection poolers to Session Pooling Mode (pool_mode = session). However, PostgreSQL uses a dedicated process-per-connection architecture where each connected backend allocates private memory (including query execution memory, catalog caches, and OS process overhead). When hundreds of distributed workers establish dedicated sessions to hold long-running advisory locks, the database connection limit (max_connections) exhausts rapidly, driving up process memory and starving high-priority transactional web queries.

3. Failover Memory Loss: Advisory locks exist exclusively in PostgreSQL volatile shared memory. They are not written to the WAL stream. If the PostgreSQL primary instance experiences a hardware failover (managed by Patroni, AWS RDS Multi-AZ, or CloudNativePG), the newly promoted standby starts with an empty advisory lock table. All active distributed locks evaporate instantly.

---

etcd: Raft Consensus, Leader Leases, and Linearizable Locking

Where Redis prioritizes throughput and PostgreSQL optimizes for collocated relational operations, etcd is engineered from first principles for distributed consistency. As the backing control plane datastore for Kubernetes, etcd guarantees linearizable reads and writes via the Raft consensus algorithm.

The Mechanics of etcd Distributed Locks

At the foundation of distributed coordination in etcd is the etcd raft consensus leader lease model. etcd models distributed locking through a combination of three core primitives: Leases, Software Transactions (`Txn`), and Revisions.

+------------------------------------------------------------------------------------+
|                                    etcd Cluster                                    |
|                                                                                    |
|  +--------------------+    Raft Log Replication    +----------------------------+  |
|  |  Raft Leader Node  |===========================>|  Raft Follower Nodes (x2)  |  |
|  +--------------------+                            +----------------------------+  |
|            |                                                                       |
|            | 1. Lease Creation: TTL = 5s (Cluster-wide heartbeat validation)       |
|            | 2. Key Revision Generation: Monotonically increasing 64-bit integer   |
|            v                                                                       |
|  +------------------------------------------------------------------------------+  |
|  | Key Space: /locks/resource_name/                                             |  |
|  |  - key_id_101  (create_revision: 5041)  <--- Current Lock Holder             |  |
|  |  - key_id_102  (create_revision: 5042)  <--- Watches key_id_101              |  |
|  |  - key_id_103  (create_revision: 5043)  <--- Watches key_id_102              |  |
|  +------------------------------------------------------------------------------+  |
+------------------------------------------------------------------------------------+

1. Lease Allocation: The client requests a Lease with an explicit time-to-live (e.g., 5 seconds). The client maintains a background gRPC bidirectional streaming connection (LeaseKeepAlive) that sends periodic heartbeats to the Raft leader.

2. Ephemeral Sequential Keys: The client executes an atomic transaction creating an ephemeral key under a common prefix, binding it to the lease:

   Key:   /locks/order_processing/c89a-412e-98fe
   Value: client_metadata
   Lease: 0x694d82b014

3. Deterministic Revision Assignment: etcd increments an internal cluster-wide 64-bit monotonic counter called the revision on every state change. When the key is committed to the Raft log, it receives an immutable create_revision.

4. Acquisition Verification: The client queries all keys under prefix /locks/order_processing/. If the client's create_revision is the lowest among all active keys in the prefix, it holds the lock.

Thundering Herd Elimination via the Watch API

In Redis, competing clients typically poll the key using spinlocks or sleep-retry loops (while !acquire() { sleep(50ms) }), generating network amplification and CPU spikes.

etcd eliminates the thundering herd problem entirely using its Watch API:

  • Competing clients do not watch the primary lock prefix.
  • Each waiting client queries the prefix, sorts the keys by create_revision, and sets a gRPC Watch event only on the key immediately preceding its own revision.
  • When Client 101 finishes and deletes its key (or its lease expires due to node death), etcd notifies Client 102 directly via streaming gRPC. Client 103 receives no event and remains dormant.
  • This guarantees strictly ordered, fair FIFO lock acquisition with zero polling overhead.

Trade-Offs of the Raft Approach

etcd provides linearizable consistency, but Raft consensus introduces distinct trade-offs:

  • Disk I/O and Network Quorum Latency: Every lock acquisition and release commits a Raft log entry that must be synchronously replicated and fsynced to disk on a majority quorum ($Q = \lfloor N/2 \rfloor + 1$). Consequently, write latency is bounded by the slowest responding member of the quorum (encompassing network round-trip time and disk Write-Ahead Log fsync), contrasting sharply with Redis's purely in-memory state modifications.
  • Cluster Operational Burden: etcd requires dedicated infrastructure (minimum 3 nodes for single-fault tolerance, 5 nodes for two-fault tolerance) with low-latency, stable network peering to prevent spurious Raft leader re-elections and heartbeating timeouts.

---

Distributed Locks: Redis vs Postgres Advisory vs etcd Architectural Decision Matrix

The following decision matrix evaluates the trade-offs across all three locking systems across seven architectural dimensions:

Architectural DimensionRedis (SET NX / Redlock)PostgreSQL Advisory Locksetcd (Raft Leases)
1. Consensus & LinearizabilityNone (Single node / Async replication); Redlock attempts heuristic quorum without true state-machine consensus.Single-node ACID serializability; no consensus across independent database clusters.Strict Raft consensus; provable linearizable reads and writes across cluster quorum.
2. Process Pause & GC VulnerabilityCritical; TTL expiration during client pause causes dual-lock holder split-brain.Safe for transaction locks (xact); session locks risk stale execution unless connection drops.Controlled via lease TTL; keep-alive pauses drop lease, but monotonic revision provides fencing.
3. Connection & Infrastructure OverheadMinimal; lightweight TCP sockets to an in-memory event loop; multiplexes high connection volumes with low per-client memory overhead.High; process-per-connection architecture where session locks saturate max_connections; incompatible with transaction-mode pooling.Moderate; multiplexed HTTP/2 gRPC streams with client-side keep-alives; requires managing a dedicated 3- or 5-node cluster.
4. Failover & Data Loss CharacteristicsHigh; asynchronous replica promotion drops locks. Redlock risks restart synchronization loss.High; advisory locks reside in shared memory and are completely dropped on replica failover.Zero; state replicated synchronously to Raft quorum WAL before client acknowledgement.
5. Throughput & Latency ProfileSub-millisecond network latency; throughput bounded by single-threaded in-memory execution and network RTT; zero disk I/O in the lock acquisition path.Low single-digit millisecond latency; operates entirely in shared memory (LockMethodData) without WAL generation, but throughput is bounded by connection pool limits and lock manager contention.Consensus-bounded latency; throughput constrained by Raft log replication and synchronous disk fsync across a majority quorum, scaling with storage write speed.
6. Liveness & Expiration MechanicsPassive TTL eviction via wall-clock millisecond timestamps.Active database connection drop (session) or SQL transaction termination (xact).Active lease keep-alive heartbeats over gRPC stream with server-side lease eviction.
7. Fencing Token EnforceabilityManual; application must separately manage an atomic incrementing generation counter.Native; easily paired with database sequences (nextval('lock_seq')) inside transactions.Native; the etcd 64-bit create_revision serves as a natural monotonic fencing token.

---

Distributed Lock Fencing Tokens Implementation: Eliminating Zombie Writes

As Martin Kleppmann demonstrated, no distributed lock manager can guarantee safety on its own. In an asynchronous network, a client holding a lock can always be delayed by an execution pause, wake up after its lock has expired, and issue a mutation to downstream storage.

The established, mathematically sound defense against stale zombie writes is Fencing Tokens.

A fencing token is a strictly monotonically increasing number (such as an etcd revision or database sequence) issued by the lock service alongside the lock grant. The downstream storage engine (PostgreSQL, Cassandra, DynamoDB, or S3) must reject any write accompanied by a fencing token lower than the highest token it has already processed.

Production-Grade TypeScript / Node.js Implementation

The following complete, runnable TypeScript implementation demonstrates a distributed lock manager paired with a fenced storage repository. It proves how a storage engine detects and rejects stale writes from a zombie client that experienced an unexpected execution delay.

/**
 * Distributed Lock Fencing Token Engine
 * Demonstrates monotonic fencing token enforcement against zombie client writes.
 */

interface LockGrant {
  resourceId: string;
  fencingToken: bigint;
  leaseExpiresAt: number;
  release(): Promise<void>;
}

interface StorageRecord<T> {
  data: T;
  lastFencingToken: bigint;
  updatedBy: string;
}

// Custom error thrown when a stale worker attempts a mutation
class StaleFencingTokenException extends Error {
  constructor(attemptedToken: bigint, currentToken: bigint, workerId: string) {
    super(
      `[SECURITY REJECTION] Stale write from worker '${workerId}'. ` +
      `Attempted token: ${attemptedToken}, but storage is already at token: ${currentToken}.`
    );
    this.name = "StaleFencingTokenException";
  }
}

/**
 * Coordination Service representing a linearizable lock coordinator.
 * Maintains an atomic monotonic generation counter.
 */
class LinearizableLockCoordinator {
  private currentEpoch: bigint = 0n;
  private activeLocks: Map<string, { token: bigint; owner: string; expiresAt: number }> = new Map();

  async acquireLock(resourceId: string, ownerId: string, ttlMs: number): Promise<LockGrant> {
    const now = Date.now();
    const existing = this.activeLocks.get(resourceId);

    if (existing && existing.expiresAt > now) {
      throw new Error(`Lock on resource '${resourceId}' is currently held by '${existing.owner}'.`);
    }

    // Monotonically increment cluster epoch counter (analogous to etcd's create_revision)
    this.currentEpoch += 1n;
    const token = this.currentEpoch;
    const expiresAt = now + ttlMs;

    this.activeLocks.set(resourceId, { token, owner: ownerId, expiresAt });

    return {
      resourceId,
      fencingToken: token,
      leaseExpiresAt: expiresAt,
      release: async () => {
        const current = this.activeLocks.get(resourceId);
        if (current && current.token === token) {
          this.activeLocks.delete(resourceId);
        }
      },
    };
  }
}

/**
 * Shared Storage Engine enforcing optimistic fencing token validation.
 */
class FencedResourceRepository<T> {
  private storage: Map<string, StorageRecord<T>> = new Map();

  /**
   * Commits a state update ONLY if the provided fencing token is strictly
   * greater than the last committed fencing token for the resource.
   */
  async updateResourceWithFence(
    resourceId: string,
    newData: T,
    fencingToken: bigint,
    workerId: string
  ): Promise<void> {
    const current = this.storage.get(resourceId);

    if (current && fencingToken <= current.lastFencingToken) {
      // Reject stale write from zombie worker
      throw new StaleFencingTokenException(fencingToken, current.lastFencingToken, workerId);
    }

    // Write accepted
    this.storage.set(resourceId, {
      data: newData,
      lastFencingToken: fencingToken,
      updatedBy: workerId,
    });

    console.log(
      `[STORAGE SUCCESS] Resource '${resourceId}' successfully updated by '${workerId}' ` +
      `with Fencing Token ${fencingToken}.`
    );
  }

  getRecord(resourceId: string): StorageRecord<T> | undefined {
    return this.storage.get(resourceId);
  }
}

/**
 * End-to-end execution demonstrating race conditions and zombie protection.
 */
async function runDistributedFencingSimulation() {
  console.log("=== STARTING DISTRIBUTED LOCK FENCING SIMULATION ===\n");

  const coordinator = new LinearizableLockCoordinator();
  const repository = new FencedResourceRepository<{ balance: number }>();
  const resourceKey = "account:us-east:9021";

  // 1. Worker 1 acquires lock with 300ms lease
  console.log("[Worker 1] Requesting lock for 300ms...");
  const lockWorker1 = await coordinator.acquireLock(resourceKey, "Worker-1", 300);
  console.log(`[Worker 1] Lock granted! Fencing Token: ${lockWorker1.fencingToken}`);

  // 2. Worker 1 enters an unexpected Garbage Collection / VM pause (500ms)
  console.log("[Worker 1] !!! Encountering 500ms Stop-The-World GC Pause !!!");
  
  // Simulation of asynchronous pause
  const worker1Execution = new Promise<void>((resolve) => {
    setTimeout(async () => {
      console.log("\n[Worker 1] Resuming after GC pause (unaware that lock expired)...");
      try {
        await repository.updateResourceWithFence(
          resourceKey,
          { balance: 1500 },
          lockWorker1.fencingToken,
          "Worker-1"
        );
      } catch (err: any) {
        console.log(`[Worker 1 Error Handled] ${err.message}`);
      }
      resolve();
    }, 500);
  });

  // 3. While Worker 1 is paused, its lease expires at 300ms.
  // Worker 2 acquires the lock at 350ms.
  await new Promise((r) => setTimeout(r, 350));
  console.log("\n[Worker 2] Requesting lock after Worker 1 TTL expiration...");
  const lockWorker2 = await coordinator.acquireLock(resourceKey, "Worker-2", 1000);
  console.log(`[Worker 2] Lock granted! Fencing Token: ${lockWorker2.fencingToken}`);

  // 4. Worker 2 performs its mutation immediately and successfully
  await repository.updateResourceWithFence(
    resourceKey,
    { balance: 2500 },
    lockWorker2.fencingToken,
    "Worker-2"
  );
  await lockWorker2.release();

  // Wait for Worker 1 to wake up and attempt its stale write
  await worker1Execution;

  // 5. Inspect final state
  const finalRecord = repository.getRecord(resourceKey);
  console.log("\n=== FINAL REPOSITORY STATE ===");
  console.log(`Current Balance: $${finalRecord?.data.balance}`);
  console.log(`Committed Token: ${finalRecord?.lastFencingToken}`);
  console.log(`Last Writer:     ${finalRecord?.updatedBy}`);
}

runDistributedFencingSimulation();

Explanation of the Mechanics

1. Token Monotonicity: Even though Worker 1 acquired the lock validly, its token was 1n. When Worker 2 acquired the lock after the lease expired, the coordinator issued token 2n.

2. Atomic Verification at Storage: In PostgreSQL, this pattern translates to a conditional SQL update:

   UPDATE accounts 
   SET balance = 1500, last_fencing_token = 1
   WHERE id = '9021' AND last_fencing_token < 1;

If the query affects 0 rows, the application detects that a newer worker already mutated the record and aborts without corrupting state.

---

Production Selection Framework: Architectural Guidelines

Selecting between Redis, PostgreSQL advisory locks, and etcd should not be driven by convenience or library availability. Architectural selection depends directly on your failure domain, consistency requirements, and infrastructure topography.

                               +----------------------------------+
                               | Do you need the lock for         |
                               | Efficiency or Correctness?       |
                               +----------------------------------+
                                      /                    \
                                     /                      \
                        Efficiency  /                        \  Correctness
                                   /                          \
            +-------------------------------+    +----------------------------------+
            | Redis (Single Node / SET NX)  |    | Can mutations execute inside a   |
            | - Low in-memory latency       |    | single PostgreSQL database?      |
            | - Ephemeral cache warming     |    +----------------------------------+
            | - Non-critical job dedupe     |              /                  \
            +-------------------------------+       Yes   /                    \  No
                                                         /                      \
                     +---------------------------------------+    +----------------------------------+
                     | Are you using PgBouncer in            |    | etcd (Raft Leases + Fencing)     |
                     | Transaction Pooling mode?             |    | - Multi-service coordination     |
                     +---------------------------------------+    | - Distributed leader election    |
                            /                         \           | - Financial balance reconciliation|
                     Yes   /                           \  No      +----------------------------------+
                          /                             \
     +-----------------------------------+    +------------------------------------+
     | Use PostgreSQL Transaction Locks  |    | Use PostgreSQL Advisory Locks      |
     | `pg_advisory_xact_lock()`         |    | - Session-level (`pg_advisory_lock`)|
     | Strictly bound to ACID transaction|    | - Direct backend connection pool   |
     +-----------------------------------+    +------------------------------------+

1. Choose Redis (SET NX) When:

  • The task is locking for efficiency: running an expensive report calculation, throttling duplicate webhooks from an external billing vendor, or pre-rendering static assets.
  • Sub-millisecond latency is required and occasional concurrent execution causes no persistent state corruption.
  • You avoid the complexity of Redlock by treating Redis as an optimistic best-effort mutual exclusion cache, backing downstream state with database-level uniqueness constraints.

2. Choose PostgreSQL Advisory Locks When:

  • Your application architecture is already anchored around PostgreSQL and you want to eliminate the operational burden of maintaining separate coordination clusters.
  • The critical section mutates data within PostgreSQL tables, allowing you to wrap business logic in BEGIN ... SELECT pg_advisory_xact_lock(id) ... COMMIT.
  • You can manage connection counts cleanly without exceeding PostgreSQL connection limits or saturating transaction poolers.

3. Choose etcd When:

  • The workload is locking for correctness: distributed transactions spanning multiple independent datastores, microservice leader election, continuous streaming pipeline partition assignments, or global scheduler orchestration.
  • You require provable Raft consensus where locks survive node crashes without split-brain anomalies.
  • You utilize etcd's create_revision as an end-to-end monotonic fencing token verified at the storage boundary.

When architecting high-throughput distributed transaction engines or multi-region data pipelines, selecting the right coordination primitive is non-negotiable. Teams building high-reliability platforms often partner with Wise Hustlers custom software engineering to design fault-tolerant concurrency controls that prevent silent data corruption under heavy concurrent loads.

---

Frequently Asked Questions

Can Redis Redlock be safely used for financial transactions without fencing tokens?

No. Redis Redlock cannot guarantee mutual exclusion for financial transactions without downstream fencing tokens. As demonstrated in Martin Kleppmann's critique, asynchronous process pauses (such as garbage collection or virtual machine descheduling) and NTP clock shifts can cause Redlock leases to expire while a client is in-flight. If the target database does not validate monotonic fencing tokens, two clients can execute conflicting financial adjustments concurrently.

Why does PgBouncer transaction pooling break session-level PostgreSQL advisory locks?

In transaction pooling mode (pool_mode = transaction), PgBouncer assigns a physical PostgreSQL connection to a client only for the duration of a single transaction block. Once the transaction completes, the connection returns to the shared pool. Session advisory locks (pg_advisory_lock) are bound to the underlying PostgreSQL server process, not the client TCP socket. Consequently, the lock remains active on the database server while the connection is handed to an entirely different client, causing severe lock leaks and phantom deadlocks.

How does etcd prevent the thundering herd problem during lock releases?

etcd prevents thundering herds through its key revision ordering and streaming Watch API. When a client attempts to acquire a lock, etcd assigns the key a monotonically increasing create_revision. Competing clients sort the keys and set a gRPC Watch event exclusively on the single key immediately preceding their own revision. When the current lock holder releases its key, etcd triggers an event only for the single next client in sequence, eliminating broadcast wake-ups.

What is the primary operational difference between `pg_advisory_lock` and `pg_advisory_xact_lock`?

The difference lies in release lifecycle and resource management. pg_advisory_lock is session-scoped; it requires an explicit call to pg_advisory_unlock or remains held until the client terminates its database connection. pg_advisory_xact_lock is transaction-scoped; it releases automatically when the surrounding transaction executes COMMIT or ROLLBACK. For almost all application workflows, pg_advisory_xact_lock is significantly safer because it eliminates orphaned locks caused by crashed application threads.

---

Sources

Related articles