Multi-Node Resilience & Failover

Cachemate is built for production mission-critical environments where cache outages and Redis failures must never take down your upstream services or databases.

---

The Failover Hierarchy

When your application executes a cache operation, Cachemate runs it through a robust failover pipeline:

[ Primary Redis Node ] ──(Network error / Down)──► [ Replica Redis Node 1 ]
                                                          │
                                                    (OOM / Down)
                                                          │
                                                          ▼
                                                [ Replica Redis Node 2 ]
                                                          │
                                                    (All Nodes Down)
                                                          │
                                                          ▼
                                                [ In-Memory LRU Cache ]
                                                    (Zero DB Outage)

---

1. Multi-Node Configuration

You can configure multiple Redis / Valkey instances by passing a list of URLs or descriptors in redisUrls:

import { Client } from 'cachemate';

const client = new Client({
  memoryMaxMb: 256,
  redisUrls: [
    // Primary instance:
    'redis://primary-redis.internal:6379',
    // Read/write failover replica 1:
    'redis://replica-redis-1.internal:6379',
    // Fallback cluster node 2 with custom max memory:
    { url: 'redis://replica-redis-2.internal:6379', maxMemoryMb: 1024 },
  ],
});

---

2. Automatic Node Failover

NodePool tracks the health of all configured nodes.

• When an operation fails with a connection error or timeout, Cachemate tags that node as isDown, logs a diagnostic warning, and advances activeIndex to the next available live node via advanceToNextUp().
• The ongoing operation is immediately re-executed on the newly active node without dropping the request or throwing an exception back to the caller.

---

3. Redis OOM Backoff Circuit Breaker

When a Redis instance reaches its memory limit (e.g. maxmemory exceeded with noeviction policy), standard Redis operations fail with: OOM command not allowed when used memory > 'maxmemory'.

In standard setups, this crashes the cache layer and fills logs with unhandled rejections.

Cachemate catches Redis OOM conditions, raises a specialized NodeOomError, and automatically:

1. Marks the affected Redis node as saturated.
2. Triggers failover to the next replica node or in-process MemoryStore.
3. Protects Redis from write amplification while memory pressure stabilizes.

---

4. In-Memory LRU Fallback

If all configured Redis instances become unreachable (e.g. major cloud provider network split), Cachemate activates its built-in MemoryStore:

• Uses a bounded, high-performance in-process LRU cache (powered by lru-cache).
• Bounded strictly by memoryMaxMb (megabytes converted to bytes with Buffer.byteLength calculation).
• Preserves all caching behavior (cache, flat, rows, and invalidate), allowing your APIs to continue serving traffic at microsecond latency without slamming your primary database.

// Inspect if client is currently operating in in-memory emergency mode:
if (client.core.pool.isMemoryFallback) {
  console.warn('⚠️ Cachemate is running on in-memory LRU fallback!');
}

---

5. Automatic Reconnection & Self-Healing

Cachemate periodically probes down nodes in the background. Once a recovered Redis node successfully accepts TCP connections and completes a handshake, Cachemate automatically promotes it back into the active pool.